Stanford professor Chris Gerdes on autonomous vehicle technology, federal safety policy, and the ethics of self-driving cars
Chris Gerdes, professor at Stanford University and former Chief Innovation Officer at the US Department of Transportation, delivers a lecture at MIT on the intersection of autonomous vehicle technology and federal safety policy.
Summary
Chris Gerdes covers both the engineering and regulatory dimensions of autonomous vehicles, drawing on his work with Stanford's self-driving race car Shelley and his role helping develop the first federal automated vehicle policy in 2016. He explains how the existing federal motor vehicle safety standards were largely designed without automation in mind, creating a regulatory gap that the new voluntary guidance framework attempts to address. Gerdes argues that the trolley problem framing of AI ethics is misleading — engineers should focus on reducing risk through system design rather than programming moral hierarchies. He also makes the case that data sharing across the industry, modeled on aviation's anonymized safety data systems, could dramatically accelerate the safety benefits of autonomous vehicles.
Key Takeaways
FULL TRANSCRIPT
Introduction
Lex Fridman: Today we have Chris Gerdes with us. He's a professor at Stanford University where he studies how to build autonomous cars that perform at or beyond human levels, both on the racetrack and on public roads. That includes a race car that goes 120 miles an hour autonomously on the racetrack, which is awesome. He spent most of 2016 as the Chief Innovation Officer at the United States Department of Transportation and was part of the team that developed a federal automated vehicle policy. He deeply cares about the role that artificial intelligence plays in our society, both from the technology side and the policy perspective. He is now, I guess you could say, a policy wonk, a world-renowned engineer, and I think also a car guy — yes. He told me that he did a Q&A session with a group of third graders last week and answered all of their hard-hitting questions. I encourage you guys to continue on that thread and ask Chris questions after his talk. Please give a warm welcome to Chris.
Background and the Cars
Chris Gerdes: Great, Lex, thanks for that introduction and thanks for having me here to talk to everybody today. This is sort of my first week back in a civilian role — I wrapped up at USDOT last week. So I'm no longer speaking in an official capacity representing the department, although some of the slides are very similar to things I used to present on behalf of the department. As of Friday this was still fairly current, but I am talking in my own capacity here.
I wanted to talk about both the technology side and the policy side of automated vehicles, and in particular how some of the techniques you're learning in this class around deep learning and neural networks place real challenges on regulators and policymakers attempting to ensure vehicle safety.
Just a bit about some of the cars in my background — I am a car guy, and I've had a chance to work on a lot of cool ones. I've actually been working in automated vehicles since 1992. The Lincoln Town Cars in the upper corner are part of an automated highway project I worked on as a PhD student at Berkeley. I then went to freight lidar, heavy trucks, and Daimler-Benz, and worked with suspensions on heavy trucks before coming to Stanford and doing things like building P1 — in the upper right corner — which is an entirely student-built electric steer-by-wire, drive-by-wire vehicle. We've also instrumented vintage race cars, electrified a DeLorean, which I'll show a little bit later, and worked, as Lex mentioned, with Shelley, which is our self-driving Audi TT — an automated race car.
In addition to the Stanford work, I was a co-founder of Peloton Technology, which is a truck platooning firm looking at bringing platooning technology — vehicle-to-vehicle communication that allows for shorter following distances out on the highway — to market.
Shelley: The Self-Driving Race Car
Chris Gerdes: To give you a little bit of a sense of what we've been doing, this is Shelley going around the racetrack at Thunderhill. She can actually go up to about 120 miles an hour or so on that track — it's really just limited by the length of the straight. It's kind of fun to watch from the outside, a little disconcerting occasionally, as you can see there's nobody in the car, although from inside it actually looks pretty chill.
We've been working with Shelley for a while out on the track. She's able to get performance now which exceeds the capability of anybody on the development team. Most of my PhD students have their novice racing license — we make sure that they get that license before going out on the track and testing. Shelley can beat anybody in the research group. She can actually beat the president of the track, David Vadim, and we've had the opportunity to work recently with Marco Andretti, the IndyCar driver who finished sixth this last year in the Indy 500. He's faster, but he's actually only about a second or so faster on a one minute and twenty-five second lap. So we're approaching his performance, and he's actually helping us get there.
The interesting thing about this is that we've approached this problem really from one of physics — force equals mass times acceleration. The car is out there calculating what it needs to do to brake down into the next corner, how much grip it thinks it has, and so forth as it goes around the track. It's not actually a learning approach at its core, although we've added on top a number of algorithms for learning. It turns out that the difference between the car's performance and the human performance — really getting that last little bit of capability out of the tires — is that the best human drivers drive instinctively in a way which is constantly pushing to the limits of the car's capability. If you sort of prejudge what those limits are, you're not going to be quite as fast. So that's one of the things we've actually been working with learning algorithms on — trying to figure out how much friction do I have in this particular corner, and how is that changing as the tires warm up and as the track warms up from the morning to the afternoon.
These are the things that you need to be fast on the racetrack, but they're also the things you need to take into account to be safe in the real world. What we're trying to do with this project is understand how the car can drive at the maximum capability of the friction between the tire and the road. Race car drivers do that to be fast — as they say in racing, if you want to finish first, you have to finish. So it's important that they be fast but also accident-free. We're trying to learn the same things so that on the road, when you may have unknown conditions ahead of you, the car can make the safest maneuver that uses all the friction between the tire and the road to avoid any accident that the car would be physically capable of avoiding. That's our goal.
We've had a lot of fun with Shelley. We've gotten to drive the car up Pikes Peak and on the Bonneville Salt Flats. Shelley actually appeared in an Audi commercial with Zachary Quinto and Leonard Nimoy, and at the end of the commercial they both look at each other and declare it "fascinating." If you're as big a science fiction fan as I am, you realize that once your work has been declared fascinating by two Spocks, there's nowhere to go. So I had to take a stint and try something different in government.
A Year at the US Department of Transportation
Chris Gerdes: I spent the last year as the first Chief Innovation Officer at the US Department of Transportation, which I think honestly was the coolest gig in the federal government. I really didn't have any assigned day-to-day responsibilities, but I got to dive in and help with all manner of really cool projects, including the development of the first federal automated vehicle policy. It was a great opportunity to see things from a different perspective.
What I wanted to do — coming into this as an engineer — is give you a perspective of what it's like from the regulatory side on vehicle safety, how they're thinking about the technologies you're developing, and where that actually leaves some opportunities for engineers to make big contributions to society.
The Current Vehicle Safety System
Chris Gerdes: Let's start with what vehicle safety looks like today. We have a system of federal motor vehicle safety standards — these are rules, minimum performance requirements, and each of them must have an associated objective test so you can tell whether the vehicle meets the requirement or not.
Interestingly, there is no federal agency that tests vehicles before they are sold. We rely in this country on a system of manufacturer self-certification. The government puts these rules out there and manufacturers certify that they can meet them, then put the vehicles on the market. The National Highway Traffic Safety Administration can purchase vehicles and test them to make sure they comply, but we rely on manufacturer self-certification. This is a different system than most of the rest of the world, which has pre-market certification — before you can sell it, the government agency has to say yes, we've checked it and it meets all the requirements. Aviation in this country has that. Aircraft require certification before they can be sold. Cars do not.
Where did that system come from? A quick history lesson: in 1965, Ralph Nader released a book entitled Unsafe at Any Speed. This is often thought of as a book about the Corvair — it's not. The Corvair featured prominently as an example of a design that Nader considered unsafe, but what was very interesting about the book was that he was actually advocating for things like airbags and anti-lock brakes back in 1965. These technologies didn't come along until much later. His argument was that the auto industry had failed — not a failure of engineering, but a failure of imagination.
If you're interested in vehicle safety, I would really recommend you read this book, because it's fascinating. There are quotes from people in the 1960s basically saying that they believed any collision above about forty or forty-five miles an hour was not survivable, and therefore there was no reason for seatbelts, no reason for collapsible steering wheels. In fact, there's a quote from somebody who had made great advances in road safety saying he couldn't conceive of what help a seatbelt would give you beyond firmly bracing yourself with your hands. Those of you who have studied physics know that's patently ridiculous, but there was a common feeling that there was no sense in doing anything about vehicle crashworthiness because once you got above a certain speed it was inherently unsurvivable.
I think it's interesting to look at that today, because if any of us were to be in a collision at around forty miles an hour in a modern automobile, we'd probably expect to walk away. So what this did was lead to a lot of public outcry and ultimately the National Traffic and Motor Vehicle Safety Act in 1966, which established NHTSA and this set of federal motor vehicle safety standards.
The Rulemaking Problem
Chris Gerdes: The process to get a new standard made — a rulemaking process in government — is very time-consuming. Optimistically, the absolute minimum it can possibly take is two years. Realistically, it's more like seven. If you think about going through this process, that's really problematic. Think about what we were talking about with automated vehicles two years ago, or seven years ago. Trying to start seven years ago and make laws that are going to determine how those vehicles operate on the road today — it's crazy. There's really no way to do that.
The other thing is that our system evolved from this sense of a failure of imagination — the government needs to say to industry, "do this, stop slacking off, these are the requirements." But I think it's hard to argue today, with all the advances in automation, that there is any failure of imagination on the part of industry. People are coming up with all sorts of ideas and concepts for new transportation and automation — tech companies, startup companies, large OEMs. There's all sorts of concepts being tested out on the road. It's hard to argue there's still any lack of imagination.
The question is: are things like this legal? Well, from the federal level, there's an interesting report that came out about ten months ago from the folks at Volpe who did a scan and said, what are the things that might prevent you, based on the current federal motor vehicle safety standards, from putting an automated vehicle out on the road? And the answer was honestly not much. If you start with a vehicle that is currently meeting all the standards, because there are no standards that relate specifically to automation, you can certify your vehicle as meeting the federal motor vehicle safety standards. Therefore there's nothing at the federal level that prevents, in general, an automated vehicle from being put on the road.
Now there are a couple of exceptions. There were a few points in there that referenced a driver, and in fact NHTSA gave an interpretation of the rule — which is one of the things they can do, saying we're going to interpret the rules we have rather than make a new rule — and they said that these references to the driver could in fact refer to the AI system. So that is now a policy statement from the department: many of the references to "driver" in the federal motor vehicle safety standards can be replaced with your self-driving AI system and the rules applied accordingly.
So in fact there's very little that prevents you from putting a vehicle out on the road if it meets the current standards. If it's a modern production car that's been automated, federal motor vehicle safety standards don't stop that. However, a lot of the designs I showed — things that wouldn't have a steering wheel or pedals — are actually not compliant, because there are requirements that you have a steering wheel and pedals. These are best practices that evolved in the days when people were not thinking of cars that could drive themselves. These things would require an exemption from NHTSA — a process of saying this vehicle is allowed on the road even though it doesn't meet the current standards because it meets some equivalent — and studying that equivalent can be a bit of a challenge.
The Federal Automated Vehicle Policy
Chris Gerdes: So the question then is: if the federal government is responsible by the Traffic Safety Act for safety on the roads but can't prevent people from putting anything out, what do you do? One approach is to say let's get some federal motor vehicle safety standards out there, but as we already said, that's probably about a seven-year process. And if you were to start setting best practices now, what would that look like?
We've got this challenge: we want to encourage this technology to come out onto the roads and be tested, because that's the way you're going to learn and get real-world data and real-world experience. At the same time, the federal government is responsible for safety on the nation's roads. It can recall things that don't work — if you put your automated system out on the highway and it's deemed to present an unreasonable risk to safety, even if you're an aftermarket manufacturer, the government can tell you to take it off the road. But the question is how can you do better, how can you be proactive?
So we know standards are maybe not the best way of doing that because they're too slow. We'd like to make sure the public is protected but this technology gets tested. The approach taken was the federal automated vehicle policy, which rolled out in September. This was an attempt to say, let's put out a different framework from the federal motor vehicle safety standards — let's put out a system of voluntary guidance. What NHTSA is doing is asking manufacturers to voluntarily follow certain guidance and submit to the agency a letter stating that they have followed a certain safety assessment.
The interesting thing is that this is set up not to tell manufacturers how to do something, but to say: these are the things we want you to address, and we want you to come to us to explain how you've addressed them — with the idea that from this, best practices will emerge. We'll be able to figure out in the future what really is the best way of ensuring some of these safety items.
The 15-Point Safety Assessment
Chris Gerdes: This rolled out in September and consists largely of multiple parts, but I think the most relevant to vehicle design is this 15-point safety assessment. I'd like to talk about a few of these in some more detail.
It starts with this concept of an operational design domain and minimal risk or fallback conditions. Instead of trying to put a taxonomy on here and say your automation system could be an adaptive cruise control that works on the highway, or it could be fully self-driving, or it might be something that operates a low-speed shuttle — the guidance asks the manufacturers to define this themselves. The definition is known as the operational design domain. You tell us where your system is supposed to work. Is it supposed to work on the highway? In restricted areas? Can it work in all weather, or is this something that operates only in daylight hours in the sunshine in South Florida? All of those are fine, but it's incumbent upon the manufacturer or developer to define the operational design domain.
Once you've defined where the system operates, you need to define how you make sure it is only operating in those conditions, how you make sure the system stays there, and what your fallback is in case it doesn't. That fallback can be different depending on the design. If this is a car which is normally human-driven, as you see from the Volvo Drive Me experiment, it might be reasonable to say we're going to ask the human driver to retake control. But clearly if you're going to enable blind passengers or have a vehicle with no steering wheel, you need a different fallback system. Within the guidance, it really allows manufacturers to have a lot of different concepts of what they want their automation to be, so long as they can define where it works, what the fallback is in the event that it doesn't, and how you have educated the consumer about what your technology does and doesn't do.
Further down, you also see validation methods and ethical considerations as aspects that are brought up. Validation methods are really interesting as they apply to AI. There are lots of different ways that you might validate an automated vehicle. You might go out on the test track and run it through a series of standard maneuvers. You may develop a certain number of miles of experience driving in real-world traffic and figure out how the vehicle behaves in a limited environment. There are questions about a test track because you don't have the unknowns that can happen in the real-world environment. But if you test in one real-world environment, you also have a question of whether this is transferable information. If I've driven a certain number of miles in Mountain View, California, does that tell me anything about how the vehicle is likely to behave in Cambridge, Massachusetts? Maybe, maybe not. It's a little bit hard to extrapolate sometimes.
Then there's also the idea of simulation and analysis. If I can record these situations, if I can actually create a virtual environment of the sorts of things I see on the road, maybe I can actually run the vehicle through many, many of these scenarios, perturbed in some way, and actually test the system much more robustly in simulation than I can ever do out on the road.
The guidance is actually neutral on which of these techniques manufacturers take and allows manufacturers to approach it in different ways. Based on conversations, when you think about how companies develop this, they do take all these different approaches. A company like Tesla, which is recording all the data streams from all their vehicles, is basically able to run ideas or technologies silently in their vehicle — they can actually test systems out, get real-world data, and then decide whether or not to make that system active. Companies that don't have that access to data really can't use that sort of development method and may rely much more heavily on simulation or test track experience. The guidance doesn't prescribe any particular blend of these, and in fact it does envision that you might have over-the-air software updates in the future.
Ethics and the Trolley Problem
Chris Gerdes: It is interesting to think about whether you have data-driven approaches — things like artificial neural networks — or whether you actually program in hard and fast rules, because as you start to think about requirements on a system, how do you actually set requirements on a system which has learned its behavior and you don't necessarily know what the internal workings or algorithms look like?
There's another one that comes up, which is the ethical consideration. I'm going to pick on MIT for a moment here. This is an area that I actually did a lot of work on with Stanford, together with some philosophers who joined our group. When people hear "ethical considerations in automated vehicles," it often conjures up the trolley car problem — this classic formulation about a self-driving car heading towards a group of ten people, where it can either plow in and kill those ten people or divert and kill the driver. What do you do?
These are classic questions in philosophy. The trolley car problem is: I have a runaway trolley car and I need to either divert it to another track where it will kill somebody wandering across that track, or the five people on the trolley car are killed. What do I do? Well, as one article points out, before automated vehicles can become widespread, car makers must solve an impossible ethical dilemma of algorithmic morality.
So if all this wasn't hard enough — you understand how tough the technology is to actually program this stuff, then you have to get the regulations right, and now you actually have to solve impossible philosophical questions — well, I don't think that's actually true. I think it's good for engineers to work with philosophers, but not to be so literal about this. This is a question that philosophers can ask, but engineers might ask a number of different questions, like: who's responsible for the brakes on this trolley? Why wasn't there a backup system? Why am I headed into a group of ten people without any capability to stop?
An engineer would in fact have to answer this question but might approach it much differently. If I look at the trolley car problem, I might say: my options are I've got a trolley car which is out of control. First of all, I'd like to have an emergency braking system — let's make sure I have that. Well, there's a chance that could break as well, so if my base braking system goes and my emergency braking system goes, my next option would be to divert it to this side track. Well, knowing that's my option, I should probably put up a fence with a warning sign that says "do not cross — runaway trolley track." Now let's say I've done all of that. The brakes fail, the emergency brakes fail, I have to divert the trolley, and somebody has ignored my sign and crossed over the fence and is now hit by the trolley. Do I feel a little differently about this whole scenario than I did at the beginning, when I was just trying to decide who lived and who died? The solution was made by thinking of it as an engineer trying to reduce risk, not by thinking of levels of morality and who deserves to live or die.
I think this is a very important issue, and the reason it's in the guidance is not to have everybody solve trolley car problems, but to try to think about these larger issues. Ethics is not just about these sorts of situations — which in automated vehicles will actually be addressed much more by engineering principles than by trying to figure out from philosophical merits who deserves to live and die. There are broader issues here. Any time you have concern for human safety: how close do I get to pedestrians? How close do I get to bicycles? How much care should I put into other people in the environment? That's very much an ethical question, and it's an ethical question that manufacturers are actually already addressing today.
If you look at the automatic emergency braking systems that most manufacturers are putting on their vehicles, they will actually use a different algorithm depending upon whether the obstacle in front of it is a vehicle or a human. They're already detecting and making a decision that the impact of this vehicle with a human could be far worse than the impact with another vehicle, and so they're choosing to brake a little bit more heavily in that case. That's actually where these ethical considerations come in. The idea of the guidance is to begin to share and have a discussion openly about how manufacturers are approaching this, with the idea of getting to a best practice where not only the people in automated vehicles but other road users feel that there's an appropriate level of care taken for their well-being.
Legality vs. Safety vs. Mobility
Chris Gerdes: The other area where ethics is important is that we have different objectives as we drive down the road. We have objectives for safety — we'd like to get there. We have objectives for mobility — we'd like to get there probably pretty quickly. And we also have the idea of legality — we'd like to follow the rules. But sometimes these things come into conflict with each other.
Let's say you're driving down the road and there's a van parked where it has absolutely no business parking. You've got a double yellow line. Is it okay to cross? Well, at least in California, there's no exception to the double yellow line representing the lane boundary for a vehicle that's parked where it has no business being parked. So according to the vehicle code, you're supposed to come to a stop. I don't think any of us would. In fact, when you're in California and you're riding through the hills and you come upon a cyclist, virtually every vehicle on the road is deviating across the double yellow line to give extra room to the cyclist. That's also not what you're supposed to do by the vehicle code — you're supposed to stay on your side of the double yellow line but slow to an appropriate speed to pass. So there are behaviors where our desire for mobility or our desire for safety are outweighing our desire for legality.
This becomes a challenge when you think about how to program the self-driving car. Should it be based on the way that humans drive, or should it be based on the way that the legal code tells you to drive? Of course the legal code was never actually anticipating a self-driving car. From a human standpoint, that double yellow line is a great shorthand that says maybe there's something coming up where you don't want to be in the other lane. But if I actually have a car with the sensing capability to make that determination itself, is a double yellow line actually all that meaningful anymore?
Speed limits are another one. If we're out on the highway, it's usually a little bit flexible. Do we give that same flexibility to the automated vehicle, or do we create this wonderful automated vehicle roadblock of vehicles going exactly the speed limit when nobody else around them is? Do we allow them to accelerate a little bit to merge into the flow of traffic? Do we allow vehicles to speed if they could avoid an accident? Is our desire for safety greater than our desire for legality? These are the sort of ethical questions that I think are really important. These are things that need to be talked through, because I believe if we actually have vehicles that follow the law to the letter, nobody will want to ride in them. We need to think about either ways of giving flexibility to the vehicles or to the law, in the sense that vehicles can drive like humans do.
This brings up some really interesting areas with respect to learning and programming. Should our automated vehicles drive like humans and exhibit the same behavior that humans do, or should they drive like robots and execute the way that the law tells them they should? Fixed rules can be one solution. Behavior learned from human drivers could be another. We might have some sort of balance of different objectives that we work out more analytically — how much do we want to obey the double yellow line when there are other things influencing it in the environment?
There are limits to any of these approaches in the extreme. As we found with our self-driving race car, if you're not learning from experience, you're not making use of all the data, and you're not going to do as well. There's no way that you can possibly pre-program an automated vehicle for every scenario it's going to encounter — somehow you have to think about interpolating, somehow you have to think about learning. At the same time, if we simply follow humans, human error is actually the cause or a primary factor in 94 percent of accidents — it's either a lack of judgment or lack of perception on the part of the human. So if we're simply following humans, we're only learning how well humans can do things and we're leaving a lot on the table in terms of the potential of the car.
This is a really interesting discussion that I think will continue both on the development side and the policy side: what is the right balance, what do I want to learn versus what do I want to program, how do I avoid leaving anything on the table?
Marty the DeLorean: Beyond Human Limits
Chris Gerdes: Because it's the point where I've had a bunch of slides with words, I want to give people a little bit of a sense for what you could be leaving on the table if you don't adapt. This is Marty — Marty is a DeLorean that we've been working with in my lab.
DeLoreans are really fantastic cars unless you want to accelerate, brake, or turn. It really didn't do any of those things terribly well. There's no power steering, there's an underpowered engine, and very small brakes. All of these things are fixable. What's nice about the DeLorean is it separates quite nicely — the whole fiberglass tub comes up, you can take out the engine, you can take out the brakes, you can make some modifications to the frame, stiffen the suspension, work with Renova Motors, a startup in Silicon Valley, to put in a new electric drivetrain, and put it all back together. When you do, you come up with a car that's actually pretty fun, and when we've programmed it to drive itself, this is Adam Savage from MythBusters going along for a drive.
What you see is Marty doing something at a level of precision that we're pretty sure no human driver can meet. It's going into a perfect drift, doing a perfect donut around a cone, and then it launches itself through the next gate sideways into the next cone. It shoots through the gate missing those cones and then launches into a tight circle around the next cone. It's doing this using an algorithm similar to orbital mechanics — it's actually orbiting these different points as it sets the trajectory.
The limit on this is tires, as you can see — as it comes around, the tires disintegrate into chunks flying at the camera. But the ability of the car to continue even as the tires heat up, to execute this pretty nice trajectory — you see it going through the gates again and launching into a stable equilibrium, putting the tire tracks right over where they were in the previous run, and then finally ending.
This is the sort of thing that I think is possible as you look at these vehicles. There's a huge potential for these things to not just drive about as well as an average human, but to far exceed human performance in their ability to use all the capabilities of the tires. Maybe that's not the way you want your daily drive to go — although when we first posted some of this video, one of the commenters said "I want this car so I can go into the store to buy donuts while it sits in the parking lot doing donuts." That wasn't a use case I had thought of.
This really illustrates how, if you limit yourself to only thinking about what the tires can do before they get to the saturation of the friction in the road, you're only taking into account one class of trajectories. There's a lot more beyond that which could be very advantageous in some emergency situations. Would it be great if the car had access to that? That's not a capability we're going to get if we only monitor day-to-day driving.
Data Sharing and the Path Forward
Chris Gerdes: One other aspect that came through in the policy, which I think is extremely important as we think about neural networks and learning, is this idea of data sharing. There's a huge potential to accelerate the development of automated vehicles if we can share some information about edge case scenarios in particular.
If you think about trying to train a neural network to handle some extreme situations, that's really much easier if your set of training data contains those extreme situations. If you think about the weird things that can happen out on the road — if you had a database of those, and those comprised your training set, you'd have a head start in terms of being able to get a neural network where you can begin to validate that it would work in these situations.
The question is: is there a way for the ecosystem around self-driving cars to actually share some of this information so that different players can share information about critical situations and make sure that if you learn something, yes, you can make your cars safer, but actually all the cars out on the road get safer? Clearly you need to balance this with other considerations — there are intellectual property concerns of the company, there are privacy concerns of any individuals who might be involved. But it does seem to me that there's a big potential here to think about ways of sharing certain data that can contribute to safety.
This is a discussion that's going to be ongoing, and I think academia can do a lot to help broker this discussion. Because at the first level, people say "data sharing — companies aren't going to share, we're not going to get the information we need." But most of the time people stay in the abstract as opposed to saying: what information would be most helpful? What information is really going to give people confidence in the safety of these cars, let regulators understand how they operate, and at the same time protect the amount of development effort that companies have put in? I think there is a solution here.
If you look at aviation, there's a really good example that already exists. It's known as the ASIAS system. It started with only four airlines that decided to share safety information with each other, and this goes through MITRE, which is a federally funded R&D center. It's actually now up to 40 airlines, and if companies get kicked out of the MITRE project, they try very hard to get back in. This is anonymized data — companies get an assessment of what their safety record is like and can compare it to other airlines in the abstract, but they can't compare it to any identifiable airline. So there's no ranking of this, it's not used for any enforcement techniques. It took people a long time to build up and begin to share that, but now there's a huge amount of trust and they're sharing more and more data, and looking at ways that they can perhaps start to code in things like weather and time of day, which had been removed for anonymization purposes in the original version of the system.
I think there are some good examples out there, and this is something that's very important to think about for automated vehicles. Those of you who are interested in developing these vehicles using techniques that rely on data are going to be an important voice for the importance of data sharing. There's a large role here to make people aware that this actually does have value in the larger ecosystem.
This is something I was able to work on more broadly as well. I was the DOT representative on the National Science and Technology Committee's Subcommittee on Machine Learning and Artificial Intelligence, and this was one of the recommendations that was really pushed forward — AI has tended to make great advances with the availability of good datasets, and in order to make those sorts of advances in transportation, this group is also advocating that those datasets need to be made broadly available.
The Vision Behind the Policy
Chris Gerdes: This is a little bit about the vision behind the automated vehicle policy — what the goal was to really achieve here. The idea of trying to move towards a proactive safety culture, not to necessarily put in regulations prematurely and try to set standards — honestly, we don't know the best way to develop automated vehicles — but to allow the government to get involved in discussions with manufacturers early and be comfortable with what's going out on the roadway, and actually to help the US continue to play a leading role in this. Obviously if vehicles are going to be banned from the roads, it would be very difficult for the country to continue to be a place where people could test and develop this technology.
The belief is that there can be an acceleration of the safety benefits through data sharing — so each car doesn't have to encounter all the weird situations itself, but can learn from what other vehicles experience. The idea is that this is meant to be an evolving framework. It comes out as guidance, it generates conversations, it generates best practices which can eventually evolve into standards and law. There's a huge opportunity here because the belief isn't that NHTSA will be doing all of the development of these best practices, but that they'll really evolve from what companies do and what all of us at universities are able to do — to generate ways to solve these problems in creative manners, ways to keep the innovation going but ensure that we have safety.
As you start to think about all of the AI systems that you're developing and you start to flip around and think about how a regulator is going to get comfortable that it's not going to do something weird — these are great research questions, great practical questions, and things that will need to be worked out going forward. I leave you with that as a challenge: think as you take this course not only about the technology that you're learning, but how do you communicate that to other people, and where are the gaps that need to be filled? I think you'll find some great opportunities for research, startup companies, and ultimately work with policy and government.
Thanks for the opportunity to talk to all of you. I want to stop there because probably the things that you want to talk about are more interesting than the things I wanted to talk about, so I'm happy to take questions.
Q&A
Audience member: Accidents are part of our economy. The excess rates are extremely low. Do you think some of these safety requirements may roll back?
Chris Gerdes: That's a great question. The question was whether, in the future when you have all vehicles automated, would we be able to actually roll back things like airbags and seatbelts and other passive safety devices in vehicles? I believe that we will. In fact, one of the things that I think is most extraordinary — if you think about this from a sustainability standpoint — when you look at the average mass of vehicles and average occupancy of vehicles in the US, with passenger cars we're using maybe about ninety percent of the energy to move the vehicle as opposed to moving the people inside. One of the reasons for that is crashworthiness standards, which are great because that's what's enabled us to survive crashes at forty miles an hour. But if we do have vehicles that are not going to crash, or if they are going to have certain modes which might be designed with very carefully designed crush areas or things like this, we could potentially take a lot of that mass out — particularly if these are low-speed vehicles designed only for the urban environment, not going to crash because they're going to drive somewhat conservatively or in some ways separated from pedestrians. Then I think you can get a lot of the mass out, and you start to actually have transportation options which, from an environmental standpoint, are comparable to cycling. I think that's actually a really good goal to strive for, although we either have to limit the environment or think in the far future with some of those techniques.
Audience member: What are you doing with Shelley? Is your mission really just to drive as fast as possible and faster than a human, or are you trying to learn something you can apply to other automated vehicles?
Chris Gerdes: It really is a desire to learn for the development of other automated vehicles. We've often said that at the point where the difference between Shelley's performance and the human driver starts to be really mundane things like our shift pattern or something which isn't applicable, we kind of lose interest. However, up to this point every insight that we've gotten from Shelley has been directly transferable. We've programmed the car to do some emergency lane changes in situations where you don't have enough room to brake, and we've actually been demonstrating in some cases that the car can do this much faster than even an expert human's response.
From the bigger picture, what's really fascinating is that we originally started out with this idea of let's find the best path around the track and track it as close as we can. But in fact, when you look at human race car drivers, what they're doing is actually very different — they're pushing the car to the limits and then seeing what paths that opens up to them. It flips the problem a bit on its head in a way that I think is actually very applicable for developing safety systems out on the road, but it's not a way that people have looked at it, to the best of my knowledge, up to this point. That's really what we're hoping — that the inspiration in trying to reproduce human performance leads us to better safety algorithms. So far that's been the case, and when that ceases to be the case, I think we are definitely much less interested.
Audience member: Who is liable if there's an accident in an automated vehicle?
Chris Gerdes: Liability is a good question. Who is liable for an accident in an automated vehicle? On the one hand, that's kind of an open question. On the other hand, we do have a court system, and whenever there are new technologies, these things are generally figured out in the courts. It can be different from state to state, so this is one aspect where some discussions — so that manufacturers aren't subject to different conditions in different states — would be helpful.
The way it works now is that it's usually not binary. We have in the US a sense of joint and several liability, so you can actually assign different portions of responsibility to different players. You have had companies like Volvo and in fact Google make statements that if their vehicles are involved in accidents, they would expect to be liable for it. People have often talked about needing something really new for liability, but I'm not sure that's the case. We do have a court system that can ultimately figure out who is liable with new technologies, and we have some manufacturers that are starting to make statements about assuming product liability.
The one thing that really could be helpful is perhaps some harmonization, because right now insurance is set state by state, and the rules in one state as to who's at fault for an accident may be very different in another state.
Audience member: What if companies, as they send in the safety letters, are using criteria to set safety that may not be broadly acceptable to the public — where the public would like these vehicles to have greater safety?
Chris Gerdes: The nice thing about this process is that we would know that — we would have a sense that companies are developing with certain measures of safety in mind, and there could actually be a discussion as to whether that is setting an acceptable level. It's a difficult question because it's not clear that people really know what an acceptable level is. Does it have to be safer than humans drive now? My personal feeling — I would say yes. Does it have to be much, much safer? Well, that's hard to say. You start to then get into the situation of: we're comfortable to a certain extent with our existing legal system and with the fact that humans could cause errors that have fatal consequences. Do we feel the same way about machines? We tend to think machines really should have a higher level of perfection.
We may as a society be less tolerant. People will often say that so long as the overall national figures go down, that would be good. But that's really not going to matter much to the families who are impacted by an automated vehicle, particularly if it's a scenario with very bad optics. What do I mean by that? If you think about the failures of mechanical systems — because they're different than the failures of human beings — they can often look really bad. If you think about a vehicle that doesn't detect something and then just continues to plow ahead, visually that's really striking, and that's the sort of thing that would get replayed and be in people's consciousness and raise some fears. That's an issue that's going to have to be sorted out.
Audience member: What's being done from a global standpoint to share ideas, share research, and work through some of these things, particularly on the policy side?
Chris Gerdes: Most of the auto manufacturers are global corporations, so a lot of the research in this is done in very different parts of the world. Renault-Nissan, for instance, is doing a lot in Silicon Valley, in Europe, and in Japan. One of the cool things I got to do as part of my role was to go with the Secretary of Transportation to the G7 transportation ministers meeting in Japan and address the ministers about the US policy on automated vehicles. One of the parts of that discussion was that the US has a very different set of rules — we have this manufacturer self-certification as opposed to pre-market certification. But testing is something that has to be done regardless, whether it's done by a manufacturer or, for instance in Germany, by the TÜV and other agencies responsible for road safety. The idea is maybe we should be sharing best practices on testing, so we have a set of standard tests and manufacturers across the globe could test to a certain set of standards that might be translated differently according to the policy and regulatory environments in different countries. That was part of the idea we advanced at the G7, and it seemed to kick off really well.
Audience member: How did you end up in the government role? Was it a conscious decision?
Chris Gerdes: I never had a conscious decision on this. I actually got a call from the White House one day — I got this email saying "I'm reaching out from the White House, please give me a call back." So of course I called back immediately. Pam Coleman was on the other end of the line, and she said, "I love doing that — when you're calling from the White House, everybody returns your call." She said: here's the situation, we're looking at a lot of these areas in the Department of Transportation that seem to hit upon your areas of expertise, and we want to talk with you in some way. The holy grail would be for you to come out and work in DC for a while.
Then I got a call from the Department of Transportation and they said, "Well, we know you wouldn't want to come out to DC for a while." And I thought, oh my god, try me. Could I do cool stuff and could I make an impact? Then I met with the Secretary of Transportation out in San Francisco, and he assured me: you would be very surprised at how much of an impact you could have. And this ended up being really true. A lot of times this stuff moves quickly, and people who are involved in policymaking may or may not have a technical background in this — they may have come through the campaign and ended up in political roles. Yet the folks that I worked with were really trying to get good information and make good decisions. I just kept getting called in for advice on all sorts of things, and I found that people actually really wanted to have that technical information and then used it.
So that's the way it happened. It seemed like an opportunity to take things that I've worked on — automated vehicles since 1992 — and then to be part of this policy development, which went really quickly. It was a one-page outline when I arrived in February, and then in September it rolled out. Along the way there was all sorts of editing and negotiations with the White House and other agencies — a fascinating process. I kind of fell into this, but I'm emerging as a policy wonk because it was a very fun experience.
Audience member: You have a lot of companies that have somewhat of a monopoly on a lot of data — Google has so much more data available. How do you incentivize companies to actually share their data when they have an awful lot invested in gathering and processing that data?
Chris Gerdes: I think the answer is to start small and to try to say: are there certain high-value things that could make the public comfortable, make policymakers comfortable, that really aren't going to be a burden on the company? One of the things from the Peloton standpoint that was bounced around at one point — our trucks use vehicle-to-vehicle communications as part of their link. When you do that, you discover that there are actually an awful lot of places where that drops out, because cell phone towers, which are not supposed to be broadcasting on that frequency, seem to create an awful lot of interference. Well, that can be very interesting from a public policy perspective — where are these incursions in that frequency range? That, for instance, might be a very useful piece of information to share with policymakers that wouldn't be any real proprietary issue to share from the company's perspective.
So I think the trick is to start small and find the high-value data where there isn't a big issue of sharing. If you go to Google and say, "All right, Google, what will it take for you to share all of the data you're acquiring from your entire self-driving car program" — Waymo now — I think that would be a very big number, and I don't think that's the starting point. You start with: what is the high-value data that's of high value for the public policy sense and really minimal hassle to the companies?
Audience member: Is there any effort underway for sharing map data, some of the edge case accident data, simulation capabilities, and things like that?
Chris Gerdes: This is one of the next steps that MIT outlined in the policy, and there are people at MIT actually working on taking some of these next steps, again in a sort of pilot or prototype mode. That's something that's currently being worked on in the department. You could probably expect to hear more from that in the not-so-distant future.
Audience member: Testing in urban and rural environments are very different. Should the government actually come up with a standard set of data that all companies have to attest to?
Chris Gerdes: One of the reasons that the policy was designed the way it was is to make sure we have this concept of operational design domain. If the only area that I've mapped and the only area that I want to drive is in a campus environment or in one quarter square mile, then the idea is that we would like the companies to explain how they handle the eventualities in that one quarter square mile. They should really have no reason to handle other situations, because their vehicle won't encounter that, so long as it's been designed to stay within its operational design domain.
In the short term, what you see is people often looking at hyperlocal solutions or the low-hanging fruit for a lot of automation. Even if you think about offering mobility as a service — if I'm going to offer an automated taxi, I'm probably going to do that in a limited environment to start with. If I'm only doing this in Cambridge, does it really matter if I can drive in Mountain View or not? The idea is to start with the definition of the operational design domain with a dataset that is appropriate for that domain. Then as people's design domains start to expand nationwide, the idea of common datasets starts to be interesting.
Although there is a sense that no finite dataset is really going to capture every eventuality. People will be able to design to the test in some ways. Is that sufficient? I think it'll make people feel better, but I personally wonder how much value there is. With test track testing, I could think of twenty different tests that automated vehicles will have to pass, and people will design ways to pass all twenty of those tests. It may make some people more comfortable, but it doesn't make me all that much more comfortable that they'd be able to handle a real-world situation.
Audience member: Could you make an open-source car under the guidance provided by USDOT?
Chris Gerdes: The question would be: from a practical standpoint, you're supposed to submit a safety assessment letter which is supposed to be signed by somebody responsible for that. So an issue if you were to open source would be: do I use this module, and who is actually signing off on what I feel comfortable signing off on? I'm not a lawyer, but I would think there would be nothing that would prevent that if you had a development team that was doing that and people who were willing to sign off on whatever version of the software was actually used in an open-source car.
The guidance does apply to universities or to other groups that would be putting a car out on the road, and I think if you look through the fifteen points, they're not really meant to be overly restrictive. I would argue that pretty much any group that is going to put real people at risk by putting an automated vehicle out on the road should really have thought through these things. I don't think it's a terribly high burden to meet. I think it would be meetable by a group — it's just a question of, from the open-source sense, how do you trace who's responsible and who's signing off on that.
Lex Fridman: I think we gave those third graders a run for their money. Thank you so much. Let's give Chris a big hand.