What can urban planning learn from California’s disengagement report 2017?

June 25, 2018 - Jonathan Fetka

Companies testing autonomous vehicles in California have to annually submit a disengagement report to Califonia´s Department of Motor Vehicles. What can urban planning learn from the provided data? What can we learn about the capabilities of the technology so far?

The global context of testing selfdriving cars

Across the globe, the potentials and hazards of autonomous driving are interpreted differently, depending on diverse local challenges and contexts. While in Arizona or in the UK autonomous driving is seen as a vital and major boost for the economy, other countries and municipalities, such as Singapore, Japan or Sweden, follow a more concrete, problem-solving approach to tackle specific societal or infrastructural challenges in their cities. In the research project AVENUE21, these differences are vital to understanding the numerous ways in which autonomous driving could affect spatial changes in cities and are therefore the basis for our research focus on relevant pioneering regions eager to implement autonomous driving. Part of the list of pioneering regions is the San Francisco Bay Area. On the one hand due to its unique technological and economic landscape, on the other hand due to the interesting regulatory framework provided by the state of California. The following review examines Californian disengagement reports, as an example of local regulatory frameworks deployed for “transportation network companies”. Those reports are the result of the mandatory safety-driver retaking control of the selfdriving car after a malfunction or a general feeling of unsafety. This reviews aim is to gain insight on the physical environment in moments of disengagement and thereby assess spatial challenges of AV deployment.

Between laissez-faire and regulation

The need to test and enhance both software and hardware for automated vehicles, the IT- and automobile industries have pressured local governments in recent years to establish the necessary regulation framework. California stands out from other regions in terms of its unique approach. When planning to test an autonomous vehicle on public streets in California today, companies have to undergo a licencing procedure and fulfil an extensive set of regulations. This has caused the state of Arizona to take an opposing stance in 2015 and allow testing with hardly any regulations at all, in order to lure tech-companies from strict California to easy-going Arizona. Thus, companies found unconfined testing conditions. Early on, however, critics addressed the dangers of testing immature technology on public streets and the risk of compromised credibility this could bring to the entire industry. Following the fatal crash of an autonomous Volvo, which was running on the software of Uber, this exactly happened. Suddenly, levels of trust in autonomous vehicles are in decline. Consequently, a formerly optimistic Arizona now withdrew from its plans of allowing completely driverless testing and Uber has ended its testing operations in the state. The developments illustrate that, at the current state of the technology, regulations such as in California do have a reason, as they set up a framework, which provides basic safety hurdles and thereby gives an overview of the current state-of-the-art of the technology.

California’s unique regulatory framework

Since 2014, California law requires companies, which are eager to test their technology on California’s public roads, to submit an annual disengagement report, which summarizes all test cases and reasons for disengagement. Article 3.7. – Autonomous vehicles of the “California Code of Regulations”, defines the rules for testing autonomous vehicles. Furthermore, article 3.8. explains how they should be implemented on public streets. For the following review, disengagement reports of 2017 and the recently released supplementary information were examined. §227.46. of article 3.7. defines the rules for “Reporting Disengagement of Autonomous Mode”:

(3) The annual report shall summarize disengagements for each month as follows:

(A) The total number of autonomous mode disengagements and the circumstances or
testing conditions at the time of disengagements including:

(i) The location: interstate, freeway, highway, rural road, street, or parking facility.

(ii) A description of the facts causing the disengagements, including:
weather conditions, road surface conditions, construction, emergencies, accidents or
collisions, and whether the disengagement was the result of a planned test of
autonomous technology. (California Code of Regulations Article 3.7., S.11)

Learnings from the 2017 Autonomous Vehicle Disengagement Reports

In 2017, twenty Tech-Companies submitted their disengagement report (2015: 7, 2016: 11). Not all of the twenty reports were analyzed, as some companies did not test in 2017 or submitted inadequate reports in regards to spatial aspects. The analyzed companies are Baidu, APTIV, Drive.ai, Mercedes, Nissan, General Motors, TeleNav, Valeo, Waymo & Zoox. Those companies drove around 500.000 miles in autonomous mode in 2017 (see table 1). To draw specific conclusions from the conditions of the testing environment is hardly possible, as information on spatial circumstances is only revealed in the case of accident reports. However, the reports show a lower share of driven miles in urban environments than on highways or interstates. Due to the high complexity of urban street environments, this does not strike as surprising.

Article 3.7. of the „California Code of Regulations“ defines the classifications for disengagements, a useful tool that’s hardly applied. In the literature analyzing disengagements, two types of disengagements are identified, PDE (passive disengagement) and ADE (active disengagement). They are classified as follows:

PDE: Hardware, Weather, Software, Road surface
ADE: Hardware, Software, Emergencies, Precautionary Intervention

How to reclassify?

While these classifications are a good tool to identify problem fields as a developer, they are not transferable to the needs of urban planning and architecture as they ignore the manifested background for disengagement. Therefore, it is necessary to reclassify the disengagements in order to gain an overview about the spatial hurdles for a successful implementation of autonomous vehicles on urban roads. Concerning the research agenda of AVENUE21, which aims to identify fields of tension between major urban development goals and autonomous driving, it is necessary to identify spatial challenges on the road to a successful implementation of autonomous driving. As the analyzed companies provided the Department of Motor vehicles in California (DMV) with more detailed reports, it was possible to reclassify the disengagements according to reasons of disengagements. In a first step, five sources of errors could be identified (see fig. 1):

Traffic: Situations which could be avoided in different spatial settings
- Unforeseen manoeuvres from other traffic participants
- Complex traffic situations, etc.
Misbehaviour of the car: Situations, in which the car makes the wrong decisions
- Wrong manoeuvres
- Errors in distance keeping
- Misidentification of objects, ….
Infrastructure: Situations which could be avoided with different infrastructural measures
- Construction sites
- Complex crossing situations
- Misdetection of lanes,…
Hardware: Disengagements due to hardware-related errors
- Loss of control, steering, signals,…
- Localisation & GPS issues,…
Software: Errors in the function of Software
- Software crashes

Spatial relevance

These summed up and categorized sources of errors still lack spatial relevance. How can such mistakes be classified according to spatial dimensions? For this reason, we examine the car in its “natural” environment. The car moves on streets alongside other traffic participants and obstacles. This means that the car itself has a sphere of basic functions; the driving process itself, it moves on certain infrastructure, such as streets or roadways, and alongside other traffic participants, which have their own movement patterns and behaviour. Therefore, the function of a successfully moving car can be roughly divided into these three spheres. An autonomous car in an urban environment as it is present now can only be successful when reducing errors in all three spheres. Figure 2 shows, that the biggest challenges lie in percepting the surrounding traffic and in the functioning of the software.

Conclusions

This short review shows, that, besides technological advancement, there are still many obstacles to overcome in order to successfully implement autonomous driving on public streets. Especially the interaction with other traffic participants proves as a scource of errors and often results in dangerous situations. In an urban planning perspective, it is important to identify major scources of errors in order to point out action areas, in which urban planning can proactively push automated and connected driving to contributing to urban development goals. This review aims to show, that implementing automated and connected mobility is not merely a technological topic, its a diversified one.

Seven aspects to consider for further research

- California has a unique and theoretically precise tool to examine the state of the art of selfdriving technology. In order to draw conclusions for (future) urban road spaces, it would be necessary to receive more accurate specifications concerning the physical environment. Considering the possible consequences an exhaustive introduction of selfdriving cars could bring onto the urban environment, a modification of the regulation is worth consideration.

- Until now, the autonomous interaction with surrounding traffic is the greatest source of errors. The presence of other traffic participants and road users as well as the recognition of their behaviour are responsible for almost 50% of all disengagements. This could fuel the debate on separate infrastructure for selfdriving cars.

- Statements on the exact location (urban street/crossing/interstate/highway…) of disengagements can not be made yet.

- Miles per disengagement do not necessarily represent the state-of-the-art, as sometimes a low mile-per-disengagement ratio is the result of a planned failure or the test of a new technological approach.

- When talking about disengagements one has to ask when the ratio will be high enough? When will the miles per disengagements be so high that autonomous vehicles outperform human drivers? In the United States an accident occurs approx. each 160.000 miles. Waymo has reached 30.000 miles with only one disengagement in November 2017. Should the number once exceed that of human drivers, will it be high enough for society to trust the technology?

- European cities generally show a higher complexity in their traffic systems and urban environments and are only comparable to American cities on an abstract level. Therefore applications, use-cases and timelines of introduction will differ.

- Therefore, infrastructural issues could in most cases be resolved not only through adaptations in the built environment, but with technology also. Thus, it yet has to be decided if and how either planning or Tech-Companies will solve current issues.

[read more=”Literature ” less=”Literature “]

Chen, Lv et.al. (2018) Analysis of Autopilot Disengagements Occurring During Autonomous Vehicle Testing. [online – https://bit.ly/2kg9k3y]

DMV (2017) Autonomous Vehicle Disengagement Reports 2017. [online – https://bit.ly/2E4m60j]

DMV (2017) Article 3.7. – Autonomous Vehicles [online – https://bit.ly/2GKB3Ch]

DMV (2018) Autonomous Vehicle Disengagement Reports 2017 – Supplemental Information. [online – https://bit.ly/2rchFsB]

Favaro, Francesca et.al. (2017) Examining accident reports involving autonomous vehicles in California. [online – https://bit.ly/2Lmlhkf]

[/read]