Back to top

Berkeley Drives Deep Into Automotive Perception

Driving the next generation of deep automotive perception research, UC Berkeley Partners for Advanced Transportation Technology (PATH) recently launched the Berkeley DeepDrive (BDD) Industry Consortium to investigate state-of-the-art technologies merging computer vision and machine learning for automotive applications. 

Dramatic progress has been made in deep learning for computer vision research, and the impact of these advances on automotive perception will be significant both for academia and industry,” says PATH Faculty Director Trevor Darrell, who leads BDD. “This is a very exciting new venture, and we look forward to new advances from our faculty researchers.”
 
BDD hosted a kickoff event March 21, bringing 22 principal investigators from PATH, the Electrical Engineering and Computer Science Department, Center for Information Technology Research in the Interest of Society (CITRIS) robotics, and the Berkeley Vision and Learning Center (BVLC) and the nine industry partners sponsoring the project: Audi/VWofA, Bosch, Ford, Toyota, Samsung Nvidia, Panasonic, Honda and Qualcomm to begin the conversation of guiding the next generation of research on deep automotive perception through the integration of  vision and vehicles.
 
“Automotive electronics is a promising new area for Samsung,” says Schuyler Cullen, Senior Director Vision Systems, Strategy and Innovation Center, adding that Samsung was among the first sponsors of the Berkeley Vision and Learning Center. “We are excited to work with this premier group of faculty and students on the next generation of algorithms, to enable new uses for our next generation of cameras and processors.”
 
The event focused on individual discussion of selected proposals by sponsors, a first overview of the BDD effort, discussion between PIs regarding cross pollination, synergistic efforts and data collection, demonstrations, getting to know the players, and planning for the future.
 
“Developing self-driving cars requires an end-to-end system with supercomputers in the cloud and a mobile supercomputer the car,” says Sanford Russell, Senior Director, Autonomous Driving Ecosystem at NVIDIA. “Our collaboration with BDD on artificial intelligence and deep learning during both training and actual driving will move us all closer to a new generation of vehicles.:
 
As a company who operates on both sides of the market — training modules and data collection and processing, Russell says this partnership is very exciting.
 
Researchers were excited to talk about their proposals. Berkeley Associate Professor of Electrical Engineering and Computer Science Alexei Efros is working on “Unsupervised Visual Representation Learning,” where computers do not do what they are told, but discovering what it is that needs to pay attention to from the data it is given, essentially taking the human out of the loop.
 
“It’s all fantastic, fun problem solving in the real world,” says Efros. “This is all happening now, and we can work with the companies who can use this is real time. I am happy to go where the data is.”
 
Berkeley Associate Professor of Electrical Engineering and Computer Science Pieter Abbeel is working on the AI to enable robots to deal with the wide variability encountered in the real world.  Per it being impractical to program in all possible situations ahead of time, his research is on reinforcement learning, where robots learn from their own trial and error, and on apprenticeship learning, where robots
learn from demonstrations.  His projects are “Benchmarks and Leaderboard for Deep Reinforcement Learning” and “Deep Learning for Tracking."

 “With the leaderboard we will be closely assessing the latest advances in deep reinforcement learning to allow for better understanding of the relative merits of the many algorithms currently being developed,” says Abbeel.  “Working with the private sector will provide us with insights into some of their practical challenges, as
well as rich data-sets that'll allow us to stress-test our machine learning approaches.”
 
Proposed by UC Berkeley faculty and researchers and approved by a BDD advisory of faculty and sponsor representatives, other projects include “Learning Deep Models Securely on Sensitive Imaging Data with Cryptographic Guarantees” and “Cross-modal Transfer Learning.” View a complete list on BDD’s website.
 
Research, implementation, and demonstrations will focus on several themes, including: low power and embedded deep learning architecture development; efficient pedestrian detection; pedestrian intent detection; Deep Driving Control policies; and scene classification and scene affordance estimation.
 
“This research makes cars potentially safer and more agile,” says Darrell.  “When computers can recognize their surroundings, it makes them more perceptive and able to react properly to other cars, pedestrians, and objects in their environment, and be robust to unexpected situations.”
 
In addition to early access to BDD developments, private industry partners will receive non-exclusive royalty-free commercial use rights for most of the software developed through the center. 
 
Ken Goldberg and Thomas West serve as associate director.