To come up to speed on self driving cars, a good start point is the CVPR 2023 Workshop on Autonomous Driving. Here are a selection of the talks at the workshop:
- Kodiak: Robust Localization: From Highways to Offroad
- Chelsea Finn: Benchmarking, Detecting, and Adapting to Distribution Shift
- Challenge – Argoverse
- Common Misconceptions in Autonomous Driving
- Alexandre Alahi: The 7 Foundational Principles Behind Autonomous Mobility
- Chen Wu, head of Perception at Waymo: Progress Towards Scalable Deployment of Autonomous Driving
- Challenge – Waymo Open Dataset
- Ashok Elluswamy: Foundation Models for Autonomy
All talks are interesting, and the range of topics covers the gamut.
I’ve rewatched Ashok Elluswamy’s Foundation Models for Autonomy talk. He is Head of Autopilot Software at Tesla. Pretty impressive! He talks about:
– Occupancy networks, which predict flow of voxels in the future
– Prediction of lanes, and of moving vehicles, trucks, pedestrians, along with full kinematics
– The models used are multi-modal, taking in not just camera video streams but also other inputs like the vehicle’s own kinematics, navigation instructions, and lanes.
– Motion planning using neural networks – doing parallelized tree search over maneuver trajectories.
(Notice key term ‘using’ – does not exclude a rules based motion planner running in parallel)
– Sophisticated auto-labeling pipeline
– Pretty exciting announcement of foundational world vision model – on development 7 months ago, then was running offline, possibly soon online on the car – built generally for cars and robots.
Tesla, and Waymo are these days the industry leaders, with a bunch of research shops and startups doing a lot of other advanced work. One difference between Tesla and Waymo: the latter is publishing research about its self driving cars implementation, and is releasing and maintaining the Waymo dataset.
The papers released by Waymo researchers are, it is said, reflecting the research stage of their process. During implementation, things are further tweaked, and updated. The papers don’t always reflect the exact updates and tweaks that make it in production.
While Tesla releases information through presentations, like the excellent presentation from Ashok at this CVPR Workshop on Autonomy. The presentations reflect the progress made by Tesla – and represent breadcrumbs that researchers and observers use to remap and understand, as much as possible, how self driving and autonomy actually are implemented in Tesla cars and humanoid robots.