Comma.ai Openpilot: Driving with George Hotz and Lex Fridman - A Deep Dive into End-to-End Autonomous Driving

The podcast features Lex Fridman driving with George Hotz, CEO of Comma.ai, demonstrating the Comma Two device running Openpilot. The system provides Level 2 autonomous driving capabilities, primarily excelling on highways and long trips. Hotz emphasizes the company's focus on achieving a truly "end-to-end" self-driving solution, particularly for lateral control, which he considers more subtle and challenging than longitudinal control. While the current longitudinal control often leverages the car's stock Adaptive Cruise Control (ACC), Comma.ai aims to transition to its own end-to-end longitudinal system. A key distinction highlighted is Openpilot's ability to handle complex road conditions, such as curvy and banked roads, and even perform lane changes on roads without clear markings, a feat that some competitor systems like GM's Super Cruise struggle with. Hotz explains that the system's "human-ideal" placement on the road, even in the absence of lane lines, demonstrates its advanced perception. The discussion also touches on the importance of clear user feedback—both auditory and visual—for engagement and disengagement, ensuring a safe and intuitive handoff between the human driver and the autonomous system. Practical insights include the system's utility for reducing driver fatigue on extended journeys, allowing drivers to "sit there and watch." Safety is paramount, with Comma.ai implementing a lower torque limit than competitors like Tesla to prevent "crazy" maneuvers and conducting rigorous "injection tests" where a safety driver attempts to destabilize the system. The company's vision for Openpilot 1.0 includes moving its entire software stack to PyTorch and achieving full end-to-end longitudinal control, which will significantly enhance its utility in city driving scenarios beyond its current highway focus. The broader implications of Comma.ai's work extend to the future of transportation, showcasing how open-source software and consumer-grade hardware can accelerate autonomous driving development. The company's global reach, with millions of miles driven across various countries, underscores the widespread adoption and testing of their technology. The tour of the Comma.ai office reveals a lean, engineering-focused operation, emphasizing continuous testing, hardware production, and a "compute cluster" dedicated to machine learning, all driven by the mission to "solve self-driving cars while delivering shippable intermediaries."

we don't have we don't have red lights and stop signs yet um i don't want to ship it until we have a real uh holistic one until we have an end to n1

super crews actually can't do it um super crews can't do it because they're they're not good enough with roll on the road uh autopilot can do it and we can do it

it's long trips it's um you know you driving from here to la you'll do it without a disengagement almost

it doesn't technically require perfect lane marking no not at all in fact you can do lane changes on roads that don't have lane markings because it's all end to end

lateral there's so much subtlety for example like when a lane splits out should you follow that split should you stay in the center or should you stay to one side or so much subtlety there

our torque limit is so much lower than teslas so you can't can't do anything crazy

without intuition builder without end to end longitudinal it's pretty uh like it's not really useful as a product

notice how there's no line there yet our placement is still almost human ideal

dude just levinson mentioned this one of his interviews he's like so you know we have all these we have the safety driver sit in the car and mark every tiny mistakes it made and then we also had the safety driver sit in the car that was being human driven and mark all the little tiny mistakes it made

solve self-driving cars while delivering shippable intermediaries

we're almost at 30 million open pilot miles wow