Tesla AI Day

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

Jason Zhang
11 min readOct 18, 2021


From Tesla AI Day

Almost a month ago Tesla hosted Tesla AI Day. In this event, Tesla introduced AI & autopilot completely and in detail for the first time.

As an AI practitioner, especially if you focus on the autonomous driving domain, you should study the first part of Tesla AI Day. A few weeks after the event, I reviewed the video “frame by frame”, searched, downloaded, read all the papers involved in the video, and took a lot of notes. Gradually, I outlined the architecture of Tesla’s FSD.

Next, I will try to explore how Tesla fulfilled its promise of artificial intelligence & autopilot from the perspective of a software engineer.

Before starting, please think about a question with me. If you act as Sr. Director of Tesla AI and lead AI Team, how will you achieve autonomous driving?

Cameras, Lidars, Machine Learning, Neural Network, Maps, HD Maps, Papers, Labels, Training, Testing, DataSets, Planning, Security, Chips, CPUs, GPUs, Mass Data Traning, ethics of AI…, all these things suddenly flooded my brain. The conclusion is that this is a mission impossible for me.

Let’s take a look at Tesla’s solution.

  1. How Do We Make A Car Autonomous?
  2. How Do We generate training data?
  3. How Do we run it in the car?
  4. How Do we iterate quickly?

In AI Day, Andrej Karpathy, the Sr. Director of Tesla AI, and his colleagues, Ashok Elluswamy, Milan Kovac, showed us their solutions around these four questions.

How Do We Make A Car Autonomous?

Basic Capacity: Vision

First look at the clip below, this is the final result of Tesla Vision in the current version. The 8 cameras(Left) around vehicle generate 3-Dimensional “Vector Space” (Right)through Neural Networks, which represents everything you need for driving, such as lines, edges, curbs, traffic signs, traffic lights, cars; and positions, orientations, depth, velocities of cars.