Tesla Autopilot hardware

Tesla Autopilot, an advanced driver-assistance system ("ADAS") for Tesla vehicles, uses a suite of sensors and an onboard computer. It has undergone several hardware changes and versions since 2014, most notably moving to an all-camera-based system by 2023, in contrast with ADAS from other companies, which include radar and sometimes lidar sensors.

Initially, the ADAS used a combination of cameras capturing the visual spectrum, forward-facing radar, ultrasonic proximity sensors, and a Mobileye EyeQ3 computer as Hardware 1, fitted to Model S vehicles starting in October 2014. After Mobileye ended its partnership with Tesla in 2016, Tesla began shipping cars equipped with an Nvidia Drive PX 2 computer and an increased number of cameras as Hardware 2. In 2019, Tesla shifted to a computer using a custom "FSD Chip" designed by Tesla, branded as Hardware 3. Starting in 2021, Tesla stopped installing the radar sensor in new vehicles, and the ADAS was updated to drop radar support. In 2022, Tesla announced it also would drop support for the ultrasonic sensors, moving the ADAS to an all-visual system. The most recent sensor and computer implementation is Hardware 4, which began shipping in January 2023.

Physical versions

Hardware 1

Vehicles manufactured after late September 2014 are equipped with a single camera mounted at the top of the windshield, forward looking radar[1][2] in the lower grille, and 12 ultrasonic acoustic location sensors in the front and rear bumpers that provide a 360-degree view around the car.[3] The computer is the Mobileye EyeQ3;[4] as implemented, this chip is built on a 40 nm process with a TDP of 2.5 W and a clock speed of 500 MHz.[5]

Feature history

Hardware

References

Chris Isidore. Elon Musk says Autopilot upgrade could be coming CNN, July 18, 2016, retrieved July 19, 2016^
Elon Musk. Twitter July 17, 2016, retrieved July 19, 2016^
LeBeau, Phil. Tesla rolls out autopilot technology

Hardware 2

HW2, included in vehicles manufactured after October 2016, includes an Nvidia Drive PX 2[15] computer.[16][17] Tesla claimed that the hardware was capable of processing 200 frames per second.[18] Elon Musk called HW2 "basically a supercomputer in a car", referring to its ability to perform up to 12 trillion floating point operations per second (TFlops), a performance similar to a $700 GTX 1080 Ti[19] consumer graphics card at the time.[20] The Autopilot computer hardware, housed just above the glovebox, is replaceable to allow for future upgrades.[21][22] Tesla claimed the HW2 suite of sensors and computation provided the necessary equipment to allow FSD at SAE level 5.[23]

The hardware includes eight cameras covering an aggregate view of 360° around the car and 12 ultrasonic sensors, in addition to forward-facing radar with enhanced processing capabilities.[23] The radar is able to observe beneath and ahead of the vehicle in front of the Tesla;[24] the radar can see vehicles through heavy rain, fog or dust.[25] The eight cameras are mounted in various locations around the vehicle: three forward-facing, next to the central rearview mirror mounted on the windshield; two front/side cameras, one each mounted in the left and right B-pillars; two rear/side cameras, mounted in the left and right front fender turn-signal repeaters; and one rear camera, above the license plate.[20]

When "Enhanced Autopilot" was enabled in February 2017 by the v8.0 (17.5.36) software update, testing showed the system was limited to using one of the eight onboard cameras—the main forward-facing camera.[26] The v8.1 software update released a month later enabled a second camera, the narrow-angle forward-facing camera.[27] With all eight cameras enabled, data extracted from Autopilot in debugging mode showed the cameras provide a black-and-white feed to the computer, possibly to improve image processing speed.[28]

The Tesla Model 3, introduced in 2017, and related Model Y, introduced in 2019, are equipped with an additional driver-facing in-cabin camera. This was disabled at launch and was intended to monitor the cabin remotely while the owner was operating the vehicle as an autonomous robotaxi,[29] but was activated in May 2021 to monitor driver attentiveness while using Autopilot in vehicles without radar sensors.[30]

Hardware 2.5

In August 2017, Tesla announced that HW2.5 included a secondary processor node to provide more computing power and additional wiring redundancy to slightly improve reliability; it also enabled dashcam and "sentry mode" capabilities.[31] During this time, the supplier for the system's radar components was changed from Bosch to Continental,[32] using the ARS4-B unit.[33]

Tesla will upgrade HW2.5 vehicles to HW3 at no extra cost if the owner purchases Full Self-Driving.

Hardware 3

According to Tesla's director of Artificial Intelligence (AI) Andrej Karpathy, Tesla had, as of Q3 2018, trained large neural networks but they could not be deployed to Tesla vehicles built up to that time due to insufficient computational resources. HW3 was designed to run these neural networks.[34] Overall, Tesla claims HW3 has 2.5× improved performance over HW2.5, with 1.25× higher power and 0.2× lower cost.[35]

HW3 is based on a custom Tesla-designed system on a chip called "FSD Chip",[36] fabricated using a 14 nm process by Samsung.[37] Jim Keller and Pete Bannon, among other architects, have led the project since February 2016.[38] FSD Chip features twelve ARM Cortex-A72 CPUs operating at 2.6 GHz, two systolic arrays (not unlike the approach of TPU)[39] operating at 2 GHz and a Mali GPU operating at 1 GHz.[35] Tesla claimed that FSD Chip processes images at 2,300 frames per second (fps), which is a 21× improvement over the 110 fps image processing capability of HW2.5.[38][40] The firm described FSD Chip as a "neural network accelerator" custom-designed for Tesla AI processing.[18] Each of the two systolic arrays on a single FSD Chip are capable of 36 trillion operations per second,[36] and there are two FSD Chips for redundancy.[41]

HW3 cars are equipped with eight cameras, in the same locations and covering the same directions as the HW2 and HW2.5 cameras. Each of the eight cameras supplied with HW3 use the same AR0136A image sensor supplied by Onsemi, which has a maximum resolution of 1280×960 (1.2-megapixel) and a 3.75 μm pixel size.[42] Initial versions of HW3 also included a Continental ARS4-B radar module.[42] The HW3 system board is the same physical size as the HW2.5 board, but carries more components.[42] This makes it possible to upgrade from HW2 / HW2.5 to HW3, which was not possible during the prior generational change from HW1 to HW2/2.5.

The company claimed that HW3 was necessary for FSD, but not for "enhanced Autopilot" functions.[43] The first availability of HW3 was April 2019.[44] Customers with HW2 or HW2.5 who purchased the FSD package were promised an upgrade to HW3 without cost;[45] however, when FSD Beta was released in 2021, Tesla owners who had subscribed to FSD and owned a vehicle with HW2 or HW2.5 were required to pay US$1500 to upgrade to HW3. Owners who had purchased FSD with a one-time fee as part of their initial vehicle acquisition were eligible for a free upgrade to HW3.[46][47] Within days, Tesla reduced the cost of the upgrade to US$1000.[48] In 2022, a Tesla owner in the State of Washington won a default judgment against Tesla which ordered the company to upgrade his vehicle to HW3 without cost; in the ruling, the judge cited Tesla's promises that all cars sold since 2016 included the hardware for FSD.[49]

Tesla Vision

In late May 2021, Elon Musk posted to Twitter that "Pure Vision Autopilot" was starting to be implemented.[50] The system, which Tesla brands "Tesla Vision", eliminates the forward-facing radar starting in May 2021 from the Autopilot hardware package on Model 3 and Model Y vehicles built for the North American market.[51] The Washington Post reported in March 2023 the immediate result was "an uptick in crashes, near misses and other embarrassing mistakes by Tesla vehicles suddenly deprived of a critical sensor."[52] For vehicles without the forward radar, temporary limitations were applied to certain features such as Autosteer, and other features (Smart Summon and Emergency Lane Departure Avoidance) were disabled, but Tesla promised to restore the features "in the weeks ahead ... via a series of over-the-air software updates".

In response, Consumer Reports delisted the Model 3 from its Top Picks in 2021, and the Insurance Institute for Highway Safety (IIHS) announced plans to delist the Model 3 as a Top Safety Pick+,[53][54]

Tesla Autopilot hardware

Physical versions

Hardware 1

Feature history

Hardware

References

Hardware 2

Hardware 2.5

Hardware 2.5

Hardware 3

Tesla Vision

Tesla Vision

Hardware 4

Hardware 4.5

Hardware 5

Hardware 6

Potential future hardware