Tesla has embarked on a groundbreaking venture with the development of its custom-built supercomputer, Dojo. Designed to train the "Full Self-Driving" (FSD) neural networks, Dojo is set to redefine the landscape of artificial intelligence and computing. Spearheaded by Elon Musk, Tesla's ambitious project is expected to reach unprecedented computing power levels by October 2024. The supercomputer aims to enhance Tesla's AI capabilities and position itself among the top five most powerful supercomputers globally by February 2024.
The core of Dojo's remarkable processing power lies in its architecture. Each tile in Dojo boasts a compute power of 9 petaflops and an impressive bandwidth of 36 terabytes per second. These tiles integrate all necessary hardware elements for power, cooling, and data transfer, enabling seamless operation. At the heart of each tile is the D1 chip, which houses 50 billion transistors and spans a die size of 645 millimeters squared. Tesla's AI team has ingeniously fused 25 such D1 chips into a single tile, creating a unified computer system.
Tesla collaborated with Taiwan Semiconductor Manufacturing Company (TSMC) to manufacture Dojo using advanced 7-nanometer semiconductor nodes. The initial version of Dojo is tailored for Tesla's computer vision labeling and training tasks. However, future iterations will expand to train general-purpose AI models, broadening its scope and utility.
Tesla envisions Dojo as a pivotal tool for storing and processing immense volumes of video data collected from its global fleet of vehicles. The supercomputer will run millions of simulations to refine its models and enhance the FSD system's accuracy and reliability. With these capabilities, Tesla aims to push the boundaries of autonomous driving technology.
In the fourth quarter, Tesla completed deploying Cortex, a significant milestone that facilitated the launch of V13 of supervised FSD. While Dojo's production began in July 2023, Elon Musk confirmed that it has been online and executing useful tasks for several months. However, Tesla has remained tight-lipped regarding the status of getting the D1 chips fully operational within Dojo.
Dojo's potential impact on AI research and development cannot be overstated. It is poised to achieve a total compute power of 100 exaflops by October 2024, equating to an astounding 1 quintillion computer operations per second. This capability will place Dojo at the forefront of supercomputing advancements, challenging existing paradigms and setting new benchmarks.
Anand Raghunathan, an expert in the field, highlights potential challenges in scaling AI models with increased data volume. He states:
"First of all, there's an economic constraint, and soon it will just get too expensive to do that." – Anand Raghunathan
He further elaborates on the importance of meaningful data:
"Some people claim that we might actually run out of meaningful data to train the models on. More data doesn't necessarily mean more information, so it depends on whether that data has information that is useful to create a better model, and if the training process is able to actually distill that information into a better model." – Anand Raghunathan
Ganesh Venkataramanan, another industry authority, underscores Dojo's technical prowess:
"We can do compute and data transfers simultaneously, and our custom ISA, which is the instruction set architecture, is fully optimized for machine learning workloads." – Ganesh Venkataramanan
Leave a Reply