Elon Musk’s latest AI startup, xAI, has already brought its new supercomputer, “Colossus,” online.
Launched over the Labor Day weekend, Colossus is designed to train xAI’s large language model (LLM), Grok, which aims to rival OpenAI’s GPT-4.
Musk, known for his rapid execution and ambitious goals, announced that Colossus is “the most powerful AI training system in the world.”
He also revealed plans to double its capacity in the coming months by acquiring 50,000 of Nvidia’s advanced H200 chips, which are twice as powerful as the H100 models currently used.
The supercomputer is located in Memphis, a site selected by xAI this June.
The speed of the project’s completion is remarkable, considering the fierce competition among tech giants like Microsoft, Google, and Amazon to secure Nvidia’s Hopper series AI chips during the current AI boom.
Musk has committed to spending $3-4 billion on Nvidia hardware this year for both xAI and Tesla, which gave xAI a head start by providing its AI chips.
Colossus will be used to train Grok-3, Musk’s third generation AI model, which is expected to be the most powerful AI in the world upon its release in December.
An early beta version of Grok-2, trained on 15,000 Nvidia H100 chips, was recently rolled out and is already being recognised as one of the most capable AI models.
The project has raised concerns in Memphis, particularly regarding the strain on the city’s resources, as Colossus requires up to 1 million gallons of water per day for cooling and could consume as much as 150 megawatts of power.
However, Musk’s commitment to speed and scale has been a key driver in xAI’s operations, as he emphasised in a recent discussion with podcaster Lex Fridman.
To further bolster xAI’s capabilities, the company raised $6 billion in a Series B funding round in May, backed by major investors such as Andreessen Horowitz, Sequoia Capital, Fidelity, and Saudi Arabia’s Kingdom Holding.
Musk has also indicated plans to propose a $5 billion investment from Tesla into xAI, which could further accelerate the company’s growth.