Amazon’s AI plans: custom chips, an Anthropic “ultracluster,” and its own foundation model

While Amazon’s new models appear to be competitive in terms of features and performance, that isn’t the main thing that the company is touting — it’s the cost.

Jon Keegan

12/4/24 2:11PM

This week at Amazon’s AWS re:Invent conference in Las Vegas, the company fleshed out its plans to both serve and compete with the larger AI industry.

AWS is largely AI agnostic. Customers can use pretty much any of the major AI models on the cloud-computing platform, running on servers that use chips from Nvidia, AMD, Qualcomm, and others.

But Amazon has also been building and selling computing powered by its purpose-built AI chips, including its latest Trainium2 chip, which Amazon is now making widely available on AWS’s EC2 service. Amazon says these new Trainium2 instances are built for training and deploying jumbo-sized large language models with better price performance than its current offerings.

Amazon also deepened its partnership with AI startup Anthropic, announcing that it’s building an “ultracluster” of “hundreds of thousands” of Trainium2 servers to train Anthropic’s next-generation LLM. Amazon recently doubled its investment in Anthropic, bringing the total to $8 billion.

Probably the most significant announcement was Amazon’s late entry to the foundational AI-model club. Named “Amazon Nova,” the new LLM comes in four flavors: a text-only Micro and three multimodal models, Lite, Pro, and Premier. Amazon touted benchmark scores for the Nova models, which place it in the same class as OpenAI’s GPT-4o and Meta’s Llama 3. Amazon’s multimodal Nova models can ingest and generate images and videos, like many of the other top models out there today.

While Amazon’s new models appear to be competitive in terms of features and performance, that isn’t the main thing that the company is touting — it’s the models’ low, low cost.

Running Amazon models on Amazon servers, powered by Amazon chips, yields significant cost savings and low latency. Amazon says its Nova models are “at least 75% less expensive” than the best-performing models available on AWS today.

Rani Molla3h

Barclays says autonomous couriers — think sidewalk robots and drones — could push delivery costs down to as little as $1 per order, from between $5 and $7 today and closer to $9 for traditional deliveries in high-labor-cost markets. If robots save $4 on every delivery, and enough companies start using them, the food delivery industry, including companies like DoorDash and Uber, could end up with $16 billion in extra profit every year, according to Barclays.

The catch: we’re nowhere near that world yet. Robots and drones handle less than 1% of deliveries today. Even by 2035, Barclays only sees penetration hitting around 10%.

Google’s Wing and Amazon have also been trying to crack last-mile product delivery — a reminder that this is part of a broader race to automate the most expensive leg of e-commerce.

Rani Molla

Lyft is building the infrastructure robotaxis can’t avoid

The ride-hailing company is building an 80,000-square-foot Nashville warehouse where humans will help robotaxis with everything but driving.

Person working at Lyft Nashville warehouse

$10B

Rani Molla5h

Uber has long had an asset-light business model: it provided the ride-hailing platform, and its contract workers brought their own vehicles. That’s changing as Uber positions itself at the center of the robotaxi era.

The Financial Times estimates that Uber has committed more than $10 billion to buying robotaxi fleets ($7.5 billion) and investing in the companies that make them ($2.5 billion). That includes yesterday’s announcement that it’s expanding its investment in Lucid, a deal worth about $2 billion, with plans to buy 35,000 vehicles.

This shift pits Uber against industry leaders like Google’s Waymo and Tesla, whose models involve company-owned vehicles running on proprietary platforms. While these autonomous fleets eliminate the need for drivers, they introduce new capital-intensive requirements for charging, cleaning, storage, and repair.

Jon Keegan23h

Report: US Treasury wants to get a look at Anthropic’s Mythos model

Anthropic’s relationship with the US government is complicated — and the Treasury Department is reportedly looking to make it even more so.

The Pentagon has officially deemed the startup a national security supply chain risk after it refused to allow its Claude AI to be used for any and all national security applications, including domestic surveillance and autonomous killing.

But since Anthropic’s unusual announcement of its next model, Mythos, other parts of the US government want to get their hands on it.

Bloomberg reports that the US Treasury is interested in getting access to Mythos for its own security testing. Last week, Treasury Secretary Scott Bessent summoned top Wall Street CEOs to Washington to discuss the cybersecurity implications of the new model.

Mythos has not yet been released to the public, as Anthropic has deemed its potential offensive cybersecurity capabilities to be too dangerous for wide release, and has opted to share the powerful new model only with a group of leading tech companies.

Anthropic wants these early access partners to test out the model, hoping to secure any major vulnerabilities before a public release. OpenAI also shared a forthcoming AI-powered cybersecurity tool with a select group of partners to shore up defenses in light of advances in detecting vulnerabilities.

European regulators were apparently left out of the loop from the Mythos announcement, and are also eager to test the new model.

US Treasury Seeking Access to Anthropic’s Mythos to Find Flaws

But since Anthropic’s unusual announcement of its next model, Mythos, other parts of the US government want to get their hands on it.

European regulators were apparently left out of the loop from the Mythos announcement, and are also eager to test the new model.

Nate Becker

4/14/26

Quantum stocks rip as IonQ discloses new contracts, hits benchmark milestones

It’s also World Quantum Day.

Amazon’s AI plans: custom chips, an Anthropic “ultracluster,” and its own foundation model

More Tech

Report: US Treasury wants to get a look at Anthropic’s Mythos model

US Treasury Seeking Access to Anthropic’s Mythos to Find Flaws

Latest Stories