‘Inference will represent the overwhelming majority of future AI cost,’ says Amazon CEO Andy Jassy. ‘We feel strong urgency to make inference less expensive for customers.’
Amazon CEO Andy Jassy took a deep dive with shareholders to explain why AWS is investing so much money in AI, how semiconductor chips are the reason AI is so expensive and how AWS is working to fix the high cost of artificial intelligence.
“AI does not have to be as expensive as it is today, and it won’t be in the future. Chips are the biggest culprit,” said Jassy in his annual letter to shareholders this month.
“Most AI to date has been built on one chip provider. It’s pricey,” he said. “[AWS’] Trainium chips should help, as our new Trainium2 chips offer 30 percent to 40 percent better price-performance than the current GPU-powered compute instances generally available today.”
Jassy was the CEO of AWS from its initial formation in the late 1990s until 2021, when he was selected to replace former Amazon CEO Jeff Bezos.
In 2015, AWS revenue was $4.6 billion. AWS currently has a $115 billion annual run rate, representing a nearly 2,400 percent total sales increase.
“Our AI revenue is growing at triple-digit YoY [year-over-year] percentages and represents a multibillion-dollar annual revenue run rate,” said Jassy.
‘Strong Urgency To Make Inference Less Expensive’
Jassy said while AI model training still accounts for a large amount of the total AI spend, “inference will represent the overwhelming majority of future AI cost.”
“Because customers train their models periodically but produce inferences constantly in large-scale AI applications, inference will become another building block service, along with compute, storage, database and others. We feel strong urgency to make inference less expensive for customers.”
More price-performant chips will help in this, he said, but inference will also get meaningfully more efficient with improvements in model distillation, prompt caching, computing infrastructure and model architectures.
He said reducing the cost per unit in AI “will unleash AI being used as expansively as customers desire” and also lead to more overall AI spending.
“It’s like what happened with AWS. Revolutionizing the cost of compute and storage happily led to lower cost per unit and more invention, better customer experiences and more absolute infrastructure spend,” Jassy said.
Over 1,000 GenAI Apps Are Being Built Across Amazon
Jassy took a deep dive into why the company is investing in AI so heavily at such a rapid pace.
“If your mission is to make customers’ lives better and easier every day, and you believe every customer experience will be reinvented by AI, you’re going to invest deeply and broadly in AI. That’s why there are more than 1,000 GenAI applications being built across Amazon, aiming to meaningfully change customer experiences,” he said.
Amazon’s CEO said AWS is quickly developing key building blocks for AI development, such as custom silicon Trainium AI chips; flexible model-building and inference services in Amazon SageMaker and Amazon Bedrock; Amazon Nova models to provide lower cost and latency for customers’ applications; and AI agent creation.
As demand for AWS grows, Amazon will need more data centers, chips and hardware.
“We spend this capital up front, even though these assets are useful for many years. In the case of data centers, for at least 15 [to] 20 years,” Jassy said.
“We continue to believe AI is a once-in-a-lifetime reinvention of everything we know, the demand is unlike anything we’ve seen before, and our customers, shareholders and business will be well-served by our investing aggressively now,” Jassy said.
‘We Don’t Always Get Everything Right’
Jassy calls himself a “superfan” of Amazon and said its employees can make a bigger impact working at Amazon than at any other company in the world.
“It’s challenging to find a company where you can make a bigger impact on the world than you can at Amazon,” he said. “And for builders who want to change the world and who have fire in their belly, there’s no better place to be than Amazon.”
Jassy said Amazon operates like the world’s largest IT startup because of its culture of questioning the norm.
“We don’t always get everything right, and we learn and iterate like crazy,” he said. “But we’re constantly choosing to prioritize customers, delivery, invention, ownership, speed, scrappiness, curiosity and building a company that outlasts us all.”
