Enterprises lean on container solutions to deploy generative AI

Dive Insight:

• Enterprises are shifting workloads to container solutions to drive generative AI adoption, according to a Vanson Bourne survey of 1,500 IT and platform engineering professionals commissioned by software provider Nutanix.
• Nearly 9 in 10 respondents reported that at least some of their organization’s applications are containerized and more than half leverage containers for all applications, the Wednesday report found.
• “We probably had 15% of our data center customers deploying containers in the past,” Lee Caswell, SVP of product and solutions marketing at Nutanix, said. “Now, we’re seeing an early indicator of a wave of containerization coming into on-prem and across hybrid cloud.”

Dive Brief:

There’s no shortage of hurdles on the path to scaling generative AI. As enterprises confront data privacy concerns, model reliability issues and an uncertain regulatory environment, IT teams are recalibrating infrastructure needs.

More than half of respondents said their organization needs to increase investments in IT to support generative AI workloads and applications. Integrating generative AI workloads with existing infrastructure was the biggest obstacle to moving use cases from development into production, the report found.

“Organizations have reached an inflection point with GenAI implementation and deployment,” Caswell said, pointing to the rise in container adoption. More than two-thirds of respondents — 7 in 10 — plan to containerize generative AI applications.

Gartner is seeing a similar dynamic. The analyst firm expects more than three-quarters of AI deployments to use container technology by 2027, up from 50% currently.

Containers and Kubernetes orchestration solutions provide the scalability, reliability and adaptability needed to support AI workloads, Gartner said in a January report. Containerization is becoming “the de facto standard” in the space, according to the firm.

While containers grew up in public cloud, the technology has transferred well to on-prem environments and private cloud estates. Workload portability across ecosystems coupled with the spread of large language models are dual drivers of the shift.

“Generative AI, by its very nature, is a hybrid cloud workflow,” Caswell said. “You’re going to be developing LLMs in the public cloud, deploying them, tuning them and maybe retraining them in your private data center, and then doing inferencing right at the edge.”