Artificial intelligence (AI) is increasingly being positioned as the key to faster software development, smarter customer experiences and more efficient operations.
Yet as organisations rush to build AI-powered applications, there is a growing recognition that success depends not only on the technology itself, but on the controls surrounding it. The challenge is no longer simply how to use AI, but how to do so safely, securely and in a way that aligns with business goals and customer expectations. Building apps with AI should make processes smoother, but a human needs to be in the loop to add guardrails to development and ensure that an app works safely and as intended.
At its summit in London in March, Datadog – a supplier which provides an observability service for cloud-scale applications and monitors servers, databases, tools and services through a SaaS-based data analytics platform – promises the audience that it will demonstrate its knowledge and prowess around AI use, showcasing where it believes the capability of AI to drive modern business operations could be realised.
In a year when AI capital spending is expected to reach $725bn in 2026, this surge in investment is driving business transformation as organisations increase their spending and reshape their operations around AI.
At the Datadog summit, Yrieix Garnier, vice-president of product management, says that the “numerous kinds of AI agents” launched by the company are helping to identify context and problems, as well as recommend fixes, because every additional change introduced creates more “stress on your system”.
“This is very siloed, repetitive and fairly slow,” Garnier says. “This is what we already solve at Datadog; we help customers close that end-to-end loop and make sure we continuously monitor systems for cycle stress. We like to give customers the right information to detect issues and the information needed to remediate them.”
Governance requirements
The company made announcements in London, specifically about its UK datacentre presence, with the opening of a new site. Datadog says this will help customers to meet data governance and security requirements as those demands continue to evolve in the wake of questions around European digital sovereignty.
With 82% of firms surveyed in a recent London Stock Exchange Group study saying they operate in multicloud or hybrid environments, companies are adapting to changing UK data governance requirements as cloud adoption continues to accelerate across regulated organisations.
Garnier says the company has invested in adding “more AI into our product to make sure that we give you those correlations of what’s happening in your environment, so you can cut through the noise and really accelerate resolution times”.
This increased use of AI provides a more automated view and fuller visibility across an entire estate, Garnier claims, as well as bringing AI into infrastructure monitoring. “It’s really about helping you detect and remediate very quickly what’s happening inside Kubernetes environments,” he says. “Understanding what’s happening inside that environment gives you the right recommendations, but also applies fixes on top of it, changing your environment.”
Meet the Concierge
At the summit, Mark O’Neill, senior manager of AI software engineering at Datadog customer Virgin Atlantic, speaks about Virgin Atlantic’s AI achievements. The company’s investment and development centred on a chatbot for its website, intended to take chatbots a step beyond simply answering questions and guiding visitors through a series of prompts, and instead to provide genuine assistance.
O’Neill describes the concept of implementing a customer-facing AI chatbot as “daunting”, particularly as it was launched over a period of just 90 days and “especially when brand reputation is so important”.
As Virgin Atlantic’s brand is built on service, personality and trust, he says, the Concierge chatbot had to fit within those parameters and support help and Q&A, Flying Club queries, flight search and holiday discovery, adding: “For trip planning, we didn’t just prompt an LLM, we observed how our frontline teams support our customers and then built those patterns directly into Concierge.”
Negative experiences with AI-powered chatbots, and their failure to solve problems rather than send users round in circles, have created the need for companies such as Virgin Atlantic to build better chatbot experiences.
O’Neill says that traditional chatbots take users down a fixed path, whereas generative AI (GenAI) can effectively tackle any starting point in a journey, which is a development Virgin Atlantic is “most proud of”. In particular, the holiday discovery function allows a user to find a flight to a specific destination on a particular date through Concierge. “The beauty of this technology is that it gives you the flexibility to have more varied conversations,” he adds.
Developed with OpenAI, O’Neill admits that there were three concerns about Concierge’s delivery: providing the wrong information to customers, personally identifiable information (PII) leakage, and not “authentically being Virgin Atlantic” by ensuring the brand’s tone was present.
To protect PII, O’Neill says a decision was made that Concierge would not contain any personal data within its system. “So, we don’t allow the model to access or process personal customer data,” he adds. “Concierge has no account context, no booking retrieval, no session memory tied to identity and we only support read-only operations.”
Perhaps with an eye on the British Airways data breach in 2018, Concierge can source information, but it cannot conduct transactions, change bookings or update account details.
Build your own LLM?
Should businesses build out their own LLM to support the use of GenAI? O’Neill says Virgin Atlantic used OpenAI’s models and application programming interfaces (APIs) as the foundation of Concierge rather than building its own LLM, adding custom prompts and a retrieval-augmented generation (RAG) database that can answer common questions.
He says the company recognised that it did not necessarily have the knowledge or skills in-house to build the system, so it worked with OpenAI’s consultancy arm, TomorrowAI. He says this provided the knowledge and expertise needed to get the project moving.
“We had the experts in the room explaining this is how you can make it safe, this is how you make it secure, and that was certainly something I pushed,” says O’Neill. “What we recognised as a business is that we fundamentally believe in this technology, and it’s here for the long term.”
So, where does Datadog fit into Concierge? O’Neill says it acts as the end-to-end observability platform. From the front end through to every interaction with OpenAI, “we’ve got that full trace of everything that’s going on: we use it for our monitoring, alerting and running evaluations”.
O’Neill says Datadog is also used in testing and in checking the accuracy of answers during development, meaning it is involved across the entire lifecycle.
AI meets the human element
The message from both Datadog and Virgin Atlantic is clear: AI can accelerate development, automate operations and improve customer experiences, but only when it is supported by strong visibility, careful governance and clear boundaries around what it is allowed to do. Human oversight remains essential, whether that means monitoring infrastructure, validating responses or ensuring that AI systems reflect the values and tone of a brand.
As businesses continue to increase investment in AI, the winners will likely be those that balance speed with control. Organisations that combine observability, security and human judgement will be best placed to build applications that are not only more capable, but also more trustworthy.
