CAMBRIDGE, Mass. — Guiding an AI project from nascent idea to wider deployment is a complex task. For CIOs, navigating that complexity is critical, as companies have started measuring performance based on how well their IT leaders are able to complete — and repeat — the process.
To ease that burden, executives are developing ways to detect the potential for success in a given use case. Some are focused on how employees weave the tool into existing processes.
For Soumya Seetharam, SVP and chief digital and information officer at materials science company Corning, an early sign of pilot success came when staffers incorporated AI model outputs as a source to prove out their forecasting accuracy.
“They actually did it in the operations review and said: ‘Our AI model said this, our internal legacy process said this, this is what we have come up with, and this is what we’re forecasting,’” said Seetharam, speaking Tuesday during the MIT Sloan CIO Symposium. “They’re not blindly using that forecast that the AI model gave. They’re looking at it, comparing it against what the legacy process did and then putting judgment on top.”
MIT researchers found last year that the large majority of generative AI pilots fail before delivering on their stated outcomes. More than half of businesses, according to a Solvd report, will potentially shut down AI pilots this year due to poor performance. Surveyed tech leaders pointed to a lack of oversight as a key factor in derailed pilots.
AI tools that can evolve past the pilot stage call for a championing leader in charge, said Mark Schmidt, CIO at Westlake, a manufacturer of petrochemicals, plastics, and building materials. Under the right leadership, one success can have a ripple effect and spread across the organization as others replicate what worked.
“You have to have the right sponsor,” said Schmidt, speaking on the same panel. “That sponsor has to have the drive and the vision to take it through.”
Data and governance
Two other critical components help determine the success of an AI pilot in the enterprise: data and governance.
Corning has multiple layers of governance, Seetharam said. The company developed a governance council for AI and machine learning, which includes general counsel and several top executives along with Seetharam. The council approves what AI use cases to publish in a common marketplace, enabling other workers across the organization to leverage tools that both work and adhere to company standards.
Even with a governance strategy in place, pilots can still fall apart if they don’t have the access to the data.
Without the right data sets to power the tools and an engaged sponsor to fuel adoption, “you’re picking something that’s probably too difficult at this point in time,” said Schmidt.
To aid in moving pilots forward, organizations must also learn from their previous mistakes, said Vipin Gupta, a former CIO turned advisor and board member.
“We tend to celebrate successes and document successes,” said Gupta, who moderated the panel. “But we don’t show the same discipline to documenting failures.”
