SINGAPORE – Artificial intelligence (AI) projects may work well during testing but run into problems when used across an entire company. The AI must handle more information, connect to more business systems, and serve more users, which can increase costs and affect the quality of its answers, according to Boomi, an enterprise software company specializing in integration, automation, data management, and AI.
“The reason they were successful was they were run on manually curated data sets,” CTO and senior director of Solution Engineering for APJ of Boomi, told Back End News during a one-on-one interview on the sidelines of the Boomi World Forum held recently. “So we knew that data was of good quality, good accuracy, and would deliver that outcome.”
Irecki said AI pilot projects often succeed because people carefully choose, clean, and check the information used during testing. That controlled setup is harder to maintain when the AI project is put into full use.
Once deployed across a company, an AI tool or AI agent may need to collect information from databases, business software, documents, and other sources. Some information may be old, incomplete, or unavailable when the AI needs it.
“The challenge of AI is when you scale it across the business, it has access to more data,” Irecki said. “That data may not be fresh; it may not be available in real time.”
Before putting an AI project into full operation, companies should be clear about what they want it to accomplish. They should then identify the information and business systems needed to deliver that result.
Companies do not necessarily have to give an AI agent access to all their data. They can limit it to the information needed for a particular task. This can make the system easier to manage and lower the chance of receiving inaccurate or unreliable answers.
“Businesses, again, coming back to that use case, need to reverse engineer: What parts of the business need to be activated to have a successful outcome for that AI agent?” Irecki said. “That then allows you to reduce that scope and make those scale-outs better.”
Companies also must determine whether an AI agent creates enough value to cover the cost of operating it.
AI models charge based on tokens, or the small pieces of text they process when reading instructions, studying information, and producing answers. The cost can increase when an AI agent handles long instructions, reads large amounts of data, or sends questions to other AI systems.
“We’ve seen in our own ecosystem some customers build agents, scale them to production, and then see the cost of those agents is not worth the return they’re providing,” Irecki said.
Some Boomi customers redesigned their AI agents to process less information and use fewer tokens. Others simplified how the agents communicate with other systems.
In some cases, AI was not the best choice. Irecki said a regular automated system that follows fixed rules could be cheaper and more dependable for certain tasks. Other work may still be better handled by a person.
“Some customers refactor the agents so they use tokens more efficiently by reducing the context window or reducing the way questions are asked,” he said. “Others have found that, for that particular process, it’s actually better to keep it as a deterministic process or have a human continue to do the work.”
For companies, a successful AI test is only the first step. Before wider deployment, they must check whether their data is reliable, whether the project has a clear purpose, and whether the benefits are worth the cost.