Deloitte finds only 11% of agentic AI reached production while Gartner predicts over 40% of projects will be cancelled by end-2027 - a deep analysis of real agent costs, the legacy ERP problem, AI gateway architecture and five cost-reduction strategies.
Agentic AI differs from ordinary chatbots by doing more than answering: it can plan, invoke tools, read data across systems, make decisions and carry out multi-step work. For example: verify a purchase order in the ERP, check the customer’s credit limit, generate documents and route the request to an approver.
That potential won agentic AI rapid attention - but many experiments still cannot scale into production systems that create business impact.
Deloitte reports 38% of surveyed organizations are piloting, 14% have systems ready to deploy, and only 11% run in production (Deloitte). Meanwhile Gartner predicts more than 40% of agentic AI projects will be cancelled by end-2027 due to rising costs, unclear business value and inadequate risk controls (Gartner).
These numbers do not mean agentic AI has no future - they signal that organizations must stop measuring success by pilot count and start measuring live processes, work quality, cost per mission and business outcomes.
Evidence note: frequently quoted figures such as "average ROI 171-192%" and "agents use 5-30x more tokens than chatbots" still lack clearly verifiable primary sources and should not be used as standard values in a business case without cited data and calculation methods.
Why agentic AI costs more than expected
Agent cost is not a single final answer - it is the entire working cycle:
- Receive and interpret the goal
- Build a work plan
- Search data or invoke tools
- Inspect results
- Revise the plan on errors
- Call the model again to summarize or act
One workflow can therefore trigger many model calls with large contexts and repeated data. If the agent enters a reasoning loop or an external tool misbehaves, cost and runtime can balloon quickly.
This is why "price per token" is not a sufficient KPI. Organizations should measure cost per outcome, such as:
- Cost per successfully resolved customer case
- Cost per incident fixed within SLA
- Cost per invoice verified
- Cost per customer onboarded
- Cost per unit of tested, deployed code
Reality check: data and legacy ERP are bigger problems than token price
Agents perform well when they access correct, contextual data with appropriate permissions. But many legacy ERPs were never designed for AI to operate on behalf of humans.
Common problems:
- No APIs, or read-only APIs
- Critical data trapped in remark fields, scanned documents or attachments
- Customer and product codes inconsistent across systems
- Business rules buried in custom code or staff tribal knowledge
- No way to define which data an agent may read, edit or approve
- Legacy systems cannot roll back transactions when an agent errs
The safer pattern: do not connect agents directly to production databases. Build a control layer between the agent and legacy systems - APIs, identity management, policy enforcement, logging and approval workflows.
An architecture for controlling cost and risk
Organizations should build an AI Gateway as the central point for all requests, handling at least five functions:
- Model routing: send simple tasks to small models, reserving high-capability models for complex work
- Prompt caching: cache reusable parts - system prompts, tool schemas, fixed reference documents
- Context control: cap context size and select only relevant data
- Budget enforcement: set ceilings on tokens, time, rounds and cost per workflow
- Observability: log prompts, models, tool calls, latency, errors and cost at every step
Circuit breakers should stop agents on abnormal conditions: excessive tool calls, over-budget spend, runtime overrun or out-of-scope data access.
For transactions with financial, legal or personal-data impact, apply Human-in-the-Loop before the agent’s final action.
Five strategies to cut cost without cutting quality
1. Model routing and tiering
Do not use top-tier models for every task. Classification, data extraction, document summarization and formatting can run on small models; reserve stronger models for multi-step planning and ambiguous decisions.
2. Context optimization
Agents do not need to see everything. Use retrieval to select relevant documents, summarize conversation history and set expiry on session memory. Cutting irrelevant data improves cost, speed and answer quality at once.
3. Prompt caching
Fixed content - company policies, product manuals, tool definitions - should be designed to be cacheable, with version control and cache invalidation when content changes. Otherwise agents may apply stale rules.
4. Model optimization
For high-volume, stable workloads, consider distillation into specialized models and quantization to cut memory and compute when using open-weight models. Self-hosting is not automatically cheaper: include GPUs, ops staffing, security, monitoring and idle capacity in the TCO.
5. FinOps for AI
AI FinOps must join three layers of data:
| Data layer | What to measure |
|---|---|
| Technology | Tokens, models, cache hits, latency, tool calls, compute |
| Workflow | Completion rate, retries, human interventions, error rate |
| Business | Revenue, cost per case, SLA, conversion, satisfaction |
Reducing cost per token is meaningless if completion rates fall until staff spend more time fixing the work.
A readiness-based framework
Beginner: experiment with boundaries
- Pick use cases with clear volume and baselines
- Start read-only, or have agents propose answers rather than act
- Set caps on cost, time and tool-call counts
- Measure cost per completed task from day one
- Avoid broad whole-ERP integrations
Intermediate: build platform and governance
- Deploy a central AI gateway with model routing
- Build a knowledge layer with metadata and access rights
- Connect ERP through policy-enforced, audited APIs
- Apply prompt caching and context optimization
- Tier agent risk levels by the authority they hold
Advanced: redesign the process
- Move from single agents to clearly separated roles where justified
- Split planning, execution, verification and approval agents
- Use specialized or open-weight models for high-volume workloads
- Run continuous evaluation for quality, cost and model drift
- Connect ROI to P&L and process-owner KPIs
Multi-agent is not a default best practice: more agents mean more tool calls, tokens, failure points and tracking overhead. Use role separation only when it measurably improves quality or control.
Conclusion for executives
Agentic AI produces ROI when organizations choose the right processes, prepare data, define authority boundaries clearly and control costs at the workflow level - not merely the token level.
Do not start by asking "which agent should we use" - ask "which process is valuable enough, how ready is its data, and how will we know the work is truly complete".
The Devsign team designs enterprise agentic AI systems with AI gateways, access control and cost-per-outcome measurement from day one.
