Businesses are embracing AI agents at a pace that is outstripping their ability to manage the risks that come with them, according to a new report from Deloitte. While adoption is accelerating across industries, safeguards around security, data privacy, and accountability are lagging behind, raising concerns about how prepared organisations really are.
Deloitte’s findings show that agentic AI systems are rapidly moving from experimental pilots into full production environments. However, many of the risk controls in place were designed for human-led processes and are proving inadequate for autonomous or semi-autonomous systems.
Only 21% of organisations surveyed said they have implemented strong governance or oversight frameworks for AI agents. At the same time, 23% are already using AI agents, a figure expected to jump to 74% within two years. Over that same period, the share of companies yet to adopt AI agents is projected to fall sharply, from 25% to just 5%.
Governance, not AI, is the real risk
Deloitte does not frame AI agents as inherently dangerous. Instead, the report points to weak governance and poor contextual controls as the primary sources of risk. When AI agents operate with too much autonomy and too little oversight, their decisions can become difficult to explain, manage, or insure.
Ali Sarrafi, CEO and founder of Kovant, describes the solution as “governed autonomy.” In practice, this means treating AI agents much like human employees, with clearly defined roles, limits, and escalation paths.
“Well-designed agents with clear boundaries and policies can move quickly on low-risk work,” Sarrafi said, “but escalate to humans when actions cross defined risk thresholds.” With detailed action logs and observability built in, agents become systems that can be inspected and trusted rather than opaque black boxes.
Why guardrails matter in real-world environments
AI agents often perform well in controlled demonstrations, but real business environments are rarely neat or predictable. Data can be fragmented, systems may not integrate cleanly, and edge cases are common.
According to Sarrafi, giving agents too much scope or context at once increases the risk of hallucinations and erratic behaviour. More robust systems break work into narrower, well-defined tasks handled by individual agents. This makes behaviour easier to predict, failures easier to detect, and intervention possible before small errors cascade into larger problems.
Accountability and insurability come into focus
As AI agents begin taking real actions inside business systems, accountability becomes critical. Detailed action logs allow organisations to review exactly what an agent did, when it did it, and under what conditions.
This transparency is especially important for insurers, who remain cautious about covering opaque AI-driven systems. Auditable workflows, human oversight for high-impact actions, and replayable decision trails make it easier to assess risk and assign responsibility when things go wrong.
Standards are emerging, but gaps remain
Shared frameworks such as those being developed by the Agentic AI Foundation (AAIF) are a step forward, helping organisations integrate different agent systems. However, Sarrafi argues that many current standards focus on what is easiest to build, not what enterprises need to operate AI agents safely at scale.
Larger organisations require standards that include access controls, approval workflows for sensitive actions, and detailed observability tools. These elements allow teams to monitor behaviour, investigate incidents, and demonstrate compliance when required.
Identity, permissions, and visibility as first-line defences
One of the clearest messages from both Deloitte and industry experts is the importance of limiting what AI agents can access and what actions they are allowed to take. Broad privileges increase unpredictability and introduce security and compliance risks.
Continuous monitoring and logging help keep agents within defined boundaries. When every action is visible and traceable, teams can quickly understand what happened, why it happened, and how to fix it.
“This visibility, combined with human supervision where it matters, turns AI agents into systems that can be inspected, replayed, and audited,” Sarrafi said. “That’s what builds trust with operators, risk teams, and insurers.”
Deloitte’s blueprint for safer AI agents
Deloitte’s proposed governance model relies on tiered autonomy. In early stages, agents may only view information or make recommendations. As confidence grows, they can be allowed to take limited actions with human approval. Fully autonomous actions come last, and only in low-risk, well-understood areas.
The firm’s “Cyber AI Blueprints” also emphasise embedding governance into everyday operations, rather than treating it as a one-off compliance exercise. Training employees is another key pillar. Staff need to understand what information should not be shared with AI systems, how to respond when agents behave unexpectedly, and how to recognise potential risks early.
A careful path forward
AI agents are set to play an increasingly central role in how organisations operate. Deloitte’s message is clear: the competitive advantage will not go to the companies that deploy agents the fastest, but to those that deploy them with visibility, control, and accountability.
With the right guardrails, shared standards, and workforce training, AI agents can be powerful tools rather than unmanaged risks. As adoption accelerates, robust governance may prove to be the most valuable investment of all.
