The Next AI Breakthrough Is Knowing When Not to Act

John Engerholm writes on why digital workers need explicit decision rights before they can exercise meaningful autonomy.

Every enterprise already has an architecture for authority. It just was never designed for machines.

A sales leader may approve a discount up to a certain threshold. Finance may release expenditure within a defined mandate. Legal may need to approve a contractual exception even when its monetary value is small. Some decisions belong to a role, some depend on circumstance, and others require several people to agree before anything can happen.

For decades, enterprises have made this work through job descriptions, delegation matrices, policies, approval structures, and institutional knowledge. Much of the system remains implicit because people are remarkably good at interpreting boundaries. They know when a decision feels unusual, when a policy does not quite fit, or when something needs to travel further up the organisation.

Digital workers do not inherit that intuition.

As agents move from retrieving information and generating recommendations to changing prices, approving exceptions, moving money, modifying contracts, and triggering workflows, this becomes a fundamental architectural problem.

We have spent enormous effort asking whether AI is capable of making these decisions. The harder question is whether it has the authority to make them.

A human approval button is not an authority model

Enterprise AI often treats human-in-the-loop as the answer to this problem. Put a person before an important action, and the system is governed.

But authority does not work that way.

Consider a hypothetical contract renewal. An agent may be authorised to renew a standard contract within existing commercial terms. A 2 percent pricing adjustment might require sales approval. A liability clause could trigger legal review regardless of contract value. A strategically important customer might require executive involvement even when every individual term sits within policy.

The required human involvement changes with the decision.

This is why digital workers need something closer to job descriptions than blanket permissions. Their mandate should specify what they may observe, recommend, decide, approve, execute, and escalate, and the conditions under which those rights change.

“Human in the loop” describes the presence of a person. It does not tell us when that person should enter, what authority they hold, or when the machine must surrender control.

That distinction becomes more important as enterprises deploy agents across workflows rather than isolated tasks. The same action may be acceptable in one context and prohibited in another. Authority is conditional on risk, value, policy, role and consequence.

The org chart is becoming incomplete

An organisational chart tells us who reports to whom. It says remarkably little about how decisions actually move through an enterprise.

A workforce containing both humans and digital workers requires another representation: an authority graph.

The authority graph asks a different set of questions. Who can make this decision? Under what conditions? Up to what limit? Which policy applies? Whose approval is required? When must authority transfer elsewhere?

Crucially, “who” can now be either human or digital.

Ontology has an important role here because authority depends on meaning. An agent needs a semantic understanding of the relationships between customers, contracts, roles, policies, decisions, risks, thresholds, and approval paths. But representation alone does not enforce authority. Policy evaluation, identity and access controls, workflow orchestration, and deterministic execution controls still determine whether an action is actually permitted.

That separation matters. The ontology can tell the system what a contractual exception is and how it relates to enterprise policy. The authority architecture determines whether this agent, in this situation, has the right to act on it.

This is also where policy itself begins to change form.

Policy Document → Semantic Policy → Executable Authority

A policy written for people can tolerate interpretation. A policy governing autonomous execution must eventually become precise enough for software to evaluate. Governance therefore moves closer to the work itself, where authority can be checked before an action occurs rather than reconstructed after it.

Autonomy should be earned, and it should be revocable

The ambition should not be maximum autonomy. It should be productive autonomy.

A useful progression is:

Observe → Recommend → Act with Approval → Act with Exception Review → Autonomous Execution

An agent can begin by observing work and producing recommendations. As its behaviour becomes better understood, its mandate can expand. Approval can move from every action to defined exceptions. Eventually, a bounded class of decisions may be executed autonomously.

But progression should never be treated as permanent promotion.

Authority should remain conditional and revocable. A change in policy, unusual transaction value, deteriorating confidence, new customer risk, or an unexpected combination of circumstances may narrow an agent’s mandate again.

This creates another requirement that will matter enormously in real deployments: an agent must know when to stop.

“I am not authorised to make this decision” may become one of the most valuable outputs an enterprise agent can produce.

And that check cannot happen only when the task begins. Enterprise state changes. A customer’s risk status can shift. Aggregate exposure can cross a threshold. Another approval may be withdrawn. Before consequential execution, the system may need to revalidate the current state and confirm that the authority under which the decision was formed still exists.

The same logic changes what enterprises should audit. Recording an AI output is insufficient. The useful record is the decision chain: what the agent knew, which policy applied, what authority it held at that moment, what approvals were obtained, whether the relevant state was revalidated, why it acted, and what happened afterwards.

This is a different conception of governance. It treats autonomy as delegated authority that can be granted, constrained, observed and withdrawn.

The organisational chart will not disappear. But it was designed for a workforce of people. As digital workers enter the operating fabric of the enterprise, the authority graph may become just as consequential: one describes where people sit; the other defines how humans and machines are permitted to decide.

Before asking how autonomous an AI agent can become, enterprises may need to answer a more fundamental question:

Can the enterprise itself express authority precisely enough for a machine to obey it?

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top