Essay · Published · 6 min read

An AI System Should Earn the Right to Act

Observation, explanation and recommendation are cheap. Action is expensive. A case for making AI-assisted systems climb a ladder of trust before they're allowed to touch anything that matters.

  • AI Governance
  • Permissions
  • Product Design
  • Automation

There is a moment in almost every conversation about AI and business software where someone asks: "But can it actually do things, or does it just talk?" The question is usually asked with a slight edge of disappointment, as if a system that only observes and recommends is somehow the lesser version of the real thing.

I've come to think that edge of disappointment is pointing at exactly the wrong instinct. The interesting design question is not whether an AI system can act. Almost anything can be wired up to take an action given enough API access. The interesting question is what it has to demonstrate first, and to whom, before that permission is granted — and how far that permission should extend once granted.

A ladder, not a switch

The mental model I keep coming back to, and the one shaping how NxSync ERP's assistance layer is designed, is a ladder rather than a switch. A system doesn't flip from "cannot act" to "can act." It climbs:

Observe. The system has access to the relevant data and can form a view. Nothing is shown to anyone yet. This rung is nearly free — most software already does this.

Explain. The system can describe what it observed, in language a person can evaluate. This is where most current AI features stop, and it's genuinely useful on its own — a system that can say "this vendor's invoice amounts have drifted 4% above contract terms over the last two quarters" has already saved someone an afternoon of spreadsheet archaeology.

Recommend. The system goes further and proposes a specific next step — "flag this vendor for renegotiation" — rather than just surfacing the observation. This rung requires more confidence in the system's judgment, because a bad recommendation, even one a human is free to ignore, still costs attention and can quietly erode trust if it's wrong often enough.

Prepare. The system does the work of getting an action ready to execute — drafts the renegotiation notice, pre-fills the fields, assembles the paperwork — but stops short of sending or committing it. This is a genuinely useful rung that a lot of automation design skips past on the way to full autonomy, and I think that's a mistake, because "prepared but not executed" is where a huge amount of real business value sits with very little of the risk.

Request approval. For anything that changes state in a way that matters — money moves, a contract term changes, a record that other systems depend on gets altered — the system asks. Explicitly, with the reasoning attached, to a person with the authority to say yes.

Act, within a defined authority. Only after approval, and only within boundaries set in advance — this system may act on purchase orders under this value, may not touch payroll, may not act on anything flagged as a compliance-sensitive record without a second approval.

Why the ladder matters more than the ceiling

Most conversations about AI autonomy focus on the ceiling — how far up the ladder can the system eventually go? I think that's the less useful question. The useful question is whether the system is honest about which rung it's actually standing on right now, for this specific action, in this specific context.

A system that can act autonomously on low-stakes, easily reversible decisions and must stop at "recommend" for high-stakes, hard-to-reverse ones isn't a less capable system than one with a single autonomy setting applied everywhere. It's a better-designed one. The ladder should have different heights for different kinds of actions, not one global autonomy dial that gets turned up as trust in the vendor grows.

This is the standard NxSync ERP's assistance layer is designed against: sensitive operations — anything touching money, anything hard to reverse, anything a regulator would ask about — route through an explicit approval step by design, not as a fallback for when the system isn't confident. Confidence is not the gate. Reversibility and stakes are the gate.

What earning the right to act actually requires

For a system to responsibly climb this ladder for a given class of action, three things need to be true, and I don't think you can shortcut any of them.

First, the system's reasoning has to be inspectable. Not just "here's what I recommend" but "here's the data I looked at and here's why this follows from it," in a form a person can actually check rather than simply trust. An unauditable recommendation is not meaningfully different from a guess, no matter how confident the language around it sounds.

Second, the authority boundary has to be explicit and enforced structurally, not just documented. "The system may approve purchase orders under $5,000" is a governance sentence. It only means something if the platform actually blocks the system from touching a $5,001 purchase order, rather than relying on the system to remember the rule.

Third, every action taken has to leave a record that lets a human reconstruct exactly what happened and why, after the fact. Not for blame — for learning. If an autonomous action turns out to be wrong, the useful question is what in the reasoning chain should have caught it, and you can only answer that if the chain was recorded, not just the outcome.

Where this stands

I'm describing a design standard here, and I want to be honest that a standard is not the same as a fully built system. NxSync ERP is under active development, and this ladder is the framework its assistance features are being designed and reviewed against, module by module. Not every part of the platform has climbed every rung yet — some are still at "observe" or "explain," some are further along, and none has been opened to the kind of unrestricted autonomous action a skeptical reader might picture when they hear "AI-native ERP." That restraint is deliberate, not a gap I'm hoping nobody notices.

If there's a genuine open question I don't have a settled answer to, it's this: how do you know, in practice, when a system has actually earned the next rung, versus when it has simply been tested enough times without an obvious failure to make everyone comfortable? Confidence built from an absence of visible failures is not the same as confidence built from understanding why the system is right when it's right. I don't think that's a problem unique to this product — it's a genuine unresolved question in how anyone evaluates AI-assisted systems responsibly, and I'd rather say that plainly than pretend the ladder resolves it by itself.

View the full NxSync ERP series

Mohammed Umair builds businesses and the systems they run on — across smart infrastructure, international trade and enterprise software. More about him.