A cross-border retail client chatted with me last week about something: they connected an agent to their customer service system and let it handle refunds automatically. The first week after launch was calm, but in the second week they discovered—an anomalous refund, without anyone noticing, had been triggered more than thirty times in a row, and it was only exposed when finance reconciled at month-end. Not much money was lost, but the fact that "the system ran on its own and no one knew" gave their CIO a cold sweat.
This is not an isolated case. Looking at the developments in enterprise-level AI agents over the past week as a whole, a clear signal emerges: the industry has already shifted from "how to make agents move" to "how to make agents move in a controllable way." The control layer is transforming from a principle on a PPT slide into a native braking system within platforms.
The first brake: assessment first, making "whether it ran correctly" observable
In the past, we judged whether an automation was reliable by pre-launch testing plus manual spot checks. But agents are different—they decide their next step based on context, and the same input may take different paths tonight and tomorrow morning. The traditional playbook of "test once and launch" is largely ineffective for agents.
Agent Evals, which Salesforce recently added to Agent Fabric, does exactly this: it continuously tests and evaluates the agent itself while also detecting changes to the APIs it depends on. In other words, the agent doesn't run in a vacuum; once any interface it calls quietly changes its response structure or the meaning of a field, the evaluation layer can raise an alert "before something goes wrong."
The lesson here is: before enterprises connect agents to ERP, they must first ask clearly—can my platform continuously verify the agent's behavior at runtime? If the answer is 'we can only rely on manual log review after launch,' then the larger this agent fleet grows, the deeper the mines are buried. Evaluation is not a post-launch patch, but the agent's factory setting.
The Second Brake: Identity First — Issue the Agent an "ID Card"
When it is no longer just humans running in the system, but also a large number of "non-human identities," a more subtle problem emerges: how do you know whether the one modifying this record is Zhang San or Li Si, or a malicious agent disguised as automation?
Socure's recently launched Agent Intelligence productized this. Its Agent Detection is now generally available, Agent TrustScore is in beta testing, and a verifiable Agent Credential is planned for launch in the first quarter of next year. The core logic is KYA (Know Your Agent)—identify agent traffic, score it by identity, authorization, and risk, and issue credentials that bind an agent back to a natural person or corporate entity. At the same time, Resemble released a behavioral detection model aimed at agents "masquerading as ordinary automation," with an internal benchmark detection rate of about 98.8%, far higher than the level of less than 15% for traditional bot tools.
For ERP scenarios, this means that "creating an account for an agent" will be like "creating an account for an employee," requiring a set of identity governance. Before allowing agents to write orders, make payments, or modify inventory, establishing an agent identity directory and trust score is a more prerequisite task than "how many agents to deploy." An agent without identity and without a trust score, no matter how smart, should not touch core documents.
The third brake: circuit breakers and caps, putting loss of control in a cage
What keeps CFOs up at night the most is “cost runaway.” Once a usage-based agent has a logic error or gets its API hammered by someone, the bill can blow through the monthly budget within hours. Salesforce’s kill-switch (one-click circuit breaker) and hard spending cap, introduced at the same time, are aimed squarely at this pain point—when something goes wrong, the operator can cut it off with one click, and a single agent’s consumption is locked within budget.
This serves as a reminder to every enterprise preparing to scale up agents: does your agent platform provide the trio of runtime evaluation, circuit breaking, and spending caps? If any one is missing, "scaling up" amounts to "taking on risk." Especially in systems like ERP that directly touch funds, inventory, and vouchers, high-stakes actions (changing accounts, making payments, issuing orders) should never be set to "fully automatic"; at minimum, there must be a human gate and a hard budget line.
Three brakes, one main line
Connecting these three things, the main line is clear: before an agent can "act," it must first be able to be "identified, assessed, cut off, and held accountable." This is the dividing line for agents moving from experimentation to large-scale production, and also the bottom line that regulators and legal teams will sooner or later press on.
For CIOs who are in the middle of solution selection or preparing to scale up, here is a very practical control-layer checklist:
First, establish identity first, grant permissions later. Every agent that can touch the system must have an identity directory, a trust score, and a clear accountable party (tied back to which natural person or department). Anonymous agents do not enter the production environment.
Second, keep evaluations always on, and always report changes. Both the agent and the APIs it calls must have runtime evaluation; when an interface changes structure, warn first before allowing it through—don't wait until something goes wrong.
Third, circuit breaking and hard caps are factory defaults, not optional add-ons. High-stakes actions always keep a human gate; per-agent consumption has a hard ceiling; anomalies can be cut off with a single click.
Fourth, keep sovereignty over the foundation in your own hands. Whether you choose a closed-source major vendor or an open-source route, ensure that identity logs can be retained by yourself, costs can be verified, and rules can be written by yourself. Take an open-source foundation such as Odoo 20 as an example: MCP-native, 100% data ownership, governance logic defined by yourself, and you will not be doubly held hostage by international platforms' compliance frameworks and pay-as-you-go billing black boxes.
In the final analysis, in the competition for agentic ERP, the first half is about who can integrate agents first, and the second half is about who can install the brakes first. Once the brakes are installed, the agent is leverage; if not, it merely automates loss of control.
