Governed agent tooling: one platform's management surface as a single toolset
An assistant that can call an enterprise platform's management API can build and change production systems. Nothing in that API decides whether it should. This is the layer that does.
- Delivery
- First-party capability build, operated by Humint Labs, running in front of an enterprise conversational AI platform's management surface.
Measured system behaviour
Reference design target: expose governed outcomes rather than raw endpoints.
Reference design target: tenant and operator permissions stay explicit.
Reference design target: tool calls retain caller, intent and outcome context.
Reference design target: consequential actions remain reviewable.
The operating challenge
An enterprise conversational AI platform keeps its entire management surface in an API: projects, conversational flows, intents, endpoints, knowledge stores, connections, snapshots, users. An assistant that can reach that API can build and change production systems, and nothing in the API decides whether it should. The obvious build is to wrap each endpoint as a tool and stop there. That produces a surface no agent can use well and no security team will sign off on: hundreds of near-identical names with no way to tell a read from a deletion, one process holding credentials for every tenant it can reach, and no durable record of which caller asked for what.
Model context turned out to be the scarce resource, not endpoint coverage. A flat list of several hundred tool names spends the context budget before the agent has finished reading the task, and the failure it produces is not an error, it is a confident call to a neighbouring tool whose description also matched. Endpoint shape and task shape are different shapes: what an operator describes as one outcome is often an ordered sequence of calls in the API, and tools drawn straight from the endpoint list push that ordering into the model, where it is guessed rather than enforced. The consequential operations are also a small minority of the surface, most of it reads, a smaller set writes, and a smaller set again is effectively irreversible against a live tenant. Multi-tenancy was an isolation problem before it was a convenience problem, since one operator legitimately holds credentials for several organisations and one server process serves several operators. More than one assistant vendor was already in play, each with its own client behaviour around tool-name length, schema strictness and parallel calls, and building a toolset per vendor would have doubled the governance surface and given the two copies somewhere to drift apart.
What we found
- Model context is the scarce resource, not endpoint coverage. A flat list of several hundred tool names spends the context budget before the agent has finished reading the task, and the failure it produces is not an error. It is a confident call to a neighbouring tool whose description also matched.
- Endpoint shape and task shape are different shapes. What an operator describes as one outcome, standing up a working conversational flow with its intents and its endpoint attached, is an ordered sequence of calls in the API. Tools drawn straight from the endpoint list push that ordering into the model, where it is guessed rather than enforced.
- The consequential operations are a small minority of the surface. Most of it reads. A smaller set writes, and a smaller set again is effectively irreversible against a live tenant: deleting a flow, restoring a snapshot over current configuration, removing a user’s access.
- Multi-tenancy was an isolation problem before it was a convenience problem. One operator legitimately holds credentials for several organisations, and one server process serves several operators. Anything that lets a tenant identifier travel as an ordinary tool argument makes cross-tenant reach a model decision.
- More than one assistant vendor was already in play, each with its own client behaviour around tool-name length, schema strictness and parallel calls. Building a toolset per vendor would have doubled the governance surface and given the two copies somewhere to drift apart.
Where the bottleneck sat
The constraint was never the API. It was that tool design is API design for a consumer that cannot read the documentation, cannot be trained on local conventions, and will make its best guess when two names are close.
Everything hard about this build follows from designing for that consumer.
Design rationale
Part of the answer was not code. Before any tool was written, the surface was sorted into three authorisation tiers, read, configure and consequential, and the list of consequential operations was agreed with the people who would carry the risk of it being wrong.
Which operations are consequential is a policy decision, not an engineering one. Writing the server first would only have encoded someone’s assumption about it faster.
Our Solution
The constraint was never the API. It was that tool design is API design for a consumer that cannot read the documentation, cannot be trained on local conventions, and will make its best guess when two names are close. Before any tool was written, the surface was sorted into three authorisation tiers, read, configure and consequential, and the list of consequential operations was agreed with the people who would carry the risk of it being wrong, since which operations count as consequential is a policy decision, not an engineering one.
What we built is one server, speaking the Model Context Protocol, in front of the platform's management surface. Tools are named and scoped by the outcome an operator wants rather than by the endpoint that happens to deliver it, and where an outcome needs several ordered calls, the ordering lives in the tool. Every tool belongs to exactly one functional category, so the surface is described to the model as a structure rather than a flat list. Each tool carries its authorisation tier at registration rather than a check applied by convention at call time, and consequential tools stay absent from a session that has not been granted them, rather than present and refusing. A session is bound to a single tenant, with credentials resolved per request from the caller's own identity and never appearing in the model's context or a tool argument, so addressing another organisation is not a call the model is able to formulate. Every call is written to an audit record on the request path, capturing caller, tenant, tool, tier, redacted arguments and outcome, whether the call succeeds, fails, times out or is refused. The server implements the protocol rather than a particular vendor's client, so vendor-specific behaviour is absorbed at the edge and the same registration, tiers and audit format serve more than one assistant vendor.
The hardest part was keeping the surface discoverable as it grew. At a few dozen tools, naming sorts itself out. At several hundred, every new tool competes with everything already registered, and the failure mode is silent, the model reaches for a neighbouring tool whose description also matched, calls it confidently, and returns something that reads as correct. The fix was structural rather than a better prompt: one canonical verb set, one noun per resource, category membership carried in tool metadata rather than in prose, and descriptions written to say what a tool does not do as clearly as what it does. Routing is treated as a regression test, a held-out set of operator requests is replayed and the tool the model reaches for is checked against the tool the request meant.
Task-shaped tools
Tools are named and scoped by the outcome an operator wants, not by the endpoint that happens to deliver it. Where an outcome needs several ordered calls, the ordering lives in the tool, so the model chooses one thing correctly instead of sequencing six things plausibly.
Category structure
Every tool belongs to one functional category, and categories are how the surface is described to the model rather than an afterthought in the docs. At hundreds of tools across dozens of categories, discoverability is a naming problem: one canonical verb set, one noun per resource, no synonyms allowed in.
Authorisation tiering
Read, configure, consequential. The tier is declared at registration rather than checked by convention at call time, so a tool cannot ship untiered. Consequential tools stay absent from a session that has not been granted them, rather than present and refusing, because a visible tool the model cannot use is a retry loop.
Per-tenant credential isolation
Credentials are resolved per request from the caller’s own identity and never appear in the model’s context or in a tool argument. A session is bound to one tenant, so addressing another organisation is not a call the model is able to formulate rather than one it is asked not to make.
Audit trail
Each call records caller, tenant, tool, tier, redacted arguments and outcome. The record is written on the request path, so a call that fails, times out or is refused is recorded the same as one that succeeds. Reviewing what an agent did is a query, not a reconstruction from platform logs.
Vendor-neutral transport
The server implements the protocol, not a particular vendor’s client. Vendor-specific behaviour, name-length limits and schema strictness among them, is absorbed at the edge so the toolset itself stays single. Two assistant vendors consume the same registration, the same tiers and the same audit format.
Keeping the surface discoverable as it grew. At a few dozen tools, naming sorts itself out. At several hundred, every new tool competes with everything already registered, and the failure mode is silent: the model reaches for a neighbouring tool whose description also matched, calls it confidently, and returns something that reads as correct. The fix was structural rather than a better prompt. One canonical verb set, one noun per resource, category membership carried in tool metadata rather than in prose, and descriptions written to say what a tool does not do as clearly as what it does. Routing is then treated as a regression test: a held-out set of operator requests is replayed and the tool the model reaches for is checked against the tool the request meant.
What changed
- An operator asks for an outcome and the server decides which calls that takes, instead of the model inferring an ordering from endpoint names.
- Whether an agent can perform a given operation has one answer, in the tier declaration, rather than a different answer per assistant vendor.
- A second assistant vendor was added without a second toolset, and so without a second place for governance to drift.
- Adding a tool now means placing it in a category and a tier before it can register, so the governance decision happens at authoring time rather than at review time.
- Cross-tenant reach stopped being a rule the model is asked to respect and became a call it cannot formulate.
- Reviewing what an agent did is an audit query, not an attempt to reconstruct intent from platform activity logs.
More case studies
Layered guardrails across voice, web chat and messaging
A policy that holds on web chat and quietly softens on voice is not a policy. The layers, the order they run in, and what the assistant does when one of them fires.
Industry: Insurance
Answering insurance enquiries from the wording, and escalating everything else
The design settled what would never be automated before it settled what would. Everything after that was a question of evidence: what the answer was built from, and whether it could be shown.
Building an automated regression harness for retrieval-grounded agents
A change to a prompt, a retriever or a base model can quietly break an answer that used to be correct. We stopped checking that by hand.