Blog

Insights and Perspectives

Practical thinking on AI product engineering, from strategy through to production.

Featured series

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

3 parts

A decision framework for choosing a licensed Copilot, configured agent or custom agent based on data, authority and operating responsibility.

Prompt injection when the model has tools

3 parts

Why filtering is not enough once a model can access tools, data or consequential actions, and how to structure the control problem.

Choosing models like a portfolio, not a religion

4 parts

A practical way to select and govern a small portfolio of models, with clear routing, substitution and exit decisions.

23 July 2026Engineering

Part 3 of 3

Part 3 of 3 · Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate series

The operating model for a custom AI agent

The ownership, evaluation, incident and lifecycle responsibilities that turn a custom AI agent into an accountable service.

AI operationsAgent governanceEnterprise architecture

21 July 2026Engineering

Part 2 of 3

Part 2 of 3 · Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate series

Copilot data governance readiness in a Microsoft estate

The information architecture, permissions and content-management checks that should precede a Copilot rollout.

Microsoft CopilotData governanceInformation architecture

16 July 2026Engineering

Series

Series start · Part 1 of 3

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

A decision framework for choosing a licensed Copilot, configured agent or custom agent based on data, authority and operating responsibility.

Microsoft CopilotAI strategyEnterprise architecture

14 July 2026Engineering

Part 3 of 3

Part 3 of 3 · Prompt injection when the model has tools series

Testing prompt-injection defences with release evidence

How to test prompt-injection controls without over-claiming detector performance or treating one exercise as a permanent result.

AI securityRed teamingRelease assurance

9 July 2026Engineering

Part 2 of 3

Part 2 of 3 · Prompt injection when the model has tools series

Prompt injection provenance and capability controls

How to classify untrusted inputs, verify authority outside the model and constrain the capability union available to a tool-using agent.

AI securityPrompt injectionAccess control

2 July 2026Engineering

Tracing a conversation end to end: model calls, guardrail decisions and routing

What a span should hold in a conversational system, why the session is the unit of tracing, and what it costs to watch. From our own instrumented front door.

Distributed tracingLLM observabilityGuardrail decisionsCost attributionData retention

30 June 2026Engineering

Series

Series start · Part 1 of 3

Prompt injection when the model has tools

Why filtering is not enough once a model can access tools, data or consequential actions, and how to structure the control problem.

AI securityPrompt injectionAgent governance

25 June 2026Engineering

Part 4 of 4

Part 4 of 4 · Choosing models like a portfolio, not a religion series

AI model lifecycle and deprecation: plan the exit before it is urgent

A governance approach for model retirement, open weights, lock-in boundaries and accountable ownership.

Model governanceAI operationsEnterprise architecture

18 June 2026Research

Orchestrating a multi-agent claims workflow: topology, handoffs and blast radius

Four agents given a shared goal will find an answer. Ask them six months later how they found it and you get a transcript, not a decision record.

Multi-agent orchestrationWorkflow state machinesAgent handoff contractsTool permission scopingBlast radius and bounded autonomy

16 June 2026Engineering

Part 3 of 4

Part 3 of 4 · Choosing models like a portfolio, not a religion series

Testing AI model substitution before a provider change forces it

Evidence and release gates for replacing an AI model without mistaking a benchmark result for workload readiness.

AI evaluationModel governanceRelease assurance

11 June 2026Engineering

Part 2 of 4

Part 2 of 4 · Choosing models like a portfolio, not a religion series

AI model routing and fallback: separate decisions, shared evidence

How to route enterprise AI workloads between models, design fallbacks and measure both paths without conflating their purpose.

AI architectureModel routingReliability

9 June 2026Engineering

Series

Series start · Part 1 of 4

Choosing models like a portfolio, not a religion

A practical way to select and govern a small portfolio of models, with clear routing, substitution and exit decisions.

AI strategyModel governanceEnterprise architecture

4 June 2026Engineering

One governed toolset, three assistants: MCP servers as enterprise infrastructure

Four assistants, one management surface, and the layer that decides what any of them may actually do.

Model Context ProtocolAgent tool designEnterprise authorisationMulti-tenancyAudit and observability

21 May 2026Research

Running an evaluation harness as production infrastructure on AWS

Evaluation suites decay rather than fail. Versioned corpora, a run manifest, separated scoring stages, a queryable history and a release gate on AWS.

Evaluation harness infrastructureCorpus versioning and reproducibilityDeterministic assertions and rubric gradingRelease gates and reliability floorsEvaluation cost modelling

7 May 2026Research

Inside the agent evaluation arena: how we score autonomous systems

We built a simulated contact centre to pit multi-agent language systems against reproducible tasks. Here is the harness, the metrics that actually predict production reliability, and where today's models keep tripping.

Agent evaluationSimulation testingReliabilityMulti-agent systems

9 April 2026Engineering

Building a retrieval pipeline that holds up in production

Chunking, embedding and retrieval on Postgres and pgvector that stays fast and accurate once real documents and real users arrive.

RAGpgvectorPostgresRetrieval

12 March 2026Engineering

Testing language-model features before they meet users

A practical evaluation method for language-model features: representative scenarios, calibrated graders, adversarial testing and release evidence that survives change.

LLM evaluationRed-teamingRegression testingRelease assurance

20 February 2026Perspectives

A practical framework for enterprise AI governance

Governance is a design problem before it is a compliance one. Here is how we build guardrails that engineering teams actually use, grounded in ISO 42001 and the direction Australian regulation is now taking.

14 September 2023Services

Using simulation tooling to train and assure AI agents

How Humint Labs has used custom simulation apps across delivery engagements to train and assure chat, messaging and voice agents, from early feasibility work through to repeatable release evidence.

AI evaluationCustom AI toolingAgent testingEnterprise AI delivery

Insights and Perspectives

Featured series

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

Prompt injection when the model has tools

Choosing models like a portfolio, not a religion

The operating model for a custom AI agent

Copilot data governance readiness in a Microsoft estate

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

Testing prompt-injection defences with release evidence

Prompt injection provenance and capability controls

Tracing a conversation end to end: model calls, guardrail decisions and routing

Prompt injection when the model has tools

AI model lifecycle and deprecation: plan the exit before it is urgent

Orchestrating a multi-agent claims workflow: topology, handoffs and blast radius

Testing AI model substitution before a provider change forces it

AI model routing and fallback: separate decisions, shared evidence

Choosing models like a portfolio, not a religion

One governed toolset, three assistants: MCP servers as enterprise infrastructure

Running an evaluation harness as production infrastructure on AWS

Inside the agent evaluation arena: how we score autonomous systems

Building a retrieval pipeline that holds up in production

Testing language-model features before they meet users

A practical framework for enterprise AI governance

Using simulation tooling to train and assure AI agents

Ready to shipEnterprise AI?