Blog

Insights and Perspectives

Practical thinking on AI product engineering, from strategy through to production.

Featured series

Where a decision model fits in enterprise service workflows

5 parts

Where a fast decision model fits in your service, where it does not, and how to trial it safely, from a repeated Humint Labs test.

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

3 parts

A decision framework for choosing a licensed Copilot, configured agent or custom agent based on data, authority and operating responsibility.

Prompt injection when the model has tools

3 parts

Why filtering is not enough once a model can access tools, data or consequential actions, and how to structure the control problem.

Choosing models like a portfolio, not a religion

4 parts

A practical way to select and govern a small portfolio of models, with clear routing, substitution and exit decisions.

27 October 2026

Part 5 of 5 in the series Where a decision model fits in enterprise service workflows

How we tested decision models

The harness, governed scenarios, answer key and scoring controls we built to compare decision models repeatably.

Decision modelsAI evaluationEvaluation harness infrastructure

26 October 2026

Part 4 of 5 in the series Where a decision model fits in enterprise service workflows

Putting decision models to work in the contact centre

Four places a decision model earns its keep in the contact centre and digital service, what a person still decides, and what each design costs to run.

Decision modelsModel routingAI evaluation

25 October 2026

Part 3 of 5 in the series Where a decision model fits in enterprise service workflows

How decision models compare in the contact centre

Which decision models are good enough for contact centre routing and handoff, what they cost, and how to choose between them, from a repeated Humint Labs test.

Decision modelsModel routingAI evaluation

6 October 2026

Part 2 of 5 in the series Where a decision model fits in enterprise service workflows

How we tested the Jev decision model, step by step

The cases, scoring rules and limits behind our Jev decision model test, written so another team can design a comparable test.

AI evaluationEvaluation harness infrastructureLLM observability

5 October 2026

Start of a series, part 1 of 5

Where a decision model fits in enterprise service workflows

Where a fast decision model fits in your service, where it does not, and how to trial it safely, from a repeated Humint Labs test.

Decision modelsModel routingAI evaluation

23 July 2026

Part 3 of 3 in the series Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

The operating model for a custom AI agent

The ownership, evaluation, incident and lifecycle responsibilities that turn a custom AI agent into an accountable service.

AI operationsAgent governanceEnterprise architecture

21 July 2026

Part 2 of 3 in the series Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

Copilot data governance readiness in a Microsoft estate

The information architecture, permissions and content-management checks that should precede a Copilot rollout.

Microsoft CopilotData governanceInformation architecture

16 July 2026

Start of a series, part 1 of 3

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

A decision framework for choosing a licensed Copilot, configured agent or custom agent based on data, authority and operating responsibility.

Microsoft CopilotAI strategyEnterprise architecture

14 July 2026

Part 3 of 3 in the series Prompt injection when the model has tools

Testing prompt-injection defences with release evidence

How to test prompt-injection controls without over-claiming detector performance or treating one exercise as a permanent result.

AI securityRed teamingRelease assurance

9 July 2026

Part 2 of 3 in the series Prompt injection when the model has tools

Prompt injection provenance and capability controls

How to classify untrusted inputs, verify authority outside the model and constrain the capability union available to a tool-using agent.

AI securityPrompt injectionAccess control

2 July 2026Engineering

Tracing a conversation end to end: model calls, guardrail decisions and routing

What a span should hold in a conversational system, why the session is the unit of tracing, and what it costs to watch. From our own instrumented front door.

Distributed tracingLLM observabilityGuardrail decisionsCost attributionData retention

30 June 2026

Start of a series, part 1 of 3

Prompt injection when the model has tools

Why filtering is not enough once a model can access tools, data or consequential actions, and how to structure the control problem.

AI securityPrompt injectionAgent governance

25 June 2026

Part 4 of 4 in the series Choosing models like a portfolio, not a religion

AI model lifecycle and deprecation: plan the exit before it is urgent

A governance approach for model retirement, open weights, lock-in boundaries and accountable ownership.

Model governanceAI operationsEnterprise architecture

18 June 2026Research

Orchestrating a multi-agent claims workflow: topology, handoffs and blast radius

Four agents given a shared goal will find an answer. Ask them six months later how they found it and you get a transcript, not a decision record.

Multi-agent orchestrationWorkflow state machinesAgent handoff contractsTool permission scopingBlast radius and bounded autonomy

16 June 2026

Part 3 of 4 in the series Choosing models like a portfolio, not a religion

Testing AI model substitution before a provider change forces it

Evidence and release gates for replacing an AI model without mistaking a benchmark result for workload readiness.

AI evaluationModel governanceRelease assurance

11 June 2026

Part 2 of 4 in the series Choosing models like a portfolio, not a religion

AI model routing and fallback: separate decisions, shared evidence

How to route enterprise AI workloads between models, design fallbacks and measure both paths without conflating their purpose.

AI architectureModel routingReliability

9 June 2026

Start of a series, part 1 of 4

Choosing models like a portfolio, not a religion

A practical way to select and govern a small portfolio of models, with clear routing, substitution and exit decisions.

AI strategyModel governanceEnterprise architecture

4 June 2026Engineering

One governed toolset, three assistants: MCP servers as enterprise infrastructure

Four assistants, one management surface, and the layer that decides what any of them may actually do.

Model Context ProtocolAgent tool designEnterprise authorisationMulti-tenancyAudit and observability

21 May 2026Research

Running an evaluation harness as production infrastructure on AWS

Evaluation suites decay rather than fail. Versioned corpora, a run manifest, separated scoring stages, a queryable history and a release gate on AWS.

Evaluation harness infrastructureCorpus versioning and reproducibilityDeterministic assertions and rubric gradingRelease gates and reliability floorsEvaluation cost modelling

7 May 2026Research

Inside the agent evaluation arena: how we score autonomous systems

We built a simulated contact centre to pit multi-agent language systems against reproducible tasks. Here is the harness, the metrics that actually predict production reliability, and where today's models keep tripping.

Agent evaluationSimulation testingReliabilityMulti-agent systems

9 April 2026Engineering

Building a retrieval pipeline that holds up in production

Chunking, embedding and retrieval on Postgres and pgvector that stays fast and accurate once real documents and real users arrive.

RAGpgvectorPostgresRetrieval

12 March 2026Engineering

Testing language-model features before they meet users

A practical evaluation method for language-model features: representative scenarios, calibrated graders, adversarial testing and release evidence that survives change.

LLM evaluationRed-teamingRegression testingRelease assurance

20 February 2026Perspectives

A practical framework for enterprise AI governance

Governance is a design problem before it is a compliance one. Here is how we build guardrails that engineering teams actually use, grounded in ISO 42001 and the direction Australian regulation is now taking.

AI strategyModel governanceAI operations

14 September 2023Services

Using simulation tooling to train and assure AI agents

How Humint Labs has used custom simulation apps across delivery engagements to train and assure chat, messaging and voice agents, from early feasibility work through to repeatable release evidence.

AI evaluationCustom AI toolingAgent testingEnterprise AI delivery

Insights and Perspectives

Featured series

Where a decision model fits in enterprise service workflows

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

Prompt injection when the model has tools

Choosing models like a portfolio, not a religion

How we tested decision models

Putting decision models to work in the contact centre

How decision models compare in the contact centre

How we tested the Jev decision model, step by step

Where a decision model fits in enterprise service workflows

The operating model for a custom AI agent

Copilot data governance readiness in a Microsoft estate

Where Copilot stops and a custom agent starts: drawing the line in a Microsoft estate

Testing prompt-injection defences with release evidence

Prompt injection provenance and capability controls

Tracing a conversation end to end: model calls, guardrail decisions and routing

Prompt injection when the model has tools

AI model lifecycle and deprecation: plan the exit before it is urgent

Orchestrating a multi-agent claims workflow: topology, handoffs and blast radius

Testing AI model substitution before a provider change forces it

AI model routing and fallback: separate decisions, shared evidence

Choosing models like a portfolio, not a religion

One governed toolset, three assistants: MCP servers as enterprise infrastructure

Running an evaluation harness as production infrastructure on AWS

Inside the agent evaluation arena: how we score autonomous systems

Building a retrieval pipeline that holds up in production

Testing language-model features before they meet users

A practical framework for enterprise AI governance

Using simulation tooling to train and assure AI agents

Ready to ship Enterprise AI?