Get in touch
Open to principal-scope roles · Phoenix, Arizona · Remote or hybrid
Open to principal-scope roles · Phoenix, Arizona · Remote or hybrid
Playbook 01 · From a demo that works to a system people depend on
Agents, grounding, evaluation, and governance: the questions I ask, the Azure build in order, and what it looked like when it was real.
Tap a tool for a quick explanation.
What is it?
Microsoft's cloud identity service for authenticating users and workloads and managing access.
Think of it as
The identity desk that checks who you are.
What is it?
The earlier name for Microsoft Foundry, a platform for developing, evaluating and operating AI applications and agents.
Think of it as
An AI development workshop with evaluation and management tools.
What is it?
A managed search service for indexing and retrieving information, including keyword and vector search.
Think of it as
The retrieval catalogue, not the model that writes the answer.
What is it?
An analytics platform bringing together data engineering, integration, warehousing and business intelligence.
Think of it as
A shared workshop for the data-to-insight journey.
What is it?
Runs event-driven code without requiring you to manage the underlying servers directly.
Think of it as
Small pieces of code triggered by events.
What is it?
Managed container hosting for services and jobs, with scaling options and less direct cluster administration.
Think of it as
Run the container without operating your own Kubernetes cluster.
What is it?
A managed gateway and management layer for publishing APIs and applying policies to their traffic.
Think of it as
The controlled front door to your APIs.
What is it?
Stores and controls access to secrets, cryptographic keys and certificates.
Think of it as
A locked cabinet for application credentials.
What is it?
Application performance monitoring within Azure Monitor, using telemetry to investigate requests, dependencies and failures.
Think of it as
Follow a request to see where it slowed down or failed.
What is it?
Microsoft's family of data governance, security and compliance capabilities, including discovery and classification.
Think of it as
Know what data you have and how it should be handled.
What is it?
Microsoft's security monitoring and investigation platform, bringing together security signals and incident response.
Think of it as
The security operations desk.
An agentic system is software that plans, retrieves, decides, and acts, with a model somewhere in the loop. The model is the least reliable component in that chain, which is why the architecture around it matters more than the model inside it. This playbook is the sequence I use to take a system from a working demo to something a regulated organisation can run: the questions first, then the build, then the proof.
I’ve shipped this twice in the last stretch: a complete agentic system on Anthropic’s Claude platform, inception through delivery, and multi-agent workflows in Azure AI Foundry grounded over governed Azure data. The patterns below are what survived both.
These get answered in writing before any orchestration code exists. If a team can’t answer them, we aren’t ready to build, and that’s useful to know on day one rather than day sixty.
The order is the method. Each step assumes the ones before it are in place, and most failed agent projects I’ve reviewed skipped straight to step six.
The same sequence holds on other platforms. On the Claude platform build, steps four and five were Anthropic’s APIs and my own tool layer; everything else was identical, because everything else is architecture, not vendor.
Identity first because the most common enterprise AI failure I’ve seen isn’t a hallucination. It’s an assistant returning data a user was never cleared for, through a service principal that was given Contributor during the pilot and never revisited.
Governed data before retrieval because an index over ungoverned data produces fluent, confident, wrong answers, and nobody can trace them. Retrieval with provenance is the difference between an answer and a rumour.
Typed tools and idempotency because agents retry. That is how they work. If a tool sends an email and the first call succeeded, you now have two emails.
The harness in CI because every change to an agentic system has an unknown blast radius. Without a regression suite you aren’t iterating; you’re gambling and calling it iteration.
Traces, not logs because logs tell you what the agent did. Only a trace tells you why. The reasoning happened at runtime and evaporated unless you captured it.
A kill switch because agents fail fast, repetitive, and confident. By the time a person notices, the agent has done the wrong thing forty times. A rollback takes ten minutes if everything goes well. A switch takes ten seconds.
Clinical intake on a HIPAA-aligned EMR. Documents arrive in five languages: referrals, labs, handwritten notes, scans. The machine does the reading: OCR, extraction with confidence scores, candidate patient matches. People do the deciding: anything below threshold goes to a reviewer with the source region highlighted, name matching proposes but never merges, and the source document stays immutable forever. The system holds 250+ active patient records. The architecture’s job was to make sure a guess could never be mistaken for a fact.
Multi-agent workflows in Azure AI Foundry. Enterprise scenarios advanced from design to working proof of concept with RAG grounding over governed Azure data, multi-agent orchestration, and tool integration against real systems of record. The hard part was never the agents. It was the governed data layer underneath them and the identity model that let them run as the user.
A complete agentic system on Anthropic’s Claude platform. Owned every milestone from inception to delivery: agent boundary design, orchestration, tool and function integration, an evaluation harness with a written failure-mode catalogue, and production hardening (guardrails, fallback paths, observability, cost controls). Shipped, not demoed.
The pilot picks the platform by accident. The service principal has standing admin rights. The index was built once from a demo folder. One model does everything and nobody can quote the most expensive run last week. The human in the loop is an Approve button nobody designed, so the human stops reviewing and starts approving. The prompt was last edited in a console on a Friday afternoon. Every one of these maps to a gate in the method, which is why the gates exist.
A signed gate record per gate. The agent map. The tool contracts. The evaluation harness and failure-mode catalogue in the repo, running in CI. A cost model per run with the ceilings set. The runbook with the off switch tested in front of the owner. And exit criteria for the pilot, with a date, written before the architecture.
Every playbook here follows the same five moves: Scope, Harden, Anchor, Measure, Sustain. The SHAMS Method is where they come from, and the gate check is how you find out which one your project is missing.
Next
Assessment, the 6R decision, migration sequencing, and the cutover that is really a change of ownership.