Open to principal-scope roles · Phoenix, Arizona · Remote or hybrid
Architect / Enterprise agent studio
I start with a business outcome and build the smallest system that can deliver it. Follow two enterprise designs from the first decision to a monitored release, using the SHAMS Method.
Interactive teaching studio · synthetic data · no live AI calls
01 · Choose the design
I keep architecture and authority separate. A more capable model does not earn more permission, and more agents do not automatically complete more work.
Proposed starting architecture
Ingest, extract, validate and stage are known steps. Start with a fixed workflow; add dynamic tool choice only when exceptions require it.
Payment and vendor-master changes remain outside this design.
Compare with the SHAMS autonomy ladder →| Pattern | When I choose it | Control that stays outside the model |
|---|---|---|
| Rules or conventional automation | Inputs and decisions are structured and stable. AI adds no measured benefit. | Validation, access checks and transaction rules. |
| Grounded assistant | Find evidence, explain it and cite it. A chat interface alone does not imply an agent. | Document access, source freshness, answer checks and abstention. |
| AI workflow | The sequence is known; extraction, classification or drafting benefits from models. | Routing, required fields, retries and release criteria. |
| Bounded tool agent | The next useful action depends on observed results; a fixed path performs poorly. | Tool allowlist, per-call authorization, step budget, stop and approval. |
| Coordinator + specialists | Independent work or genuinely different contexts justify delegation in evaluations. | Separate identities, narrowed permissions, typed handoffs and one outcome owner. |
These are design patterns, not a ranking. Computer use is a tool modality, not an authority level. Prefer scoped APIs where possible; a desktop operator needs an isolated session, constrained credentials and explicit action boundaries. Pattern source.
02 · Build through SHAMS
These are five lenses I revisit throughout delivery. The line connects the reading journey; it is not a runtime call or a ladder of increasing autonomy.
S · Scope · Gate 01
Prepare a checked invoice draft and its evidence for an accounts-payable reviewer.
Compare against manual entry plus OCR and deterministic validation on the same documents. Count accepted records, corrections and reviewer minutes, including exceptions.
An owner, a task boundary, a manual or rules-based baseline, a definition of done and a list of unacceptable outcomes. Agree on a pilot end date before building.
H · Harden · Gates 02, 03, 05
One tenant, an approved document collection and a staging ledger. Read purchase orders and vendor records. Never issue a payment, change bank details or approve your own output.
Enforce these rules at retrieval and tool execution. A prompt, skill, MCP connection or approval screen cannot replace server-side authorization. A delegated agent receives the intersection of its own scope and the caller's scope, never broader inherited powers.
Identity and data-flow diagrams, tool schemas, input validation, reviewer roles, expiration rules, idempotency, a retry policy and a tested stop path.
A · Anchor · Gate 04
Quarantine and scan uploads; validate file type and size; classify the document; extract fields and table structure with page coordinates. Preserve the original and its content hash under the retention policy.
Check required fields, duplicate invoice keys, totals, currency and purchase-order references in code. Retrieve only permitted policy passages. Treat document text as evidence, never as a new instruction or permission.
Invoice INV-DEMO-104, page 1; purchase order PO-DEMO-28; current invoice policy with owner, version and review date. All records in this lesson are synthetic.
Separate the source of truth, the search index, the current context and stored conversation state. Each has its own access and deletion rules. Re-check permission on later turns; do not let a cache or memory reintroduce revoked material.
A source register, ingestion and deletion pipeline, permission propagation tests, citation format, freshness rule and an explicit insufficient-evidence outcome.
M · Measure · Gates 06, 07
Measure field exact-match and document-level acceptance on held-out vendors, layouts, scans and languages. Calibrate review thresholds against observed field errors. Extraction confidence is not proof that an invoice is legitimate.
Compare the full system with its baseline on the same held-out tasks. Repeat variable model trials. Check the final business state in code, use domain reviewers for judgment, and calibrate model graders against those reviewers. Report denominators and failures by slice.
| Test condition | Required observable outcome |
|---|---|
| Unreadable total | Hold for correction; preserve the original page and missing-field reason. |
| Duplicate invoice | Reject the duplicate staging key; do not create a second record. |
| Planted payment instruction | No payment capability exists; deny the attempted tool and record the event. |
| Write timeout | Reconcile by idempotency key before retrying; report unknown until verified. |
Total cost per accepted outcome = model + extraction + search + infrastructure + retries + human review, divided by accepted outcomes. If none succeed, report no successful outcome, not a zero cost.
A versioned test set, agreed quality and latency thresholds, a risk review, unit economics and release criteria. No universal confidence score or benchmark can approve this system for your estate.
S · Sustain · Gate 08
Watch correction rate by field and vendor, exception queue age, duplicate prevention, extraction drift, receipt reconciliation and total cost per accepted record.
Progress from offline tests to shadow operation, then a limited pilot and a staged release when agreed gates pass. Shadow runs do not execute business writes. Pin the prompt, model deployment, retrieval configuration, policy and tool schema together.
An on-call owner, incident thresholds, a review queue with an escalation time, versioned release evidence, rollback instructions and a tested kill switch. Stopping an agent does not undo completed external actions; reconcile them separately.
| Test condition | Required observable outcome |
|---|---|
| Unreadable total | Hold for correction; preserve the original page and missing-field reason. |
| Duplicate invoice | Reject the duplicate staging key; do not create a second record. |
| Planted payment instruction | No payment capability exists; deny the attempted tool and record the event. |
| Write timeout | Reconcile by idempotency key before retrying; report unknown until verified. |
| Test condition | Required observable outcome |
|---|---|
| Another employee's document | Exclude it before model context; do not reveal its title or existence. |
| Missing policy | Abstain and name the supported escalation route. |
| Access revoked mid-run | Re-check authorization; refuse the write despite earlier approval. |
| Request to grant access | Offer the request process; the grant operation is not an available tool. |
03 · Follow the work
Move through a synthetic run and inspect the public record. Change a condition to see where it stops. This demonstrates enforcement logic; it does not test a real model or cloud deployment.
Write-related conditions apply when the design includes a business write. Call counts are illustrative tool calls in this simulation, not a cloud cost estimate.
Enterprise runtime · example architecture
● Solid ink: evidence⇢ Cobalt dashed: control⋯ Cobalt dotted: reviewDashed enclosure: boundary
Instruments represent software. The solid figure is a person. The illustration is a teaching abstraction, not an infrastructure inventory.
Step 1
READY
Validate the caller and attach the permitted tenant and task scope before accessing evidence.
identity.verifiedSimulate the authorized reviewer. This decision applies only to the displayed synthetic proposal.
stage_invoice_draft(invoice_id=INV-DEMO-104, purchase_order=PO-DEMO-28)Only events reached so far appear here. Statuses are simulated. This is application evidence, not hidden model reasoning.
04 · Map it to a platform
This is a reference architecture I would adapt to the client's estate. The model is a replaceable component inside a governed application, with a documented processing boundary.
| Control or capability | Azure reference implementation | What still needs to be designed |
|---|---|---|
| Employee and service identity | Microsoft Entra ID; managed identities for supported service calls. | Delegated user scope, tenant isolation, tool identity and separation of reviewer duties. A service identity is not the employee's permission. |
| Document intake | Quarantine storage, scanning integration, Azure Blob Storage and Document Intelligence. | File validation, rights to process, page provenance, retention, schema checks and human correction. Source. |
| Authorized retrieval | Azure AI Search with an application-enforced security-filter pattern, or a supported document-permission feature after verification. | Trusted principal resolution, mandatory tenant/ACL filters, revoke/delete propagation and permission-aware caches. Do not accept an ACL supplied by the model. Source. |
| Application and orchestration | A versioned service on Container Apps or App Service; queues and durable state where needed. Foundry Agent Service is an option when its verified capabilities fit. | Business state machine, typed handoffs, bounded loops, cancellation, per-run budgets and durable approval state. Verify any managed-runtime tool, network and model combination before selecting it. |
| Model adapter | Approved, pinned Azure model deployment. | Evaluate prompts, tool schemas and output validation per model. Provider adapters do not make different models behaviorally equivalent. |
| Protected actions | Application-owned tool gateway, optionally fronted by API Management; scoped business APIs. | Authorize immediately before execution, validate arguments, bind review to a payload hash, expire approvals, use idempotency and reconcile receipts. MCP can expose tools; it does not grant authority. |
| Observation and release | OpenTelemetry application spans, Azure Monitor and Application Insights; versioned evaluation and release pipeline. | Redaction, retention, access to traces, outcome checks, review alerts, cost attribution and rollback drills. Instrument policy decisions as well as model calls. Source. |
Claude in Foundry has Azure-hosted and Anthropic-hosted options, with different processing boundaries and per-model availability. Anthropic is the seller and operator for both. Confirm the exact hosting version, region, terms and lifecycle; do not infer data residency from the Azure portal. Configure applicable inference-time content safety instead of assuming deployment-time filtering. Availability · Data processing.
I preserve application authorization, evidence provenance, tool contracts and approval records. I replace the model adapter, check message and tool-use semantics, rerun the evaluation suite and verify the selected endpoint's data-processing terms. Direct Anthropic API access is a separate procurement and data-flow decision from either Foundry hosting option. A model fallback also needs approval for its data boundary and a passed evaluation; it must not silently reroute sensitive content.
A specialist architecture needs independent evaluation of delegation and result aggregation. Specialists get bounded work orders, limited context and a shared run identifier. Only the authorized execution service can commit a business change. The coordinator cannot manufacture permission by asking a different specialist.
05 · Observe and improve
I ask which part failed before deciding what to train. A thumbs-up, a longer prompt and a model upgrade are different interventions. None bypasses release evaluation.
Diagnose first
Label adjudicated extraction errors and train a new custom document model when justified. Hold out unseen layouts and suppliers. Rebuild the index for policy changes. These are separate changes from fine-tuning a language model.
Repair approved content, permissions or indexing. Re-evaluate retrieval and answers. This updates evidence, not language-model weights.
| Change | What changes | Evidence I require |
|---|---|---|
| RAG / knowledge update | Approved source material, chunks, metadata or retrieval. Model weights stay unchanged. | Retrieval recall, freshness, citations, deletion and permission propagation. |
| Prompt, skill or tool update | Instructions, available procedures, schemas or executable code. | Task outcomes, argument validation, error handling and regressions. |
| Conversation memory | Application-managed state loaded into a later context. | User/tenant scope, provenance, expiry, correction and deletion. It is not automatic model learning. |
| Custom document-model training | An extraction or classification model learns from labeled documents. | Held-out layouts, field errors and representative document slices. Keep train and evaluation sets separate. |
| Language-model fine-tuning | Weights are adapted, only if the selected provider and deployment support it. | Data rights, privacy review, documented train/eval separation and a measured advantage over simpler changes. Do not assume Claude fine-tuning is available in the chosen deployment. |
Log correlation IDs, versions, source identifiers, policy decisions, redacted tool arguments and verified results. Capture sensitive payloads only under an explicit access and retention policy. Traces show observed behavior; they do not reveal a model's complete internal reasoning or prove the answer correct.
06 · Take it to review
Keep the choices, boundary, evidence, tests and operating plan together. The record labels unresolved decisions; it is not a signed approval or a claim that a system has been deployed.
Inputs stay in this page's memory until you download them or leave. No storage or model call is used by this studio. Avoid entering confidential client details.
Enable JavaScript to generate a record from your selected design. The complete worked examples above remain available without it.
Sources and limits
The worked designs and SHAMS mapping are my proposed teaching architecture. Product details below were checked on 12 October 2026. Verify the exact service, model and deployment at implementation time.
Distinguishes predefined workflows from model-directed agents; complexity should be earned by the task.
Checked 2026-10-12Evaluate environment outcomes and combine code, model and human graders; model graders need calibration.
Checked 2026-10-12Field confidence and extraction accuracy need interpretation and representative evaluation; custom document training is distinct from LLM training.
Checked 2026-10-12Application-supplied principal filters trim retrieval. String filters are not independently an authentication mechanism.
Checked 2026-10-12Hosting options, model availability and lifecycle vary. Configure applicable content safety during inference; do not assume deployment-time filtering.
Checked 2026-10-12Azure-hosted and Anthropic-hosted options have distinct processing boundaries. Anthropic is the seller and operator for both.
Checked 2026-10-12Foundry tracing can store OpenTelemetry traces in Application Insights. Instrument application checks as well as model calls.
Checked 2026-10-12