An offline baseline that remains useful

examples/baseline.mjs uses Node.js 22+ and built-in libraries. The example README records the checked environment and commands. There is no installation, lockfile, API key, or network because there is no external dependency. The parser accepts narrow Key: value text. It is not OCR, a PDF reader, or a general-purpose document parser. That limitation is an experimental advantage: we can observe what simple rules catch before testing whether a model adds value.

node --test examples/check-contracts.test.mjs
node examples/baseline.mjs examples/fixtures/invoice-valid.txt acme
node examples/baseline.mjs examples/fixtures/invoice-invalid.txt acme
node examples/baseline.mjs examples/fixtures/purchase-order.txt acme

These are reproducible commands, not a fabricated transcript. The chapter's verification record should contain the actual session results. The second argument stands in for an authenticated tenant supplied by a trusted caller. A real API must derive it from authentication and check authorization. Typing it into a CLI does not recreate that boundary.

A valid invoice yields needs_review, fields invoice_id, amount, and currency, plus source line and rule for each field. A malformed line yields contract error and a nonzero process status. A purchase_order may have a valid tenant and type but no supported extraction rules; it yields needs_review, fields: {}, and an “unsupported document type” issue. An explicit fallback is better than plausible unsupported fields.

{
  "tenant": "acme",
  "documentType": "purchase_order",
  "status": "needs_review",
  "fields": {},
  "issues": ["unsupported document type: purchase_order"],
  "provenance": {}
}

The JSON illustrates the contract for the corresponding fixture; inspect the actual output when running it. A three-letter currency code validates syntax, not whether it is valid or appropriate. Two decimal places validate amount syntax, not line-item arithmetic or an agreed price. A source line explains where text came from, not whether the document itself is genuine. A real review screen should show the original beside the proposal, allow correction, and preserve a versioned record.

What this toy cannot establish

We still need real authentication, safe upload, a durable queue, idempotency, a transactional store, retention policy, pilot consent, access control, log redaction, and an incident owner. We also need a labeled set that reflects permitted real documents. A passing local test suite verifies the local contract; it does not measure time to review, job loss, or model quality. Those gaps determine the next work. A diagram does not close them.

Three reversible decisions before selecting a model

ADR-001: rules remain the baseline

ADR-001 chooses rules for narrow invoices with human review. Alternatives are adding AI on day one or asking staff to type every field manually. Rules require maintenance by document type and fail on broader variation. Manual entry consumes operator time. AI could expand coverage, but introduces variable costs, evaluation work, data handling, and incorrect outputs. Revisit when: the unsupported share, correction time, and per-field error rate have been measured on a consented set. If rules create too much work and AI improves the task while preserving budget and integrity, test it behind the same contract. Rollback means switching AI proposals off and keeping the baseline and review queue. We have no measurement that already warrants the change.

ADR-002: start with a modular monolith

ADR-002 favors clear API and worker module boundaries with an initially simple store rather than independent services. Services may scale components and teams separately; they also add deployment, network, distributed contracts, and operating work. A monolith may suffer contention between import and extraction. Revisit when: a representative load test measures receipt p95, queue depth, resource use, and correlated failures. Separate a service only when scale or failure isolation addresses a measured bottleneck better than a separate worker process or capacity adjustment. Reversibility calls for stable module contracts and versioned messages. No load graph exists yet.

ADR-003: autonomy stops at proposal

ADR-003 assigns tenant, authorization, schema, state transitions, budget, and effects to backend code; approval belongs to a human. A fixed classify → extract → validate → review workflow is sufficient initially. An agent choosing tools dynamically makes sense only if variable tasks show measurable gains and the runner enforces step, time, permission, and cost limits. Automatic approval might reduce human labor, but it increases risk at the point of consequence. Revisit when: proposals have been compared with blinded human review and correction rates by category. Never infer system integrity from a model score. Rollback means stopping model calls and leaving items pending. Anthropic's workflow/agent distinction helps name the options; it does not prescribe autonomy for RelayOps.

Each ADR records rejected options, costs, reversibility, and a condition for reopening the choice. A decision record is useful when evidence can change a future decision. Otherwise it is decorative history.

Four failure modes that can masquerade as architecture

1. A diagram with no decision criterion. Symptom: API, queue, and database boxes have arrows, but nobody knows why a queue is needed or when to split a service. Cause: treating a representation as a decision. Response: attach a quality scenario to each major component. Here, a queue would protect an acknowledged import from disappearing; it still needs an ack/retry test. If a validated load shows synchronous processing is enough, the queue can wait. Operators care about a trustworthy receipt and a visible item, not the number of boxes.

2. Selecting AI before writing requirements. Symptom: a team buys model capacity, then asks how to handle an unrecognized document. Cause: mistaking linguistic flexibility for a requirement. Response: run the baseline, label unsupported types, measure review cost and per-field errors. Introduce AI as a hypothesis for reducing measured work. It might not be worthwhile: if most traffic fits rules and review is quick, integration cost may exceed the benefit.

3. Equating model quality with system integrity. Symptom: demos show the right amount and ID while tenant or item version is wrong. Cause: assessing an answer in isolation and omitting boundaries. Response: adversarial tests for cross-tenant access, illegal transitions, duplicates, and failure after ack; human review for content; schema and provenance metrics separate from accuracy. An excellent model can feed a store that accepts a wrong key. A secure API can still show an incorrect value that needs correction. Measure both layers independently.

4. Architecture without a budget or operations owner. Symptom: the plan permits unbounded calls, with nobody owning caps, incidents, or retention. Cause: treating cost and operations as an appendix. Response: set per-tenant and per-window limits before paid calls; account for review time, storage, and investigation as well. Name who changes a limit, receives an alert, stops processing, and replays work after a failure. Without owners, a “scalable” design can scale only the bill and the error queue.

An evidence plan that can change a decision

The brief maps each hypothesis to a discriminating test, owner, and unresolved risk. It also orders the work. First, submit a document whose tenant disagrees with the caller and prove local rejection. Then build authentication and a store, and repeat with two actual identities. In parallel, build a labeled synthetic set for rules and review before obtaining consented pilot documents. Measure receipt-to-visible-item latency over an explicit window. Inject failure between acknowledgment and persistence; reconcile jobs. Simulate a spending cap without real billing. Finally, test a load that could distinguish internal modularity from services.

Evaluation needs two ledgers. Task quality: eligible documents, correct fields against adjudicated labels, human correction time, and share sent to review. System integrity: cross-tenant attempts rejected, duplicate effects, jobs without terminal state, illegal transitions, and cost per tenant. A scripted stub can test a runner and contracts; it cannot measure the accuracy of a real model. Synthetic fixtures help reproduce bugs but cannot stand in for market document diversity.

A failed measure should reopen a specific decision. High unsupported share may reopen ADR-001. Bad p95 under worker contention may reopen ADR-002. Frequent correction plus high cost may keep ADR-003 restrictive. A small or biased sample yields no conclusion: enlarge or change the test instead of filling the gap with an opinion.

Cost, security, and a path back

For cost, separate provider fees, compute, storage, and operator minutes. Model prices change; any future simulation needs a date, currency, charging unit, cache assumptions, and sensitivity to volume. Without a pilot, “cost per document” is a scenario, not data. The spending ceiling belongs in the backend: when it is reached, an item must remain available for manual review rather than triggering attempts to evade the cap.

For security, this chapter uses synthetic fixtures only. A future pilot needs explicit consent, less collected PII, redacted logs, retention and deletion decisions, scoped queries and cache keys, and purpose-bound access records. We make no compliance claim. Instructions embedded in a document are untrusted content, never authorization. Backend code must not accept a tenant selected by document text or a model-generated tool call.

For reversibility, retain the baseline, version schemas and proposals, preserve origin, and separate side effects. Turning off AI should leave a needs_review queue, not lost work. Approval and external dispatch are different steps; this chapter dispatches nothing. A future rollback requires knowing which version processed each document, who approved it, and which effects happened. The local CLI does not implement that ledger.

A decision checklist before a first pilot

• Operator, potential buyer, and affected parties identified without claiming validated demand.
• Tenant derived from authenticated context; document content grants no access.
• Proposal, validation, review, and effect have distinct owners and permissions.
• Each priority quality attribute has stimulus, environment, response, measure, window, and denominator.
• Scenario numbers labeled as targets; results have traceable commands and samples.
• A non-AI alternative runs and remains a control.
• Unsupported documents have a visible fallback; malformed contracts fail clearly.
• ADRs contain alternatives, cost, revisit triggers, and rollback.
• Budget, retention, incidents, and replay have named owners.
• Pilot awaits authorization, consent, and isolation tests with real persistence.

Exercises with checkable outcomes

Basic: write an isolation scenario

Write a scenario for a caller authenticated to acme who submits a document declaring Tenant: other. Specify stimulus, environment, response, and measure. Pass condition: another engineer can turn the text into a test checking rejection and no writes in either tenant.

Worked answer: “Given an API authenticated as acme and a store with two organizations, when an import declares other, the API returns a contract error before enqueueing; write counts for acme and other remain zero for that request.” The current CLI can only test local rejection. node examples/baseline.mjs examples/fixtures/invoice-valid.txt other uses the fixture declaring Tenant: acme with other as the expected context, so it fails. To prove zero writes, implement and instrument a store in a later chapter. Do not describe the absence of a database as a passing database test.

Intermediate: add an unsupported document

Create a synthetic fixture with Tenant: acme and Type: purchase_order, or another unknown type. Run the CLI. Confirm status: needs_review, fields: {}, and an issue naming the type. Pass condition: no invoice fields are invented and nothing is approved automatically.

Worked answer: examples/fixtures/purchase-order.txt is an existing input. Run the command in the README. The type !== 'invoice' branch returns an explicit issue. That proves parser fallback, not that an operator can resolve the document without an interface and training. To estimate value, label a batch of orders and measure manual review time.

Advanced: try to disprove the modular monolith decision

Define a load envelope and an experiment that could show ADR-002 is wrong. Pass condition: state volume, duration, import p95, backlog, CPU/memory, correlated failures, a decision threshold, and a cheaper alternative to try before splitting services.

Worked answer: a hypothetical test might say: “Over one hour at 20 synthetic imports/s with two workers, if receipt p95 exceeds 500 ms due to extraction CPU contention despite separate pools, evaluate an independent worker service.” These are teaching assumptions, not observed RelayOps throughput. First test whether a separate process in the same deployment and a concurrency cap solve the problem. Services become compelling when repeated measurements, operating cost, and ownership favor separation. Never cite the hypothetical number as a benchmark.

Glossary and next step

Tenant: organization whose context determines data access. Provenance: traceable origin of a field, such as a line and rule. ADR: decision record with alternatives and revisit condition. SLI: measured service indicator; SLO: target value for an indicator. Idempotency: repeating an operation without duplicating its effect. Trust boundary: point where data crosses different levels of trust or permission.

This chapter leaves a design that tests can contradict. The next chapter develops identity and isolation: tenant and document enter a real transaction, and the synthetic cross-tenant attempt becomes more than a text example. The series index organizes that planned path; it does not establish publication of a future chapter. For now, the most useful decision remains runnable offline: extraction can help people without receiving approval, access, or spending authority.

An honest architecture review must also record what would change this proposal. If consented real documents fall outside the parser, the non-AI baseline remains a useful control, but its coverage must be measured by type and field. If the proposed queue fails to improve review time under representative load, simplify the design. If the financial cap blocks necessary review, operations must renegotiate the budget and manual path before enabling more paid calls. If persistence isolation fails, no extraction result justifies a pilot. The next ADR should follow that evidence, with an owner and review date, instead of keeping components because they look coherent on a diagram.

Primary references

• AWS Well-Architected Framework, accessed September 27, 2026: questions for examining decisions and trade-offs, not certification of this design.
• Google SRE, Service Level Objectives, accessed September 27, 2026: a method for defining indicators and objectives around user experience; RelayOps scenarios remain targets.
• Anthropic, Building effective agents, published December 19, 2024 and accessed September 27, 2026: distinguishes workflows from agents; the article itself warns that some tooling has changed since 2024.