Case intake from 4 days to 11 minutes with a multi-agent workflow

How we measured. These are client-reported operational figures from the firm's own case management records over the post-launch period, not a controlled study and not our instrumentation. Intake processing time is wall clock from documents landing in the queue to a structured case record ready for attorney review; the four-day baseline is the firm's own pre-project figure for the same span. The 100% review figure is the one number we assert independently, because it is a property of the system: no code path files a case without a human confirming it, and the audit log records who confirmed what.

What we are not claiming. No revenue, case win rates or approval outcomes. We have no visibility into any of those and will not attribute them to an intake system. Eleven minutes is an observed operating average, not a latency guarantee — fifty scanned pages in three languages take longer than six clean PDFs.

*Can a law firm let AI agents read client documents without losing control of what goes into a case file?*

A single family-based case can arrive as a hundred pages of passports, birth certificates, tax transcripts, pay stubs and prior filings — scanned crooked, photographed on phones, in more than one language. A paralegal reads all of it and types the facts into the case system. That is where four days went.

Every vendor will sell that firm a document AI. Firms do not buy, and the reason is not accuracy scepticism. A wrong date on an I-130 is not a bad user experience, it is a filing problem, and no partner accepts a system that cannot show where a value came from.

The output is typed, not textual. Nobody needs a summary of a birth certificate. The case system needs date_of_birth, country_of_birth, certificate_number — right type, right format, or nothing.

Confidence had to be per field, not per document. A passport page yields a perfect number and an unreadable expiry date. Scoring the document as a whole either blocks clean fields or waves through bad ones.

"Human in the loop" is usually theatre. Most implementations show a reviewer pre-filled fields and a Confirm button. Reviewers confirm. Review had to direct attention to the fields that need it and make checking cheaper than retyping.

Audit is a product requirement. Every value needs its source: which document, page, agent, model version, timestamp, and which human confirmed it.

Five agents with narrow jobs and a hard gate, not one general agent with tools. Narrow agents are testable; a general agent is a demo.

flowchart TD
    A["Client documents<br/>portal upload, email, scan"] --> B["Ingestion agent<br/>OCR, page split, dedupe, orientation"]
    B --> C["Classification agent<br/>document type + language per page"]
    C --> D{"Type recognised?"}
    D -- no --> R["Unclassified queue<br/>paralegal labels it"]
    D -- yes --> E["Extraction agent<br/>typed schema per document type"]
    E --> F["Confidence scorer<br/>field-level score + page citation"]
    F --> G{"Every required field<br/>above threshold?"}
    G -- no --> H["Low-confidence queue<br/>flagged fields surfaced first"]
    G -- yes --> I["Case assembly<br/>merge + cross-document conflict check"]
    R --> C
    H --> J["Human review gate<br/>paralegal confirms or corrects"]
    I --> J
    J --> K["Attorney sign-off"]
    K --> L["Case management system"]
    B -.-> M[("Audit log<br/>field, source page, agent,<br/>model version, actor, timestamp")]
    E -.-> M
    F -.-> M
    J -.-> M

Two properties matter more than the boxes. The review gate has no bypass — assembly always terminates at review, whether confidence was high or low. And every arrow into the audit log is append-only, so the record of who changed what survives later edits to the case.

1. Blueprint: wrote the schema before writing the agents. We started from the case system's required fields and worked backwards to a typed schema per document type — passport, birth certificate, marriage certificate, tax transcript, I-94, prior notices — with formats, required flags and a validation rule each. An extraction agent with a strict target schema is a solvable problem. One without is a chat window.

2. Split classification from extraction. Classification decides what a page is; extraction decides what it says. Separating them means a misfiled page fails loudly instead of quietly producing plausible garbage in the wrong schema. Unrecognised pages go to a queue a paralegal clears in seconds, and those labels feed the eval set.

3. Scored confidence per field, with provenance. Each field carries a score and a citation — document, page, region. Above threshold, it passes to assembly. Below, it is flagged and pushed to the top of the reviewer's screen with the source page rendered beside it. The reviewer looks at the document, not at the model's opinion of it.

4. Built the review UI as the product. Flagged fields first, source image beside the value, keyboard-only correction, visible count of what remains. Correcting a field is two keystrokes. That is why review happens instead of being rubber-stamped.

5. Cross-checked documents against each other. Names and dates repeat across a case. Assembly compares them and raises a conflict on disagreement — exactly what a tired human misses at 6pm and a machine catches for free.

The eval set. Real cases the firm had already processed by hand, held out, spanning the document types in scope and weighted toward the ugly end: phone photographs, skewed scans, faint stamps, non-English documents, multi-page records split across files. Hand-keyed values from existing case records were ground truth.

The metric that mattered. Not document-level accuracy — field-level exact match on required fields, split by whether the field passed the confidence gate. Two numbers drove every decision. Precision above the gate: of the fields passed as high confidence, how many were exactly right — the number that governs trust, because these are the fields a reviewer skims. Flag rate: what share got pushed to the low-confidence queue — the number that governs cost, because a system that flags everything is accurate and worthless. We tuned the threshold to push errors out of the first and into the second.

Latency. Intake is batch, so we optimised end-to-end wall clock per case, not token latency. The path with a percentile budget is the review UI: page image and field must render fast enough that a reviewer never waits between fields, because the moment they wait they start confirming without looking.

Failure modes, and what happens when the system is wrong.

The system's job is not to be right. It is to be checkable, and to be wrong in ways a paralegal catches in seconds.

The tradeoff. We chose a higher flag rate over a higher pass rate. That costs review time on every case, forever. We took it because a confidently wrong date reaching a filing is not symmetric with a paralegal checking one extra field.

The alternative we rejected. Fine-tuning one model to read every document type. It benchmarked reasonably and we walked away, because a fine-tuned monolith gives no per-field confidence, no provenance, and no way to fix one document type without retraining the thing that reads all the others.

What broke. Multi-page documents split across separate uploads. The classifier scored pages in isolation and confidently mislabelled continuation pages carrying no header — page one of a tax transcript is obvious, page four is not. We made classification page-sequence aware and grouped pages during ingestion, but it shipped wrong first, because our early eval documents were all single-file.

Build the review UI first. We built the pipeline, then the review screen. That is backwards. The review screen is where the threshold, the flag rate and the economics become visible.

Design the unclassified queue as a labelling surface. It started as a spillover bucket and became an important feedback source. It should have fed the eval set automatically from day one.

Ship our own timing telemetry. The headline figures are the client's, and we report them as theirs — but an independent end-to-end number should have existed in release one.

The engagement continues: extending the typed schema to more document types, tightening confidence on handwritten and stamped documents, and feeding reviewer corrections back into the eval set so the flag rate falls without the error rate rising.