# Document Extraction

> The capability that turns unstructured construction documents — invoices, insurance certificates, submittals, contracts — into structured, validated, traceable data.

- Source: https://briq.ai/acu/object/document-extraction
- Department: Data Foundations & AI Practice (https://briq.ai/acu/department/data)
- Catalog code: AIP 302 · Level: Advanced · Track: Intelligence · 12 min read
- Also known as: Intelligent document processing, IDP, Data capture, OCR extraction, Document parsing

## Definition

Document extraction is the capability that converts unstructured or semi-structured documents into structured, validated fields that downstream systems can act on. In construction it targets the paper-native artifacts that dominate the workflow: subcontractor invoices, AIA G702/G703 pay applications, certificates of insurance, lien waivers, W-9s, certified payroll (WH-347), submittals, and delivery tickets. It is not plain OCR, which only renders characters, and it is not a mailbox rule; extraction understands what a field means, validates it against expected structure, and records where on the page each value came from. An extraction is only complete when it carries both the value and its provenance and confidence.

## Why it matters

Construction runs on documents that arrive in every conceivable format, and the cost of manually keying them is enormous and invisible because it is spread across dozens of roles. Extraction moves that labor from people retyping numbers into the reclaiming of exceptions, which is where human judgment actually adds value. The savings are real, but they are secondary to speed and consistency.

The risk it addresses is worse than the labor. A miskeyed invoice amount, an expired insurance certificate accepted as current, or a lien waiver that does not match the payment it releases are the kinds of errors that surface months later as overpayments, uninsured exposure, or lost lien rights. Extraction with validation catches these at intake, when they are cheap to fix, rather than at audit, when they are not.

Extraction is a control point, not just a convenience. Because a certificate of insurance, a lien waiver, and a pay application all gate a payment, structured extraction lets those gates be checked automatically: does the COI cover the required limits and dates, does the waiver reference the correct payment, does the pay application's retainage math foot. The document stops being a formality and becomes an enforced control.

The subtlest reason it matters is trust in everything built downstream. Every dashboard, forecast, and AI answer that touches invoice or payroll data inherits the accuracy of the extraction. A confident wrong extraction accepted as truth propagates silently into job cost, into WIP, and into decisions — which is why confidence scoring and human review of low-confidence fields are not optional refinements but the core of the capability.

## Lifecycle

1. **Intake and classification** — A document arrives — email, upload, portal, scan — and is classified by type before anything is extracted, because an invoice and a certificate of insurance need entirely different field maps. Misclassification here poisons everything downstream, so ambiguous documents should route to a human, not a best guess.
2. **Pre-processing** — Scanned and photographed pages are deskewed, denoised, and split into logical documents (a single PDF often contains an invoice, a waiver, and a COI stapled together). Poor pre-processing is a leading cause of extraction failure that gets misattributed to the model.
3. **Field extraction** — The relevant fields are located and read — vendor, invoice number, amounts, dates, line items, policy limits, coverage dates. Each value is captured with a bounding location on the page and a confidence score, not as a bare string.
4. **Normalization** — Values are cleaned into consistent formats: dates to a standard, amounts to numbers, and vendor names resolved toward canonical entities. This is where extraction connects to the data foundation, because an unresolved vendor name is only half-useful.
5. **Validation** — Business rules run: does the invoice math foot, is the COI within its coverage dates, does the lien waiver reference the matching payment, is the certified payroll within prevailing-wage rates. Validation is where extraction earns its keep and where most real value lives.
6. **Human review of exceptions** — Low-confidence fields and failed validations route to a person who corrects them, ideally seeing the source page highlighted at the value in question. The review interface, not the model, is what determines whether the capability is fast or maddening.
7. **Posting to systems** — Validated, structured data is written to the system of record — invoice to accounts payable, COI to the compliance register — with a link back to the source document so the record is auditable.
8. **Feedback and retraining** — Every human correction is captured and used to improve extraction on the document layouts that produced it. Extraction that does not learn from its corrections plateaus and quietly frustrates its users into abandoning it.

## Anatomy

- **Document classification** — The determined type (invoice, COI, waiver, W-9, WH-347, submittal). Everything downstream branches on this; a wrong class is a silent, total failure.
- **Extracted field values** — The structured outputs — vendor, amounts, dates, policy numbers, line items. The payload, but useless without the fields around it.
- **Confidence scores** — Per-field certainty. The dial that routes a value to auto-post or to human review; the difference between a tool and a trap.
- **Source location / bounding box** — Where on the page each value came from. What lets a reviewer verify in one second instead of hunting through the document.
- **Provenance link** — A durable reference to the original document. Turns the posted record into auditable evidence rather than an orphaned number.
- **Line-item structure** — Tables broken into rows and columns — critical for pay applications and invoices where the total is meaningless without the breakdown.
- **Validation results** — Pass/fail for each business rule (math foots, dates valid, references match). The layer that converts reading into control.
- **Entity resolution mapping** — The link from an extracted vendor name to the canonical vendor. Without it, extracted spend cannot be aggregated correctly.
- **Exception flags** — Explicit markers for what needs human attention and why. A good flag names the specific problem, not just 'low confidence'.
- **Correction history** — What a reviewer changed and to what. The raw material for improving extraction and for auditing who touched a value.
- **Coverage / completeness check** — Whether every expected field for the document type was found. A missing required field is a failure even if what was found is correct.
- **Processing metadata** — Timestamps, model version, and pre-processing steps applied. Needed to diagnose why last month's extraction differs from this month's.

## Failure modes

- **Hallucinated extraction accepted as truth** — The system returns a plausible amount or date that is not actually on the page, and because it looks reasonable and carries no visible uncertainty, it is posted. The error surfaces at reconciliation weeks later. Confidence scoring and source-location verification exist precisely to prevent this, and skipping them is the most dangerous shortcut in the whole capability.
- **OCR treated as extraction** — A team deploys character recognition, gets clean text, and calls it done — but nothing understands which number is the retainage and which is the total, and nothing validates the math. The result is faster typing, not extraction, and the validation controls that provide the real value are absent.
- **Silent misclassification** — A conditional lien waiver is classified as unconditional, or a quote is classified as an invoice. Extraction proceeds confidently against the wrong field map and posts a well-formed wrong record. Classification failures are the most damaging because everything downstream trusts them.
- **No validation, only capture** — Fields are extracted accurately but nothing checks that the invoice foots, the COI is current, or the waiver matches the payment. Expired certificates and unbalanced invoices flow straight through, and the documents that were supposed to be controls become rubber stamps.
- **The review queue that punishes the reviewer** — Exceptions route to a person, but the interface shows a form with no source page, so the reviewer re-opens the original PDF and hunts for each value. Review becomes slower than manual entry, adoption collapses, and the capability is blamed for the interface's failure.
- **Extraction that never learns** — Reviewers correct the same layout every month and the corrections go nowhere. Accuracy on that vendor's format never improves, trust erodes, and the team drifts back to keying by hand while the extraction license quietly renews.
- **Provenance dropped at posting** — The structured value is written to accounts payable but the link to the source document is not, so when the amount is later disputed nobody can pull up what was actually on the invoice. The number exists; the evidence does not.

## Metrics

- **Straight-through processing rate** — Share of documents fully extracted, validated, and posted without human touch. The headline efficiency metric — but meaningless if accuracy is not held constant.
- **Field-level accuracy** — Correctness per field against a verified ground truth, weighted by how much a wrong value costs. A 99 percent name accuracy with 90 percent amount accuracy is a bad system.
- **Extraction confidence calibration** — Whether stated confidence matches actual accuracy. Miscalibrated confidence is worse than none, because it routes wrong values to auto-post.
- **Exception rate and resolution time** — Share of documents needing review and how long review takes. Reveals both extraction quality and whether the review interface works.
- **Validation catch rate** — Errors caught by business rules before posting — expired COIs, unbalanced invoices, mismatched waivers. Measures the control value, not just the reading value.
- **Correction / defect rate post-posting** — Errors found after data was posted. The true accuracy metric, since it counts what slipped past every gate.
- **Model version drift** — Change in accuracy across layout or model updates. Catches the case where an update quietly degraded a vendor's format.

## The AI shift

- **Conversational** — You can interrogate a document instead of reading it: ask what the retainage on this pay application is, whether this certificate of insurance meets the contract's required limits, or which line items changed from last month's invoice, and get an answer pointing at the exact place on the page. The document becomes queryable, and the query cites its own source.
- **Generative** — Beyond reading, extraction feeds generation: a validated pay application can produce its own exception summary, a set of invoices can generate a reconciliation narrative against the commitment, and a batch of COIs can produce a compliance status letter. The generated artifact is trustworthy because it is built on validated, provenance-linked fields rather than a summary of unstructured text.
- **Orchestrated** — Extraction stops being a standalone step and becomes a gate in a flow: an invoice is extracted, matched three-way against the purchase order and receipt, checked against the commitment, and either advanced for payment or routed to an exception — with the COI and lien waiver checked in the same motion. The document participates in the process instead of sitting in an inbox.
- **Autonomous** — The routine flow runs unattended within limits: high-confidence, fully validated documents post automatically, while anything below a confidence threshold, anything that fails a business rule, or anything above a dollar limit routes to a human. Autonomy here means the boring 80 percent processes itself with a complete audit trail, and the exceptions — where judgment and money live — always reach a person.

## Prompts

### Conversational — Checking a certificate of insurance against contract requirements.

```text
I am uploading a certificate of insurance and our subcontract's insurance requirements. Extract from the COI the named insured, each coverage type, its per-occurrence and aggregate limits, the policy effective and expiration dates, and whether our entity is listed as additional insured. Then compare every value against the contract requirements and tell me specifically where it complies and where it falls short — coverage type, limit gap, expired or soon-to-expire dates, missing additional insured or waiver of subrogation. Cite the exact location on the COI for each extracted value, and flag anything you could not read confidently rather than guessing a limit.
```

**Expected output:** A structured compliance check that pairs each extracted COI value with the contract requirement, names every gap specifically, and cites the source location — with explicit uncertainty flags on anything unreadable, not a blanket 'compliant'.

**Follow-ups:**

- Which of these deficiencies would block us from allowing the sub on site?
- Draft the deficiency notice to the subcontractor listing exactly what must be corrected.
- When does the earliest-expiring policy lapse, and what should we set a reminder for?

### Generative — Turning a batch of extracted invoices into a reconciliation narrative.

```text
You have the structured extraction of this month's subcontractor invoices for one project, plus the commitment amounts and prior billings. Produce a reconciliation summary: for each subcontractor, show billed-to-date against the committed amount and prior billings, flag any invoice whose line items do not foot, any that exceed the remaining commitment, any missing a matching lien waiver, and any where retainage was not withheld correctly. Write it as a narrative a project accountant can act on, with a clear exceptions list at the top and the routine items summarized below. Reference the source invoice for every exception you raise.
```

**Expected output:** An exceptions-first reconciliation narrative built on validated fields, with each flagged item traceable to its source invoice and a clear line between hold-payment and post-with-note.

**Follow-ups:**

- Which of these exceptions should hold payment versus which can post with a note?
- Draft the query email for the one invoice that exceeds its commitment.
- Total the correctly and incorrectly withheld retainage across the batch.

### Orchestrated — Running an invoice through the full payment gate.

```text
An extracted subcontractor invoice has arrived. Run it through our payment controls: perform a three-way match against the purchase order and the delivery receipt, check the billed amount against the remaining commitment, verify a current lien waiver and certificate of insurance are on file for this vendor, and confirm retainage is withheld at the contract rate. Return a single decision — advance for payment, hold, or route to exception — with each supporting check shown as pass or fail and cited to the underlying document or record. Where a check fails, name exactly what is missing or mismatched. Do not advance anything for payment that fails a control, and do not approve payment yourself.
```

**Expected output:** A single advance/hold/exception decision with every gate shown pass or fail and cited to its source, failures named specifically, and payment approval explicitly left to a human.

**Follow-ups:**

- For the failed checks, draft what each responsible party needs to provide.
- Does this invoice's line detail match the schedule of values on the commitment?
- Which control failures are recurring for this vendor?

### Autonomous — Standing policy for how invoice extraction should run unattended.

```text
Operate subcontractor invoice intake continuously under these rules. Classify each incoming document; if classification confidence is below threshold, route to a human rather than guessing. Extract all fields with confidence and source location. Auto-post only invoices that meet all of: every required field extracted above the confidence threshold, invoice math foots, three-way match passes, amount within the remaining commitment, current COI and lien waiver on file, retainage correct, and total below the auto-post dollar limit. Route everything else to the exception queue with the specific reason. Never auto-post an invoice that fails any validation, never auto-post above the dollar limit regardless of confidence, and never post any value without its provenance link. Escalate to me any invoice that exceeds its commitment or references an expired insurance certificate.
```

**Expected output:** An unattended intake loop that posts only fully validated, high-confidence, under-limit invoices, routes everything else to a reasoned exception queue, preserves provenance on every record, and never approves payment or crosses a control boundary without a human.

**Follow-ups:**

- Show me this week's straight-through rate and the top three exception reasons.
- Which vendors' layouts are generating the most corrections, so we can improve them?
- List everything currently held and why, ranked by dollar value.

## Maturity ladder

- **Level 0 — Level 0 — Manual keying** — Documents are read and retyped by people. Errors are individual, unmeasured, and discovered downstream; the labor cost is real but invisible.
- **Level 1 — Level 1 — OCR / templates** — Text is recognized and fixed-layout templates capture some fields, but understanding and validation are absent and any format change breaks the template.
- **Level 2 — Level 2 — Extraction with confidence** — Fields are understood across variable layouts, each value carries a confidence score and a source location, and low-confidence values route to human review.
- **Level 3 — Level 3 — Validated and integrated** — Business rules validate every document, entities resolve to canonical records, provenance links persist, and validated data posts to the system of record automatically.
- **Level 4 — Level 4 — Autonomous with learning** — High-confidence validated documents process straight through inside dollar and control guardrails, exceptions route with specific reasons, and every correction improves accuracy on that layout.

## FAQ

### How is document extraction different from OCR?

OCR renders the characters on a page into text; extraction understands what those characters mean and validates them. OCR can tell you a document contains the number 47,500; extraction knows that number is the retainage, that it should equal ten percent of the work completed, and whether it does. The distinction matters because the value of the capability lives in the understanding and validation, not the character reading.

### What confidence threshold should auto-post a value?

There is no universal number, because the right threshold depends on what a wrong value costs. A vendor name on a well-known invoice can auto-post at a lower bar than an amount on a large payment. The discipline is to set thresholds per field and per dollar exposure, hold accuracy constant as you raise the straight-through rate, and verify that stated confidence is actually calibrated to real accuracy before trusting it to route anything.

### Do we still need people if extraction is accurate?

Yes, but doing different work. People stop keying routine documents and start resolving exceptions — the low-confidence fields, failed validations, and genuinely ambiguous documents where judgment matters. A well-designed capability makes that exception work fast by showing the source page at the value in question, and it feeds every correction back to improve future accuracy.

### What is the single most dangerous failure to guard against?

A confident wrong extraction accepted as truth. Because the value looks plausible and carries no visible uncertainty, it posts and propagates into job cost, forecasts, and decisions before anyone checks it. Confidence scoring, source-location verification, and business-rule validation exist to catch exactly this, which is why a system that returns bare values with no provenance or confidence is more dangerous than manual keying, not less.

## Related objects

- [Construction Data Foundation](https://briq.ai/acu/object/construction-data-foundation)
- [Accounts Payable Invoice](https://briq.ai/acu/object/ap-invoice)
- [Certificate of Insurance (COI)](https://briq.ai/acu/object/certificate-of-insurance)
- [Lien Waiver](https://briq.ai/acu/object/lien-waiver)
- [Pay Application (AIA G702/G703)](https://briq.ai/acu/object/pay-application)
- [Human Review & Approval Controls](https://briq.ai/acu/object/human-review-controls)