Procurement Automation

Human Review in Procurement Automation: What Stops a Misread Value?

Ten controls that stop a wrong extracted value reaching a purchase order: source text checks, code-only maths, blocking flags and an approval log.

By
PhaseVane
Published
Reading time
7 min
On This Page

A model reads a supplier quote and returns a neat table. Every cell is filled, every number is formatted, and nothing about the table tells you which value is wrong. That is the problem a finance controller is right to worry about. The generative AI profile from NIST (the US National Institute of Standards and Technology) calls it confabulation: "confidently stated but erroneous or false content" (S1). The question is not whether a model will ever misread a document. It is what stops a misread value reaching a purchase order.

This guide lists the controls that answer that question, what each one catches, and what it costs the reviewer. Each is shown on four example supplier quotes, so you can judge whether a system you are evaluating has them.

In short

  • Separate three jobs. The model extracts. Code calculates and checks. A person decides. A system that lets the model do the second or third job has no controls worth the name.
  • Every value carries the text it came from, and code checks that text exists in the document. A value without verifiable source text is treated as unread.
  • Problems block. An expired price, a minimum order quantity (MOQ) above your need or a total that does not add up holds the quote out of the ranking until a person on our team accepts or corrects it, with a reason.
  • Everything is logged. Who changed what, from what, to what, and why.
  • Accuracy is measured, not asserted. Ask for field-level results on your own documents before you rely on any claim.

The principle: extract, calculate, decide

Job Done by Why
Extract: copy values from a document into fields Model Reading varied layouts is what models are good at, and what retyping costs
Calculate and check: totals, conversions, dates, rules Ordinary code Same inputs, same answer, every time; testable in advance
Decide: accept a flag, send a question, choose a supplier A person Accountability, and knowledge the documents do not contain

1. A fixed schema that allows "not stated"

The model fills a defined set of fields and may return null when a value is absent. Without that option, a model under instruction to fill every field has a reason to guess. Note what a schema does and does not guarantee: Anthropic's documentation describes structured outputs as constraining responses to a schema so they are valid and parseable (S2). Valid is not correct. A well-formed field can hold a wrong number. The same documentation warns that a refusal or a token limit can produce output that does not match the schema (S2), so each response is checked and a failed file is shown as failed, never skipped.

Catches: Supplier C states no MOQ; the field stays empty and raises an information flag instead of becoming "1".

2. Source text for every value, verified by code

Each extracted value comes with the exact text it was read from and its page, sheet and cell, or line. Code then checks that the text actually appears in the document. This matters because, according to Anthropic's documentation, the API's native citations feature cannot be used together with structured outputs (S3), so a system that wants both a fixed schema and evidence has to ask for the snippet as a field and verify it independently. Scanned PDFs with no text layer are not citable (S3); their values cannot be verified this way and go to manual entry.

Catches: a value whose snippet is not found in the document is marked unread, whatever the model's confidence.

3. No arithmetic by the model

Totals, currency conversions, landed-cost lines and date calculations are done in code. The model is told not to calculate, and nothing it calculates is used.

Catches: Supplier D shows a line total of 27,850.00. Code computes 700 × 39.50 = 27,650.00 and raises a blocking mismatch of 200.00.

4. Deterministic rules, written down in advance

Each check is a rule anyone can read: validity date before the comparison date; MOQ above required quantity; terms that are not one of the eleven Incoterms rules; required field missing. Rules are tested against known documents before use.

Catches: Supplier B's quote was valid 14 days from 25 September, so it expired on 9 October, three days before the comparison. Its MOQ of 1,000 exceeds the requirement of 700.

5. Blocking flags that need a reason

Severity matters. Information flags inform; warnings ask for attention; blocking flags hold the quote out of the clean ranking until a person accepts or corrects them. Accepting needs a one-line reason. Nothing is deleted.

Catches: Supplier B is lowest on paper (AUD 23,923.10 landed against A's 24,706.48) but cannot appear as the recommended clean quote while two blocks stand.

6. Ambiguity kept as text

When a document is ambiguous, the wording is kept verbatim and the structured value left empty, with a warning. The system never resolves ambiguity silently.

Catches: Supplier C's "price delivered your port Sydney" is kept as written, flagged as possibly CFR or CIF, and treated cautiously (an insurance allowance is added) until the supplier answers. "Valid this month" on an email of 5 October is read as 31 October and flagged for confirmation.

7. Buyer inputs held apart and labelled

Exchange rates, freight estimates, insurance, destination charges and duty rates are the buyer's inputs, entered and dated by a person, shown on the comparison and never inferred by the model.

Catches: a comparison that silently used last month's exchange rate.

8. Edits re-run everything

When a reviewer corrects a value, validation and calculation run again from scratch. The corrected cell is marked "reviewed by user".

Catches: after Supplier D confirms the correct total, the reviewer edits 27,850.00 to 27,650.00. The mismatch clears and D enters the clean ranking at 27,870.00 including its 220.00 delivery.

9. An approval log

Every acceptance and every edit is logged.

Time Reviewer Quote Field or flag From To Reason
12 Oct 10:42 Buyer 1 D line_total_stated 27,850.00 27,650.00 Supplier confirmed by email 12 Oct, typo on quote
12 Oct 10:47 Buyer 1 C VALIDITY_AMBIGUOUS flag accepted Read as 31 Oct; question sent to supplier
12 Oct 10:51 Buyer 1 B VALIDITY_EXPIRED flag open Awaiting revised quote; B excluded from award

10. Accuracy measured on your documents

Field-level results are reported: how many fields were extracted, how many a reviewer corrected, and every miss, with whether a rule caught it. PhaseVane's own release gate for Quote Desk, before it is offered for sale, is at least 95% of required fields correct after review rules are applied, with every miss flagged rather than silent, on at least 30 real quotes from each of two design partners. That is a target to be met, not a result: no accuracy figure has been measured yet.

What these controls do not fix

  • A wrong requirement. If the request for quotation (RFQ) said 700 and you need 900, every check passes.
  • Wrong buyer inputs. The system applies your freight estimate consistently; it does not know it is out of date.
  • A supplier who misstates a fact. Extraction copies what the document says.
  • Item matching across suppliers. Different descriptions for the same item still need a person to confirm they are the same thing.

Eight questions to ask any vendor

  1. Does the model ever calculate a number that appears in the output?
  2. Can a field be returned as "not stated"?
  3. Does every value show the source text, and is that text checked against the document by code?
  4. What happens to a scanned page with no text layer?
  5. Which flags block export or ranking, and who can clear them?
  6. Is a reason required to accept a flag, and is it logged?
  7. Can you show field-level accuracy on 20 of our documents, including every correction?
  8. Where are our documents stored, for how long, and who can see them?

For the task split these controls sit inside, see which steps of quote comparison can be automated. To weigh review time against error risk, use the time and error model. The same controls apply to PO acknowledgement checks.

See the controls working

Quote Desk is PhaseVane's quote comparison system, designed around these ten controls. It is in development and not on sale.

See the Quote Desk walkthrough. Watch each control in this guide fire on the four example quotes: the source text, the blocking flags, the edit that clears one, and the approval log. See the walkthrough

Questions about how this would apply to your documents? Book a 20-minute workflow call.

Sources

Checked 9 October 2026.

Watch each control in this guide fire on four example quotes: the source text, the blocking flags, the edit that clears one, and the approval log.

See the Quote Desk Walkthrough

Related Research

See all research notes

Topic: Procurement Automation

  1. Cost Driver Monitoring: Is This Supplier Price Increase Justified?

    Map each supplier cost element to a free public index, track it monthly, and test a price increase claim against the weighted move. Worked example.

    Procurement Automation
    Read More
  2. Freight Quote Comparison Automation: Can It Keep the Detail?

    How automated intake turns forwarder quotes from email, PDF and Excel into one comparison without dropping surcharges, TBA lines or validity.

    Procurement Automation
    Read More
  3. Procurement Automation: Which Workflow Should You Automate First?

    Score purchasing workflows on volume, error cost, document regularity, data access and decision risk to pick the first worth automating.

    Procurement Automation
    Read More