Skip to main content

What are you looking for?

Explore our services and discover how we can help you achieve your goals

Client Workspace

Extracting invoice and contract fields with vision-language models: validation rules and review routing

A vision-language model can read invoice and contract fields straight from the page image, but it should never post them on its own. Netbase pairs each extracted value with validation rules and several confidence signals, posts only fields that pass every check, and routes the rest to a person who sees the value beside its source.

Book a solution review See the intelligent document processing solution

By Netbase AI and Automation Engineering Desk · Reviewed by David (CEO) · Updated · 10 min read

star

This guide is for finance, procurement and legal-operations leads and the AI lead who must make document intake safe. It sits under our generative AI development work and pairs with the intelligent document processing solution.

In this guide

Why is a vision-language model not enough on its own?

A vision-language model reads layout, tables and handwriting in one pass, removing much of the template work of older optical character recognition (OCR) pipelines. The risk: it writes a plausible value even when the image is unclear. Anthropic's vision documentation states that a model might hallucinate or make mistakes on low-quality, rotated or very small images, and that its coordinates are approximate.

Constrained output does not close the gap. OpenAI's structured outputs guide says the schema guarantees required fields and allowed values, while the application must still validate business rules the schema does not express. A well-formed invoice record can still carry the wrong total or a swapped date.

Self-reported confidence is the third weak point. Research found modern neural networks poorly calibrated, and a score a model writes about its own answer is weaker evidence still. Treat it as one signal, never as permission to post.

Which rule catches which extraction error?

Every field gets rules from four classes plus grounding. A finance or legal owner adds the company's own rules before the first pilot.

  • Invoice number, PO number

    Format rule
    Pattern per supplier
    Arithmetic or cross-field rule
    Not already posted (duplicate check)
    Master-data rule
    PO exists and is open
    Grounding rule
    Value appears in the text layer
  • Supplier name and tax ID

    Format rule
    Tax ID pattern and check digit
    Arithmetic or cross-field rule
    Matches the supplier on the PO
    Master-data rule
    Supplier exists in the vendor master
    Grounding rule
    Found near the header block
  • Bank details

    Format rule
    IBAN or account pattern with check digits
    Arithmetic or cross-field rule
    Unchanged since the last paid invoice
    Master-data rule
    Matches the verified vendor record
    Grounding rule
    Found in the payment block
  • Amounts and tax

    Format rule
    Currency and decimals valid
    Arithmetic or cross-field rule
    Lines add up to subtotal; subtotal plus tax equals total
    Master-data rule
    Tax rate allowed for that supplier and country
    Grounding rule
    Each amount appears in the text layer
  • Dates

    Format rule
    Real calendar date
    Arithmetic or cross-field rule
    Due date after invoice date; contract end after start
    Master-data rule
    Within the PO or contract period
    Grounding rule
    Found next to its label
  • Contract parties and clauses

    Format rule
    Party names not empty
    Arithmetic or cross-field rule
    Signatory party matches a party in the header
    Master-data rule
    Party exists in the customer or vendor master
    Grounding rule
    Clause text quoted with its page reference

A changed bank account on an otherwise valid invoice should always go to a person, whatever the model's confidence.

In contracts, parties and dates behave like invoice fields; obligations, renewal terms and liability caps are interpretations. Ask for the exact clause text and page, and let a reviewer confirm the reading.

Which confidence signals should drive routing?

No single signal is trustworthy on its own.

Criterion Self-reported score Two-pass agreement Grounding against the text layer
How it works The model rates each field Two prompts or two models extract the same field and must agree The value must appear in OCR or PDF text at the place the model points to
What it catches Fields the model knows are unclear Unstable reads and invented values Values not present on the page at all
What it misses Confident errors Errors both passes share Scanned pages with no reliable text layer
Cost Lowest One extra call per document An OCR step and a matching rule
Use it when As a tie-breaker, never alone Amounts, identifiers and bank details Digital PDFs and good scans

Rule results are the fourth signal and the strongest one: a field that fails a validation rule goes to review even when every score is high. Microsoft's Document Intelligence guidance makes the same point for its own scores, combining field and word confidence and using them to decide between automatic acceptance and human review.

How should fields be routed: post, review or reject?

Routing works per field first, then per document. Four outcomes cover most cases:

  • Post

    Condition
    All rules pass, signals agree, field is not on the always-review list
    What the reviewer sees
    Nothing; the value is logged with its source region
  • Review the field

    Condition
    One signal is weak or one soft rule fails
    What the reviewer sees
    The value beside a crop of the page, with the failed check named
  • Review the document

    Condition
    Several fields fail, document type is uncertain, or totals do not reconcile
    What the reviewer sees
    The whole document with all extracted fields
  • Reject

    Condition
    Not an invoice or contract, unreadable, or a duplicate
    What the reviewer sees
    The reason, sent back to the sender's queue

Set thresholds per field, not per document. A reference note can post on weaker evidence than an amount or a bank account. Keep an always-review list for fields where an error is expensive, and audit a random sample of posted documents as well. AWS describes the same pairing in its Augmented AI documentation: human review of low-confidence predictions plus random samples.

Research on per-field selective risk control frames routing as accepting as many fields as possible while keeping their error rate under a set limit, and shows naive thresholds can break that guarantee when documents cluster. Set thresholds on held-out documents of your own and re-check them by supplier and layout.

How to build the pipeline in the right order

  1. Collect a labelled test set

    Real documents per main supplier or template, with correct values, kept apart from prompt examples.

  2. Write the field schema and the rule matrix

    One owner per field; the finance or legal owner signs the rules.

  3. Classify before extracting

    Decide the document type first, then use a schema and prompt for that type.

  4. Extract with grounding

    Ask for each value with its page and region, and match it against the text layer where one exists.

  5. Run the rules and combine the signals

    Every field leaves this step with an outcome from the routing table.

  6. Build the review screen

    Value beside the source crop, with the failed check named.

  7. Set thresholds on the test set

    Agree the error rate among posted fields and the review share with the process owner.

  8. Feed corrections back

    as new test cases and, later, training data.

  9. Monitor after launch

    Track review and correction rates by supplier and layout.

Reviewer corrections are labelled data. Our guide to preparing fine-tuning data covers how to keep them clean and out of the test set, and monitoring belongs with responsible AI and MLOps.

When should you not use a vision-language model?

  • The invoice is already structured. A structured e-invoice in the European standard EN 16931 carries its fields as data; parse it, do not read its picture.
  • One supplier, one fixed template. A template rule or a classic OCR zone is cheaper and easier to test.
  • High volume on a narrow task. A trained extraction model or a small language model may cost less per page.
  • No one owns the review queue. Without a person who clears routed fields, automation only moves the backlog.

Documents are also an attack channel. OWASP lists indirect prompt injection through external content as a top risk; treat document text as data, never as instructions, and give the extraction step no write access.

Where AI fits

AI classifies documents, reads fields from the image and points to where each value sits. Engineers still write the schemas and rules, set thresholds, own the test set and decide which fields may ever post without a person. Netbase works with the major commercial and open-weight models, chosen per project. Each item below states how mature it is at Netbase.

What delivery record exists, and what does not?

Netbase has delivered document AI for clients that are not named, alongside retrieval-augmented knowledge assistants and MLOps pipelines. The anonymous document AI platform record publishes no client name, document types, models, accuracy or volume, and neither does this page. The delivered AI feature we name with a client is the recommendation engine for 4over4, which suggests products from browsing history and previous purchases; it is not a document system. Most Netbase projects run on fixed-price contracts, so the scope of rules, signals and review screens is agreed before the build.

What are the alternatives to building your own pipeline?

Option Choose it when Watch for
Prebuilt invoice model from a cloud document service Invoices in common layouts, few custom fields Contract clauses and custom rules still need your own layer
Packaged document processing platform Many document types, a team that prefers configuration to code Fit with your rules, review screen and data location
Custom pipeline on a vision-language model Mixed layouts, contracts, rules tied to your ERP or CRM You own the test set, thresholds and monitoring
E-invoice parsing Suppliers send structured invoices Paper and PDF suppliers still need extraction

Whatever you choose, the rule matrix and routing table still apply. Our computer vision and document AI service covers documents that arrive as photos.

What are the limits of this guide?

  • It describes a method, not a measured result; error and review rates depend on your documents and must be measured on your own test set.
  • Legal interpretation of contract clauses stays with qualified people; extraction only finds and quotes the text.

Frequently asked questions

For mixed layouts it often can read fields directly, but keeping an OCR or PDF text layer is still useful, because matching each value against that text is one of the strongest checks that the value is really on the page.

There is no universal number. Set thresholds per field on a labelled test set of your own documents, stricter for amounts and bank details than for reference notes, and re-check them by supplier and layout.

No. Route the single field to review when the rest of the document passes, and send the whole document only when totals do not reconcile, several fields fail or the document type is uncertain.

Next step

Send us sample invoices or contracts and the rules your team applies today, and we will book a solution review to map rules, signals and routing for your intake.

Generative AI development with evaluation and human review Generative AI development with evaluation and human review

Netbase builds generative AI features for product and operations teams: drafting, summarising, rewriting and content generation inside the software your people and customers already use. Every feature ships with an evaluation set, a human review step where output matters and limits on cost. Generative AI is a capability we are growing, and we label its maturity honestly.

Learn More
line
Computer vision AI for images and artwork Computer vision AI for images and artwork

Netbase applies computer vision to images and artwork: checking customer uploads before print, tagging and sorting product images, and converting raster images into vector files. Document workflows such as invoices and forms belong to our intelligent document processing solution. Computer vision is a capability we are growing.

Learn More
line
Intelligent document processing with AI: from orders, invoices and forms to validated data Intelligent document processing with AI: from orders, invoices and forms to validated data

Intelligent document processing is AI-driven intake that reads orders, invoices and forms, extracts the fields your systems need, validates them against business rules and routes each record to the right queue, with people reviewing anything uncertain. Netbase offers it as a growth capability built on its computer vision and automation work, including a document AI platform for an unnamed client.

Learn More
line

Work with Netbase remotely

Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.

Contact Netbase

Discuss a project

Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.
Project enquiries

project@netbasejsc.com

WhatsApp

+84 937 869 689

Office address

91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam

Get in touch

Tell us what you want to build, modernize, or operate.

Tell us what you want to build, modernize, or operate.

Contact Netbase