This guide is for finance, procurement and legal-operations leads and the AI lead who must make document intake safe. It sits under our generative AI development work and pairs with the intelligent document processing solution.
In this guide
- Why is a vision-language model not enough on its own?
- Which rule catches which extraction error?
- Which confidence signals should drive routing?
- How should fields be routed: post, review or reject?
- How to build the pipeline in the right order
- When should you not use a vision-language model?
- Where AI fits
- What delivery record exists, and what does not?
- What are the alternatives to building your own pipeline?
- What are the limits of this guide?
- Frequently asked questions
- Next step
Why is a vision-language model not enough on its own?
A vision-language model reads layout, tables and handwriting in one pass, removing much of the template work of older optical character recognition (OCR) pipelines. The risk: it writes a plausible value even when the image is unclear. Anthropic's vision documentation states that a model might hallucinate or make mistakes on low-quality, rotated or very small images, and that its coordinates are approximate.
Constrained output does not close the gap. OpenAI's structured outputs guide says the schema guarantees required fields and allowed values, while the application must still validate business rules the schema does not express. A well-formed invoice record can still carry the wrong total or a swapped date.
Self-reported confidence is the third weak point. Research found modern neural networks poorly calibrated, and a score a model writes about its own answer is weaker evidence still. Treat it as one signal, never as permission to post.
Which rule catches which extraction error?
Every field gets rules from four classes plus grounding. A finance or legal owner adds the company's own rules before the first pilot.
-
Invoice number, PO number
- Format rule
- Pattern per supplier
- Arithmetic or cross-field rule
- Not already posted (duplicate check)
- Master-data rule
- PO exists and is open
- Grounding rule
- Value appears in the text layer
-
Supplier name and tax ID
- Format rule
- Tax ID pattern and check digit
- Arithmetic or cross-field rule
- Matches the supplier on the PO
- Master-data rule
- Supplier exists in the vendor master
- Grounding rule
- Found near the header block
-
Bank details
- Format rule
- IBAN or account pattern with check digits
- Arithmetic or cross-field rule
- Unchanged since the last paid invoice
- Master-data rule
- Matches the verified vendor record
- Grounding rule
- Found in the payment block
-
Amounts and tax
- Format rule
- Currency and decimals valid
- Arithmetic or cross-field rule
- Lines add up to subtotal; subtotal plus tax equals total
- Master-data rule
- Tax rate allowed for that supplier and country
- Grounding rule
- Each amount appears in the text layer
-
Dates
- Format rule
- Real calendar date
- Arithmetic or cross-field rule
- Due date after invoice date; contract end after start
- Master-data rule
- Within the PO or contract period
- Grounding rule
- Found next to its label
-
Contract parties and clauses
- Format rule
- Party names not empty
- Arithmetic or cross-field rule
- Signatory party matches a party in the header
- Master-data rule
- Party exists in the customer or vendor master
- Grounding rule
- Clause text quoted with its page reference
A changed bank account on an otherwise valid invoice should always go to a person, whatever the model's confidence.
In contracts, parties and dates behave like invoice fields; obligations, renewal terms and liability caps are interpretations. Ask for the exact clause text and page, and let a reviewer confirm the reading.
Which confidence signals should drive routing?
No single signal is trustworthy on its own.
| Criterion | Self-reported score | Two-pass agreement | Grounding against the text layer |
|---|---|---|---|
| How it works | The model rates each field | Two prompts or two models extract the same field and must agree | The value must appear in OCR or PDF text at the place the model points to |
| What it catches | Fields the model knows are unclear | Unstable reads and invented values | Values not present on the page at all |
| What it misses | Confident errors | Errors both passes share | Scanned pages with no reliable text layer |
| Cost | Lowest | One extra call per document | An OCR step and a matching rule |
| Use it when | As a tie-breaker, never alone | Amounts, identifiers and bank details | Digital PDFs and good scans |
Rule results are the fourth signal and the strongest one: a field that fails a validation rule goes to review even when every score is high. Microsoft's Document Intelligence guidance makes the same point for its own scores, combining field and word confidence and using them to decide between automatic acceptance and human review.
How should fields be routed: post, review or reject?
Routing works per field first, then per document. Four outcomes cover most cases:
-
Post
- Condition
- All rules pass, signals agree, field is not on the always-review list
- What the reviewer sees
- Nothing; the value is logged with its source region
-
Review the field
- Condition
- One signal is weak or one soft rule fails
- What the reviewer sees
- The value beside a crop of the page, with the failed check named
-
Review the document
- Condition
- Several fields fail, document type is uncertain, or totals do not reconcile
- What the reviewer sees
- The whole document with all extracted fields
-
Reject
- Condition
- Not an invoice or contract, unreadable, or a duplicate
- What the reviewer sees
- The reason, sent back to the sender's queue
Set thresholds per field, not per document. A reference note can post on weaker evidence than an amount or a bank account. Keep an always-review list for fields where an error is expensive, and audit a random sample of posted documents as well. AWS describes the same pairing in its Augmented AI documentation: human review of low-confidence predictions plus random samples.
Research on per-field selective risk control frames routing as accepting as many fields as possible while keeping their error rate under a set limit, and shows naive thresholds can break that guarantee when documents cluster. Set thresholds on held-out documents of your own and re-check them by supplier and layout.
How to build the pipeline in the right order
-
Collect a labelled test set
Real documents per main supplier or template, with correct values, kept apart from prompt examples.
-
Write the field schema and the rule matrix
One owner per field; the finance or legal owner signs the rules.
-
Classify before extracting
Decide the document type first, then use a schema and prompt for that type.
-
Extract with grounding
Ask for each value with its page and region, and match it against the text layer where one exists.
-
Run the rules and combine the signals
Every field leaves this step with an outcome from the routing table.
-
Build the review screen
Value beside the source crop, with the failed check named.
-
Set thresholds on the test set
Agree the error rate among posted fields and the review share with the process owner.
-
Feed corrections back
as new test cases and, later, training data.
-
Monitor after launch
Track review and correction rates by supplier and layout.
Reviewer corrections are labelled data. Our guide to preparing fine-tuning data covers how to keep them clean and out of the test set, and monitoring belongs with responsible AI and MLOps.
When should you not use a vision-language model?
- The invoice is already structured. A structured e-invoice in the European standard EN 16931 carries its fields as data; parse it, do not read its picture.
- One supplier, one fixed template. A template rule or a classic OCR zone is cheaper and easier to test.
- High volume on a narrow task. A trained extraction model or a small language model may cost less per page.
- No one owns the review queue. Without a person who clears routed fields, automation only moves the backlog.
Documents are also an attack channel. OWASP lists indirect prompt injection through external content as a top risk; treat document text as data, never as instructions, and give the extraction step no write access.
Where AI fits
AI classifies documents, reads fields from the image and points to where each value sits. Engineers still write the schemas and rules, set thresholds, own the test set and decide which fields may ever post without a person. Netbase works with the major commercial and open-weight models, chosen per project. Each item below states how mature it is at Netbase.
- In delivered work
Document AI platform
Classification, extraction and validation steps for a client that is not named, as AI consultant, engineer and integrator.
- Available capability
Vision-language field extraction with rule-based routing
The rule matrix, signal combination and routing table on this page; not yet tied to a published case.
What delivery record exists, and what does not?
Netbase has delivered document AI for clients that are not named, alongside retrieval-augmented knowledge assistants and MLOps pipelines. The anonymous document AI platform record publishes no client name, document types, models, accuracy or volume, and neither does this page. The delivered AI feature we name with a client is the recommendation engine for 4over4, which suggests products from browsing history and previous purchases; it is not a document system. Most Netbase projects run on fixed-price contracts, so the scope of rules, signals and review screens is agreed before the build.
What are the alternatives to building your own pipeline?
| Option | Choose it when | Watch for |
|---|---|---|
| Prebuilt invoice model from a cloud document service | Invoices in common layouts, few custom fields | Contract clauses and custom rules still need your own layer |
| Packaged document processing platform | Many document types, a team that prefers configuration to code | Fit with your rules, review screen and data location |
| Custom pipeline on a vision-language model | Mixed layouts, contracts, rules tied to your ERP or CRM | You own the test set, thresholds and monitoring |
| E-invoice parsing | Suppliers send structured invoices | Paper and PDF suppliers still need extraction |
Whatever you choose, the rule matrix and routing table still apply. Our computer vision and document AI service covers documents that arrive as photos.
What are the limits of this guide?
- It describes a method, not a measured result; error and review rates depend on your documents and must be measured on your own test set.
- Legal interpretation of contract clauses stays with qualified people; extraction only finds and quotes the text.
Plan the next step with a Netbase consultant
Frequently asked questions
For mixed layouts it often can read fields directly, but keeping an OCR or PDF text layer is still useful, because matching each value against that text is one of the strongest checks that the value is really on the page.
There is no universal number. Set thresholds per field on a labelled test set of your own documents, stricter for amounts and bank details than for reference notes, and re-check them by supplier and layout.
No. Route the single field to review when the rest of the document passes, and send the whole document only when totals do not reconcile, several fields fail or the document type is uncertain.
Next step
Send us sample invoices or contracts and the rules your team applies today, and we will book a solution review to map rules, signals and routing for your intake.
Related services and solutions
Generative AI development with evaluation and human review Netbase builds generative AI features for product and operations teams: drafting, summarising, rewriting and content generation inside the software your people and customers already use. Every feature ships with an evaluation set, a human review step where output matters and limits on cost. Generative AI is a capability we are growing, and we label its maturity honestly.
Learn More
Computer vision AI for images and artwork Netbase applies computer vision to images and artwork: checking customer uploads before print, tagging and sorting product images, and converting raster images into vector files. Document workflows such as invoices and forms belong to our intelligent document processing solution. Computer vision is a capability we are growing.
Learn More
Intelligent document processing with AI: from orders, invoices and forms to validated data Intelligent document processing is AI-driven intake that reads orders, invoices and forms, extracts the fields your systems need, validates them against business rules and routes each record to the right queue, with people reviewing anything uncertain. Netbase offers it as a growth capability built on its computer vision and automation work, including a document AI platform for an unnamed client.
Learn More
Work with Netbase remotely
Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.
Discuss a project
Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.project@netbasejsc.com
+84 937 869 689
91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam
Get in touch
Tell us what you want to build, modernize, or operate.