This guide is for heads of data, analytics leads and CTOs at mid-market companies who are being asked to buy a catalog or governance platform and want to know what is enough. It sits under Netbase's data engineering and platforms service, and it pairs with the guides on change data capture for operational databases and on streaming or batch latency per business metric, whose outputs (capture method and freshness tier) belong in the catalog entry of each table.
In this guide
- What does a 20-person data team actually need from a catalog?
- What is the minimum viable catalog?
- Which lineage use cases pay back?
- Catalog in code, open-source catalog or commercial platform?
- When should a data team not buy a catalog platform?
- How to roll out a minimum viable catalog
- Where AI fits
- What delivery record exists, and what does not
- Alternatives: other ways to make data findable
- Limits of this guide
- Frequently asked questions
- Next step
What does a 20-person data team actually need from a catalog?
A team of this size needs three things from a catalog: people can find the right table without asking in chat, every important table has a named owner, and an engineer can see what depends on a table before changing it. Search across every asset, automated classification and approval workflows are useful later, but they are not what slows a team of twenty down.
At this size the usual failure is not a missing tool. It is two tables with similar names, no one sure which is current, and a dashboard that breaks after a column rename because nobody knew it read that column. A catalog that covers the tables people actually query, with an owner and a status on each, removes most of that friction.
Public guidance for lean teams makes the same point: with one warehouse, clear naming, a short data dictionary and the schema browser of the BI tool often cover the need, and a dedicated catalog becomes worth it once nobody can reliably say which table is the right one.
What is the minimum viable catalog?
The minimum viable catalog is one entry per table people query, kept in the same repository as the pipeline code, so it is reviewed with every change. It starts with the most-queried tables and the tables behind board or finance figures, not with every table in the warehouse.
-
Name and status
- What it holds
- Table name, and whether it is current, deprecated or experimental
- Who fills it
- Data engineer who builds it
-
Description
- What it holds
- One or two plain sentences: what one row means and what the table is for
- Who fills it
- Table owner
-
Owner
- What it holds
- A named role or person who answers questions and approves changes
- Who fills it
- Head of data assigns
-
Source and method
- What it holds
- Source system and how data arrives (batch export, API, change data capture)
- Who fills it
- Data engineer
-
Freshness tier
- What it holds
- How often it is refreshed and the freshness consumers rely on
- Who fills it
- Table owner with consumers
-
Sensitivity
- What it holds
- Whether it holds personal or restricted data, and the access rule
- Who fills it
- Table owner, checked by security
-
Key consumers
- What it holds
- The dashboards, reports or services that read it
- Who fills it
- Generated from lineage where possible
-
Known limits
- What it holds
- Gaps, late data, definitions that differ from another team's
- Who fills it
- Table owner
The rule of thumb: an entry without an owner is not finished, and a table without an entry should not feed a dashboard. Transformation tools such as dbt already let teams write descriptions next to the model code and generate a browsable documentation site with a dependency graph, which is often the first catalog a team needs.
Which lineage use cases pay back?
Lineage is worth its upkeep only where it answers a question someone asks often or where a wrong answer is expensive. The matrix below ranks the common use cases by the level of lineage each needs.
| Use case | Lineage level needed | Payback | Choose it when |
|---|---|---|---|
| Impact analysis before a change | Table level, from source to dashboard | High: prevents broken reports after renames and drops | Always; it is the first use case to support |
| Incident root cause | Table level with job run history | High: shortens the search for which job produced a bad figure | Several pipelines feed shared tables |
| Regulated or contested figures | Column level, for a small set of fields | High for those fields only | A figure goes to auditors, regulators or the board |
| Migration or tool change | Table level, a full inventory | Medium, once per migration | A warehouse or BI move is planned |
| Access and deletion requests | Column level for personal data fields | Medium to high where personal data is held | Personal data spreads across many tables |
Column-level lineage across the whole warehouse rarely pays back for a team of twenty: it is costly to keep accurate, and most questions are answered at table level. Keep column-level lineage for the few fields where a wrong answer has legal or financial weight.
Catalog in code, open-source catalog or commercial platform?
| Option | Strength | Risk | Choose it when |
|---|---|---|---|
| Catalog in code and the transformation tool | Lowest running cost; entries are reviewed with code changes | Covers only what the tool builds; weak for sources and BI objects | One warehouse, one transformation project, one data team |
| Open-source catalog (for example DataHub) | Search and lineage across sources, warehouse and BI; no licence fee | Someone must run, upgrade and connect it | A second team produces data, or consumers cannot be listed |
| Commercial governance platform | Workflows, policy, classification and support included | Licence cost and a long rollout; adoption often lags | Compliance asks for it, or many teams share regulated data |
Open frameworks such as OpenLineage let jobs emit lineage events in a standard format, so the lineage you collect while on the first option can be loaded into a catalog later instead of being redrawn.
When should a data team not buy a catalog platform?
Do not buy a catalog platform when one team owns one warehouse and one transformation project, when nobody has time to keep owners and descriptions current, or when the real problem is unclear table names and duplicate models. A platform indexes the confusion; it does not remove it, and an empty or stale catalog teaches people to stop looking.
The triggers that justify a platform are organisational, not a table count: a second team producing data, consumers no one can list, a regulated report, an upcoming migration, or repeated incidents caused by unknown dependencies. When two or more apply, evaluate an open-source or commercial catalog against the entries you already keep in code.
How to roll out a minimum viable catalog
-
List the tables people query
Use warehouse query logs or BI usage to find the most-read tables and the tables behind finance and board figures; that list is the first scope.
-
Assign an owner per table
The head of data names an owner for each table on the list; a table nobody will own is a candidate for deprecation.
-
Write the entries next to the code
Add the eight fields to the model files or a catalog file in the same repository, and review them in the same pull request as the pipeline change.
-
Turn on the lineage you already have
Use the dependency graph of the transformation tool, and add lineage events from orchestration jobs where the tool does not see them.
-
Add one check in continuous integration
Fail a change that adds a table without an owner and description, or that drops a column a dashboard still reads.
-
Mark deprecated tables
Give every replaced table a status and a successor, and remove it after its consumers have moved.
-
Review the triggers twice a year
Check the upgrade triggers above; move to a catalog platform only when they apply, and load the existing entries into it.
Where AI fits
AI helps with the writing and checking around the catalog, not with deciding ownership. An assistant can draft table descriptions from model code and column names, suggest which columns may hold personal data, and flag entries whose description no longer matches the model. Netbase works with the major commercial and open-source AI tools and models, chosen per project.
Table owners still approve every description and sensitivity label, and engineers still decide which lineage is recorded; no model changes an owner, an access rule or a pipeline on its own.
- In delivered work
MLOps pipeline
An MLOps pipeline among the AI projects Netbase has delivered for clients that are not named; an AI delivery record, not a catalog deployment.
- Available capability
AI and data services
AI-drafted catalog descriptions and sensitivity suggestions are a way Netbase approaches the work, not yet tied to a published case.
What delivery record exists, and what does not
Netbase has delivered AI projects for clients that are not named, including MLOps pipelines, and its named AI delivery is the 4over4 recommendation engine. Its ERP and e-commerce work includes API sync of product catalogues, inventory, orders and customer data with ERP systems, where the question of which system owns a record comes up first.
Delivery runs in a six-step lifecycle from discovery and architecture planning to rollout and ongoing support, and most Netbase projects are delivered on fixed-price contracts. No published Netbase record describes a data catalog or governance platform rollout for a client, and no adoption, cost or time figure is claimed here.
Alternatives: other ways to make data findable
- A shared data dictionary. A single page listing the main tables and their meaning works for a very small team; choose it when there is one analyst group and few tables, and replace it once entries drift from the code.
- BI semantic layer descriptions. Describing metrics and fields in the BI tool helps business users; choose it when most questions are about dashboards, not tables, and keep the table entries in code anyway.
- Warehouse-native catalogs. Cloud warehouses and lakehouse platforms ship their own catalogs with access control and some lineage; choose them when nearly all data lives on one platform.
- A naming convention first. Clear prefixes for staging, intermediate and reporting tables remove much of the "which table?" problem; choose it before any tool.
Limits of this guide
- The field set and the payback matrix are planning aids, not benchmarks; adapt them to your regulation and your team's structure.
- "Twenty people" is a size band. A team with several separate business units may hit the upgrade triggers earlier.
- No tool is recommended here; the right catalog depends on your warehouse, your BI tool and who will run it.
Plan the next step with a Netbase consultant
Frequently asked questions
Often not at first, because dbt documentation already holds descriptions next to the model code and shows a dependency graph. A separate catalog becomes useful when sources, BI objects or a second team fall outside what the dbt project can describe.
Usually only for a small set of fields, such as those behind regulated, financial or contested figures and fields that hold personal data. For most other questions, table-level lineage gives the same answer at a fraction of the upkeep.
The table owner, usually the analytics engineer or data engineer responsible for the model, owns its entry, and the head of data assigns owners. An entry without a named owner goes stale quickly and should block the table from feeding a dashboard.
Move when two or more triggers apply: a second team producing data, consumers nobody can list, a regulated report, a planned migration or repeated incidents from unknown dependencies. Load the entries you already keep in code into the new platform rather than starting over.
Next step
If you want a second opinion on how much catalog and lineage your team needs, Netbase can review your most-used tables and owners and propose a minimum catalog and an upgrade path: book a solution review.
Related services and solutions
Data engineering platforms for reporting and AI Netbase builds data pipelines and analytics foundations for companies whose data is spread across a store, an ERP, a CRM and spreadsheets. We collect it, clean it, model it and check its quality, so reports agree with each other and AI projects start from data people trust. Data engineering is a capability we are growing alongside the platforms we deliver.
Learn More
AI and data services, from readiness check to production feature Netbase provides AI and data services for companies that want to move from an AI idea to a feature that works inside a real product or process: readiness assessment, data preparation, model and automation development, and production support. Delivered AI includes 4over4's recommendation engine, plus RAG, document AI and MLOps projects for unnamed clients; the rest is growing.
Learn More
Enterprise integration hub ready for AI agents: one monitored layer for commerce, ERP and CRM data An enterprise integration hub is a central layer that moves orders, customers, stock and invoices between your commerce, ERP and CRM systems, with monitoring, retries and replay in one place, and scoped APIs that AI agents can call safely. Netbase builds it as a custom layer on the stack you already run, starting from the flows that break most often.
Learn More
Work with Netbase remotely
Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.
Discuss a project
Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.project@netbasejsc.com
+84 937 869 689
91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam
Get in touch
Tell us what you want to build, modernize, or operate.