This guide is for innovation leads, product owners, CTOs and operations directors who are asked to fund a first pilot of connected devices, a shared ledger or an augmented or virtual reality feature. It sits in the IoT development cluster next to the IoT project scoping checklist, which covers what to build; this page covers how to prove it is worth building.
In this guide
- Why do so many emerging-tech pilots end without a decision?
- What belongs in a one-page pilot charter?
- Proof criteria and stop rules by technology
- How do you run the pilot from charter to decision?
- Is it a proof of concept, a pilot or a limited release?
- When should you not run a pilot at all?
- Where AI fits
- What delivery record exists, and what does not
- Alternatives: who can run the pilot
- Limits of this guide
- Frequently asked questions
- Next step
Why do so many emerging-tech pilots end without a decision?
Most stalled pilots were never designed to end. The team agreed to "try the technology" but not what result would count, who would judge it or by when. The question is too broad, there is no baseline, and nobody wrote down the result that would stop the work, so the pilot is extended and the sponsor loses interest.
Emerging technology adds one more trap: a sensor that reports or a headset demo that impresses proves feasibility, not that anyone will pay for, operate or repeatedly use the result.
What belongs in a one-page pilot charter?
A charter is a single page that the sponsor, the delivery team and the people who will operate the result all sign before work starts. Netbase uses seven fields.
-
Question
- What to write
- One decision the pilot informs, phrased as yes or no
- Weak version to avoid
- "Explore IoT"
-
Scope boundary
- What to write
- One site, one workflow, one user group; what is out
- Weak version to avoid
- "The whole operation"
-
Baseline
- What to write
- How the work runs today, measured before the pilot
- Weak version to avoid
- "We think it is slow"
-
Proof criteria
- What to write
- Two to four measurable results with thresholds
- Weak version to avoid
- "Users like it"
-
Stop rules
- What to write
- Results or events that end the pilot early
- Weak version to avoid
- None written
-
Decision owner and date
- What to write
- One named role and a calendar date
- Weak version to avoid
- "The committee, later"
-
What happens on go
- What to write
- Next budget step, owner and production scope
- Weak version to avoid
- "We will see"
A pilot that covers several sites, every device type or all customers is a rollout in disguise; when it struggles, nobody can tell whether the idea or the scale is the problem.
If no baseline exists, measuring it is the pilot's first task, and the decision date moves back by the time it takes.
Proof criteria and stop rules by technology
Each technology fails in its own way, so the criteria differ. The table gives a starting set to adapt; the thresholds themselves come from the buyer's baseline and business case.
| Technology | What the pilot must prove | Typical proof criteria | Stop rules agreed in advance |
|---|---|---|---|
| IoT | Real devices on the real site deliver usable data into a business decision | Data completeness from installed devices; time from install to first data; an operator acts on an alert; signed remote update works | Coverage or power cannot be fixed without new hardware; data never reaches a decision; install needs a specialist every time |
| Web3 | Several parties need a shared record none of them controls, and accept the operating duties | Each party runs its part of the flow; key custody and recovery work; the ledger answers a question a database could not | A single trusted operator would do; partners will not hold keys or pay fees; legal or custody review blocks go-live |
| XR | Target users choose to use the feature again on the devices they actually own | Repeat use within the pilot window; task completion on target devices; content can be updated without the vendor | Most target users lack capable devices; use drops after the first session; content cost per item breaks the business case |
For IoT, check device identity and software update against the NIST IR 8259A baseline. The IoT row follows Blues' pilot planning guide, which splits pilot metrics into technical, operational and commercial groups and lists scaling, iterating or stopping as the three possible outcomes. The Web3 row asks the question behind NIST's blockchain technology overview: does the problem need shared, tamper-evident records among parties that do not fully trust one another?
The XR row depends on reach. If the feature runs in the browser through the W3C WebXR Device API or a platform viewer, device coverage is a criterion; if it needs a native app, install and repeat use are. The WebXR or native AR guide covers that choice.
How do you run the pilot from charter to decision?
-
Write the question first
One decision, answerable yes or no.
-
Measure the baseline
Record how the work runs today: time, error rate, cost driver or usage, whichever the question needs.
-
Set two to four proof criteria
Each one has a threshold, a data source and a person who reads it. More criteria make every result arguable.
-
Write the stop rules
List the results and events that end the pilot early, including safety, security and data protection events, and who may call a stop.
-
Fix the decision owner and date
One role decides on one calendar date, from the criteria, not from impressions.
-
Build the smallest end-to-end path
One real path from device, wallet or headset to a business outcome beats a broad prototype that never reaches a user.
-
Review weekly against the charter
Trouble shows early and scope changes stay visible.
-
Decide go, adjust or stop on the date
Adjust means a new, shorter charter with a changed question, not an open extension.
Is it a proof of concept, a pilot or a limited release?
The three are often mixed up, and each needs different criteria. A proof of concept asks whether something can work at all, usually in a lab or on test data. A pilot asks whether it works here, for these users, at an acceptable cost. A limited release asks whether it holds up at production quality for a first group.
A proof of concept judged by pilot criteria looks like a failure; a pilot judged by proof-of-concept criteria is declared a success, then dies because nobody planned operations. The charter should state which of the three it is.
When should you not run a pilot at all?
Skip the pilot when a cheaper test answers the question. A spreadsheet trial or a manual process can test whether people act on sensor-style data before any device is bought. A plain database answers most shared-record questions that turn out to have one trusted operator, as the wallet custody guide and its siblings argue for Web3. A video or a web 3D viewer can test interest before a headset build.
Also skip it when nobody can own the result. If no team will operate the devices, hold the keys or update the 3D content after the pilot, a positive result has nowhere to go, and the money is better spent on the operating plan first.
Where AI fits
AI helps a pilot in the reading, not the deciding. It can summarise weekly results against the charter, flag device readings that look abnormal and draft the review notes. Engineers still set the thresholds with the business, check each flagged reading against the source data and keep the stop decision with the named owner. Netbase works with the major commercial and open-source AI tools and models, chosen per project.
- In delivered work
Recommendation engine
A recommendation engine delivered for 4over4's online printing store; it is not a device or ledger pilot.
- Available capability
Anomaly detection on device data
AI with IoT is a Netbase capability area; not yet tied to a published case.
What delivery record exists, and what does not
No Netbase IoT, Web3 or XR pilot is published yet, and this guide claims none. Netbase designs, builds and integrates software across technologies and industries, including blockchain development and AR and VR development, as a capability statement rather than a case. IoT with AI at the edge is an investment direction, and mobile and IoT are among the delivery trends Netbase covers.
The delivery method behind the charter is on record. Netbase follows a six-step lifecycle that starts with discovery and strategic alignment and runs agile execution with outcome-based milestones, governed by weekly reviews and KPI dashboards. Netbase has delivered projects for clients in manufacturing and logistics that are not named, including an operations and integration platform; that record publishes no figures and involves no devices.
Most Netbase projects are delivered on fixed-price contracts, and milestone-based models are also offered, which suits a pilot with a fixed decision date. Consulting, solutions and custom development lead; dedicated teams are a secondary option. No pilot duration, device count, cost or result is claimed here.
Alternatives: who can run the pilot
| Option | Strength | Risk | Choose it when |
|---|---|---|---|
| In-house team | Knows the site, users and data | Little time; may lack device, ledger or 3D skills | The question is narrow and a team already owns the area |
| Platform vendor pilot | Fast start on its own product | Criteria favour the product; lock-in before a decision | You have already chosen the platform and test fit only |
| Independent build partner | Writes criteria and builds the path end to end | Needs a clear owner on the buyer side | Several layers are involved and you want a neutral go or stop |
| Research or lab project | Good for open technical questions | Rarely reaches real users or operations | You need a proof of concept, not a pilot |
Whoever runs it, the sponsor signs the question, the criteria and the stop rules.
Limits of this guide
- The proof criteria are examples. Thresholds must come from your own baseline and business case.
- Regulated settings such as medical devices, payments or safety systems add approval steps that a pilot charter does not replace.
- Web3 legal and custody questions need advice in your jurisdiction; this page does not give it.
Plan the next step with a Netbase consultant
Frequently asked questions
Long enough to reach the decision date with enough data to read the proof criteria, and no longer. The charter fixes the date in advance; the right length depends on the baseline, install time and how often the measured event happens.
Exit criteria are the results that decide go, adjust or stop on the decision date. Stop rules are results or events that end the pilot early, before that date, such as a security incident or a fault that needs new hardware.
One business role who will fund and operate the result, named in the charter. A committee can advise, but a single owner deciding on a fixed date keeps the pilot from drifting.
Next step
If you have a pilot idea or a draft charter, book a solution review and we will mark the question, the criteria and the stop rules to settle before you build.
Related services and solutions
IoT development with AI analytics that connects devices to the business Netbase provides IoT development for manufacturers, logistics operators and product companies to connect devices and sensor data to the business platforms that act on them, with AI that spots anomalies and forecasts what needs attention. We design the device-to-cloud data path, build the back end, dashboards and integrations, and treat security, data ownership and AI as design decisions.
Learn More
Blockchain development: smart contracts, wallets and tokenisation built to last Netbase designs and builds blockchain applications: smart contracts, wallet integrations, tokens and permissioned ledgers, connected to the web, mobile and business systems you already run. We start by checking whether a blockchain is the right tool at all, test every contract against known attack patterns and plan an independent audit before anything holds real value.
Learn More
AR and VR development: apps, guided work and remote assistance Netbase designs and builds augmented and virtual reality applications: product visualisation in a phone or browser, guided work and remote assistance for field and workshop teams, and VR training and walkthroughs. Each one is connected to the catalogue, service and learning systems you already run. We start by checking whether spatial content helps the task at all.
Learn More
Enterprise integration hub ready for AI agents: one monitored layer for commerce, ERP and CRM data An enterprise integration hub is a central layer that moves orders, customers, stock and invoices between your commerce, ERP and CRM systems, with monitoring, retries and replay in one place, and scoped APIs that AI agents can call safely. Netbase builds it as a custom layer on the stack you already run, starting from the flows that break most often.
Learn More
Work with Netbase remotely
Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.
Discuss a project
Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.project@netbasejsc.com
+84 937 869 689
91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam
Get in touch
Tell us what you want to build, modernize, or operate.