Skip to main content

What are you looking for?

Explore our services and discover how we can help you achieve your goals

Client Workspace

Supporting AI Features After Launch: An AI Support SLA Addendum

A live AI feature needs support terms an ordinary application SLA lacks: a severity scale for quality drops, a scheduled drift check against a frozen evaluation set, a change class for prompt and model edits, and written triggers that choose rollback, prompt fix, retrieval refresh or retraining. Netbase writes these into the support scope it agrees per contract.

Book a solution review See the related solution

By Netbase AI and Automation Engineering Desk · Reviewed by David (CEO) · Updated · 10 min read

star

This guide is for the CTO, engineering lead or product owner whose application already has a support agreement and now carries an LLM answer, a retrieval assistant, a classifier or a recommendation. It sits beside the general bug-fix workflow and the support tier model of this cluster, and the legacy modernization service it belongs to.

In this guide

Why does an ordinary application SLA miss AI problems?

An ordinary application SLA measures whether the system is up and whether a reported bug is fixed. An AI feature can stay up, return a fast answer and still be wrong more often than last month, so none of the usual targets fires. The support agreement needs a measure of answer quality, not only availability.

The Google SRE book defines a service level indicator as a carefully defined quantitative measure of some aspect of the service, and an SLA as a contract that adds consequences to the objectives it contains, per the Google SRE book. For an AI feature the missing indicator is a quality score on a fixed evaluation set, and the missing obligations are who runs that check, who is told when it drops and what happens next.

Three causes make AI support different from bug support. The inputs move: customers ask new questions and data feeding the model changes. The model moves: a provider updates or retires a hosted model, and Anthropic, for example, gives at least 60 days' notice before retiring a publicly released model on its model deprecations page. The team moves things: a prompt edit or a new retrieval source changes answers. A support agreement that covers only the third cause misses most of the work.

Which AI quality severities belong in the support agreement?

Add four AI rows to the severity matrix you already use for outages. Each row names what counts as evidence, so nobody argues about whether a quality drop is real.

  • Q1 harmful or unsafe output

    What counts as evidence
    One confirmed answer that leaks data, breaks a policy or acts without approval
    First action
    Switch the feature off or fall back to the non-AI path
    Who is told
    Product owner, security contact
  • Q2 quality below the agreed floor

    What counts as evidence
    Score on the frozen evaluation set falls below the floor written in the addendum
    First action
    Roll back the last change or pin the previous model
    Who is told
    Product owner
  • Q3 drift trend

    What counts as evidence
    Score falls between the floor and the baseline on two scheduled checks in a row
    First action
    Open an investigation ticket with the trace samples
    Who is told
    Feature owner
  • Q4 cost or latency drift

    What counts as evidence
    Spend or response time per request moves outside the agreed band with quality unchanged
    First action
    Review routing, caching and prompt length
    Who is told
    Feature owner, finance contact

The floor and the bands are numbers you and your support partner agree for your feature; this guide does not suggest values. Response and resolution targets for each row are set per contract, in the same way as for outage severities.

Drift checks, prompt changes and triggers: how to choose the response

A confirmed drop needs one of four responses. The table below is the trigger-to-response part of the addendum; the right column is the rule that keeps the team from retraining by reflex.

Trigger Likely cause Response Do not
Quality falls right after a prompt, model or retrieval change The change itself Roll back to the last version that passed the evaluation set Patch forward under time pressure
Provider announces a model retirement Model lifecycle Run the evaluation set on the replacement, then release as a planned change Wait for the retirement date
Inputs shift (new topics, new phrasing, new product range) Data drift Update the prompt, examples or retrieval sources; extend the evaluation set Retrain before the evaluation set reflects the new inputs
The right answer itself changed (new policy, new rules, new catalogue) Concept drift Refresh the retrieval index; retrain a trained model on labelled fresh data Treat it as a model bug
Scores degrade slowly with no change and no input shift Unknown Investigate traces and labels before acting Retrain without a cause

Google Cloud lists five ways to trigger a retraining pipeline: on demand, on a schedule, when new data is available, on performance degradation and on significant changes in data distribution, in its MLOps architecture guide. Latitude maps prompt or model drift to an immediate revert and concept drift to retraining or updated retrieval grounding. For an LLM feature that calls a hosted model, "retraining" usually means changing the prompt, examples or retrieval sources; fine-tuning is the last option, not the first.

How to write the AI support addendum

  1. Freeze an evaluation set

    Collect real requests with agreed good answers, including the hard and the unsafe ones. Name its owner on the client side; the support team may extend it but not change agreed answers alone.

  2. Agree the floor and the baseline

    Score the live feature on the set at handover. The baseline is that score; the floor is the level below which Q2 applies.

  3. Write the drift-check calendar

    State what runs and how often: evaluation-set replay, input distribution review, cost and latency review, provider lifecycle check. Keep the frequency proportional to how fast inputs change.

  4. Define the prompt and model change class

    Every prompt, model, example or retrieval-source change is a versioned change with an evaluation run, a named approver and a rollback path, released through the same change and versioning process as code.

  5. Write the trigger table

    Copy the table above, add your own thresholds and name who may order a rollback without waiting for approval.

  6. Add the AI rows to the severity matrix

    Q1 to Q4, with evidence, first action, contacts and the response targets your contract sets.

  7. Report monthly

    One page: evaluation scores against baseline, changes released, triggers fired and their outcome, open investigations and model retirement dates ahead.

For the detection side of step 3, see AI agent drift detection, which covers golden-set replay and alert thresholds in detail.

Who owns the evaluation set when support is outsourced?

The client owns the evaluation set and its agreed answers, because only the business can say what a correct answer is. The support partner runs it, proposes additions from real failures and reports scores. If the partner also writes the expected answers, a quality drop can disappear simply by changing the answer key.

Where AI fits

AI helps run the support routine, not decide it. A model can cluster failing requests, draft new evaluation cases from support tickets and summarise trace samples for the person on duty. Engineers still confirm each case, approve every prompt or model change and decide on rollback. Netbase works with the major commercial and open-source AI tools and models, chosen per project.

What delivery record exists, and what does not?

Netbase runs, maintains and upgrades what it builds, remotely and alongside the client's in-house team, under an agreed support scope, with response targets set per contract. NDAs, DPAs and SLAs are available on request.

Netbase has delivered AI projects for clients, including retrieval-augmented knowledge assistants, document AI and MLOps pipelines; these clients are not named (see the anonymised MLOps pipeline record). It also built an AI chatbot for WhatsApp Business connected to a client's CRM, delivered in milestones through to documentation, knowledge transfer and a support period; that client is not named either. The delivered AI feature named with a client is 4over4's recommendation engine, which suggests products from browsing history and previous purchases.

Most Netbase projects are delivered on fixed-price contracts; milestone-based, monthly team retainer and KPI-linked models are also offered. No response time, evaluation score, drift rate or retraining result is published; the addendum structure comes from public guidance and delivery practice. The service offer itself is described on the AI support and maintenance page.

Alternatives: where the AI support terms can live

Option Strength Risk Choose it when
AI rows inside the existing application SLA One agreement, one severity matrix, one report AI checks get skipped when the team is busy with bugs One AI feature, low change rate, same team supports both
Separate AI support agreement Clear owner, own calendar and budget for evaluation work Two partners can blame each other at the boundary Several AI features, or a different team runs the AI
In-house AI owner, partner supports the rest Business knowledge stays close to the answer key One person becomes a single point of failure You have a product owner who can own the evaluation set

When not to write an addendum: a feature that only drafts text a person always edits before use, with no automated action and little volume, may need nothing more than a quarterly review and the provider lifecycle check.

Limits of this guide

  • Thresholds, floors and response targets are yours to set; the guide gives the structure, not the values.
  • It covers features in production, not model development or the first launch.
  • It is not legal advice; regulated uses of AI carry their own obligations.

Plan the next step with a Netbase consultant

Frequently asked questions

Check as often as its inputs change. A feature that answers questions about a catalogue that changes weekly needs a weekly evaluation run; one that classifies stable documents may need a monthly run. Add an unscheduled run after every prompt, model or retrieval change.

Retrain only when the cause is confirmed and a prompt, example or retrieval change cannot fix it. For a feature built on a hosted language model, most drift is fixed in the prompt or the retrieval sources, and fine-tuning is a last resort.

The product owner approves a prompt change after it passes the agreed evaluation set, and the support team releases it with a rollback path. A rollback to the last passing version can be ordered without waiting for approval.

Treat the announced retirement as a planned change. Run the evaluation set on the recommended replacement, fix prompts where scores drop and release before the retirement date, so requests never fail on the day the old model stops.

Next step

If an AI feature in your application needs support terms your team and partner can actually run, or a drift check set up before the next model retirement, book a solution review.

AI-assisted legacy modernization that keeps your orders and data safe AI-assisted legacy modernization that keeps your orders and data safe

Netbase provides legacy application modernization services for commerce and operations teams to move ageing platforms, such as Magento 1 stores, onto supported technology without losing orders, customers or data. AI speeds up code analysis and data mapping; every migration runs on a written, human-approved plan with rehearsal, staged cut-over and rollback, proven by Netztech's move to Magento 2.

Learn More
line
Responsible AI and MLOps for AI in production Responsible AI and MLOps for AI in production

MLOps and responsible AI keep an AI feature trustworthy after launch. Netbase's practice for monitored, governed AI in production covers evaluation before every release, versioning of models, prompts and data, monitoring of quality and cost, and incident controls with a named owner. It is a growth capability, backed by an MLOps pipeline delivered for an unnamed client.

Learn More
line
AI support and maintenance for models, LLM features and agents in production AI support and maintenance for models, LLM features and agents in production

Netbase keeps AI systems useful after launch. We monitor quality, drift, latency and cost, retrain or re-prompt when the data moves, migrate to a replacement model before a provider retires the old one, and run incident response with rollback and a named owner, for models, LLM features and agents already in production.

Learn More
line
AI knowledge assistant: answers from company knowledge, with sources and access control AI knowledge assistant: answers from company knowledge, with sources and access control

An AI knowledge assistant is an internal tool that answers employees' questions from your company's own documents and systems, cites the source behind every answer and shows each person only what they may see. Netbase offers it as a growth capability, designed with retrieval, evaluation and human review from the first pilot, and has delivered one for an unnamed client.

Learn More
line

Work with Netbase remotely

Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.

Contact Netbase

Discuss a project

Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.
Project enquiries

project@netbasejsc.com

WhatsApp

+84 937 869 689

Office address

91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam

Get in touch

Tell us what you want to build, modernize, or operate.

Tell us what you want to build, modernize, or operate.

Contact Netbase