This guide is for the CTO or AI lead who hears "we should fine-tune" and must know whether it pays off and what it means for hosting. It sits under our generative AI development service; serving, versioning and monitoring adapted models belong to our responsible AI and MLOps work.
In this guide
- What does LoRA actually change in a model?
- Which five questions decide whether LoRA is worth it?
- LoRA, full fine-tuning or a managed provider?
- Where will the adapted model run, and what does that change?
- How to run a LoRA pilot that can say no
- Where AI fits
- What delivery record exists, and what does not?
- Alternatives: when is parameter-efficient fine-tuning the wrong choice?
- What are the limits of this guide?
- Frequently asked questions
- Next step
What does LoRA actually change in a model?
LoRA, short for low-rank adaptation, freezes the original model weights and trains a small set of extra matrices next to them. The original LoRA paper reports that, against full fine-tuning of GPT-3 175B, it cut trainable parameters by 10,000 times and GPU memory by three times, with quality on par or better on the models it tested.
The trained result is an adapter: a small file that only works on top of the exact base model it was trained against. That one fact drives most of the hosting decisions later in this guide. You can merge the adapter into the base weights and serve one model, or keep it separate and switch adapters per request.
Parameter-efficient fine-tuning, or PEFT, is the wider family. The Hugging Face PEFT documentation describes it as training a small number of extra parameters, which lowers compute and storage costs while aiming for performance comparable to a fully fine-tuned model. QLoRA adds a quantised base model: the QLoRA paper fine-tuned a 65-billion-parameter model on a single 48 GB GPU.
What LoRA does not change is the data effort: collecting, reviewing and testing examples costs the same as for full fine-tuning.
Which five questions decide whether LoRA is worth it?
Answer these in order. One "no" means stop and fix that point first, or choose an alternative from later in this guide.
-
Is the gap behaviour rather than knowledge?
Wrong format, tone, labels or reasoning steps on cases the model has the facts for
- A "no" means
- Missing or changing facts: use retrieval
-
Have prompting and retrieval been measured and found short?
Scores on a frozen test set, with the best prompt and retrieval set-up, below the agreed bar
- A "no" means
- You are guessing; build the test set first
-
Is the task stable for months?
Labels, policy and output schema are settled
- A "no" means
- Every policy change means a new training run
-
Is reviewed training data ready?
Examples curated, labelled to a guide and split away from the test set
- A "no" means
- Data work comes first; training on noise teaches noise
-
Is there a named hosting path and owner?
Someone owns the base model version, the adapter registry and rollback
- A "no" means
- The adapter becomes an orphan the first time the base model changes
Question 2 is the one teams skip. OpenAI's supervised fine-tuning guide also asks for evaluations to be set up before fine-tuning, so the result can be compared with the prompted base model. Question 4 is covered step by step in our guide to preparing fine-tuning data.
Volume is a tie-breaker: a long prompt full of examples is paid for on every call, while an adapted model can carry that behaviour with a shorter prompt. Run that calculation with your own traffic, but never instead of questions 1 to 5.
LoRA, full fine-tuning or a managed provider?
| Criterion | Provider-managed fine-tuning | LoRA or QLoRA on an open-weight model | Full fine-tuning |
|---|---|---|---|
| What you train | The provider's model through its API | Small adapters on a base model you choose | Every weight of the model |
| Hardware you run | None | One or a few GPUs for training and serving | The largest training budget of the three |
| Data control | Training data goes to the provider under its terms | Data and weights can stay in your cloud | Data and weights can stay in your cloud |
| Main risk | Tied to the provider's model lifecycle | Adapter tied to one base model version | Cost, and forgetting skills outside the task |
| Choose it when | You already use that provider and data may leave your cloud | Data must stay in-house, or many task variants share one base | LoRA has been tried and measured short on a hard domain |
Full fine-tuning still has a place. A study titled LoRA Learns Less and Forgets Less found that in standard low-rank settings LoRA substantially underperformed full fine-tuning on programming and mathematics, while keeping more of the base model's ability on other tasks. For most business tasks, which are narrower than those domains, start with LoRA and move up only on measured evidence.
Where will the adapted model run, and what does that change?
Hosting is where a cheap training run turns into a long-term commitment. Three patterns cover most cases.
| Pattern | How it works | Good for | Watch out for |
|---|---|---|---|
| Merged weights | Adapter folded into the base model and served as one model | One task, steady traffic, lowest latency | Every adapter version is a full model copy to store and roll back |
| Many adapters, one base | Base model loaded once; requests name the adapter they need | Several tasks, tenants or languages on shared hardware | Adapter swaps, memory planning and per-adapter monitoring |
| Provider-hosted | The provider serves the tuned model behind its API | Teams without GPU operations | Model retirement dates and data terms set by the provider |
Serving engines now support the second pattern directly. The vLLM LoRA documentation describes per-request adapter selection with minimal overhead, and warns that loading adapters at runtime carries security risks outside an isolated, fully trusted environment. Research systems go further: S-LoRA reports serving thousands of adapters on one GPU or across several.
Whichever pattern you pick, version the base model and each adapter together, keep the frozen test set next to them, and plan the day the base model is replaced: every adapter trained on it must be retrained and re-scored. Our guide to MLOps and LLMOps for production AI systems covers that release pipeline.
How to run a LoRA pilot that can say no
-
Write the acceptance bar
Agree the metric and the score that counts as good enough before any training.
-
Freeze the test set
Label it with your best reviewers and keep it out of every training file.
-
Score the baselines
Run the best prompt and, where facts matter, retrieval against the test set.
-
Train one small adapter
Use a modest reviewed set and default settings; record the base model version.
-
Compare on the same test set
Read errors by type, not only the average, and check other tasks for forgetting.
-
Price the hosting path
Estimate serving, monitoring and retraining effort for the pattern you would use.
-
Decide go, grow or stop
Ship if the bar is met, add data only where errors cluster, or stop and keep the prompted baseline.
A clear "stop" is a useful result: it costs far less than a production adapter nobody can maintain.
Where AI fits
AI is both the subject and a helper here: a stronger model can draft labels and grade outputs during the pilot. Engineers still set the acceptance bar, review every training label, read the error analysis and sign off the hosting plan. Netbase works with commercial and open-weight models from OpenAI, Anthropic (Claude), Google (Gemini) and Meta (Llama), among others, chosen per project. Each item below states how mature it is at Netbase.
- In delivered work
MLOps pipeline
An MLOps pipeline delivered for a client that is not named, the place where base model and adapter versions belong.
- In delivered work
RAG knowledge assistant
A retrieval-augmented knowledge assistant delivered for a client that is not named, the usual first answer when the gap is knowledge.
- Available capability
Parameter-efficient fine-tuning pilots
The five-question gate, pilot steps and hosting choice described on this page; not yet tied to a published case.
What delivery record exists, and what does not?
- What exists. Netbase has delivered retrieval-augmented knowledge assistants, document AI and MLOps pipelines for clients that are not named, and built a WhatsApp chatbot that combined a large language model with a separate intent engine, also for a client that is not named. The named delivered AI feature is 4over4's recommendation engine, which suggests products from browsing history and previous purchases. Most Netbase projects are delivered on fixed-price contracts.
- What does not. No published Netbase case reports a LoRA adapter in production, a training cost, a serving latency or an accuracy gain from fine-tuning, and none is offered here. The gate, comparison and hosting patterns are method guidance; the figures on this page come from the cited sources.
Alternatives: when is parameter-efficient fine-tuning the wrong choice?
- Better prompts with examples. Choose them when the task still changes every few weeks or when a few examples in the prompt already meet the bar.
- Retrieval over your documents. Choose it when answers fail because facts are missing, private or changing. Our guide to RAG architecture for knowledge assistants and the enterprise knowledge and RAG service cover that path.
- A smaller model for one narrow task. Choose it when the output is fixed and volume is high; LoRA is often how that model is adapted, and our guide to small language models for narrow business tasks covers the size decision.
- Rules or a classic classifier. Choose them when the pattern is fixed, such as known codes or structured fields, and a simple model already passes the test set.
What are the limits of this guide?
- It covers supervised adaptation of text models; preference tuning and image models need their own checks.
- The external figures were measured on public models and benchmarks; your own test set decides the real gain.
Plan the next step with a Netbase consultant
Frequently asked questions
Often close on narrow tasks, and the original paper reports on-par quality on the models it tested. On hard domains such as programming and mathematics, research found LoRA learns less than full fine-tuning, so measure on your own test set.
Usually not. Fine-tuning changes behaviour such as format, tone and labelling; facts that change or must be cited are better supplied through retrieval, which can be updated without a training run.
No. Provider-managed fine-tuning needs none, and an open-weight base model with adapters can run on rented cloud GPUs. Own hosting is a data-control and volume decision, not a requirement of the method.
It stops matching. An adapter only works on the base model version it was trained on, so plan to retrain it and re-score it on the frozen test set before switching.
Next step
Send us the task, a sample of failing outputs and the prompt you use today, and we will book a solution review to run the five questions with you and plan a pilot that can say no. You can also see our MLOps service, which covers serving and versioning adapted models.
Related services and solutions
Generative AI development with evaluation and human review Netbase builds generative AI features for product and operations teams: drafting, summarising, rewriting and content generation inside the software your people and customers already use. Every feature ships with an evaluation set, a human review step where output matters and limits on cost. Generative AI is a capability we are growing, and we label its maturity honestly.
Learn More
Enterprise AI knowledge assistants: RAG engineering with access control Netbase engineers retrieval-augmented generation (RAG) over company knowledge: the ingestion, search, permission checks, cited answers and evaluation that let an internal assistant or a website RAG chatbot answer from your own documents. It is a growth capability with access control designed in from the first prototype, and Netbase has delivered a RAG knowledge assistant for an unnamed client.
Learn More
Responsible AI and MLOps for AI in production MLOps and responsible AI keep an AI feature trustworthy after launch. Netbase's practice for monitored, governed AI in production covers evaluation before every release, versioning of models, prompts and data, monitoring of quality and cost, and incident controls with a named owner. It is a growth capability, backed by an MLOps pipeline delivered for an unnamed client.
Learn More
AI knowledge assistant: answers from company knowledge, with sources and access control An AI knowledge assistant is an internal tool that answers employees' questions from your company's own documents and systems, cites the source behind every answer and shows each person only what they may see. Netbase offers it as a growth capability, designed with retrieval, evaluation and human review from the first pilot, and has delivered one for an unnamed client.
Learn More
Work with Netbase remotely
Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.
Discuss a project
Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.project@netbasejsc.com
+84 937 869 689
91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam
Get in touch
Tell us what you want to build, modernize, or operate.