For most companies, the right move is to start with retrieval, not fine-tuning. Build a RAG system, test it against real questions, and only fine-tune when you need consistent behaviour, a strict output format, or want to shrink a large model into a cheaper one through distillation. Fine-tuning solves format and behaviour problems, not knowledge gaps, and it carries real GDPR and data-governance weight once your company’s own data enters the training set. Done.lu’s own project work backs that sequencing.

---

TL;DR:

• Fine-tuning mainly modifies a model’s default behavior and is suitable for tasks like enforcing a specific tone or formatting, not for updating knowledge.
• Most business use cases benefit more from retrieval-based systems and prompt engineering before considering fine-tuning due to cost and complexity.
• Data preparation for fine-tuning must prioritize security, anonymization, and clear documentation, especially for regulated sectors with GDPR concerns.
• The decision to fine-tune should depend on whether information changes frequently, citations are needed, output format consistency is critical, or query costs are a constraint.
• Maintaining a fine-tuned model requires ongoing monitoring of performance, format accuracy, and prompt retraining or retraining when data drift occurs.

---

Table of Contents

• What fine-tuning actually changes and practical business use cases
• Fine-tuning techniques, data scale, and rough cost brackets
• Preparing company data safely for fine-tuning
• RAG first, then fine-tune: a short decision checklist
• Evaluating and maintaining a fine-tuned model after launch
• Practitioner view: Done.lu’s typical project flow for company-data fine-tuning
• How Done.lu can help with GDPR-aware fine-tuning pilots
• Sources

What fine-tuning actually changes and practical business use cases

Fine-tuning adjusts a model’s internal weights so it behaves differently by default, no prompt required. RAG supplies knowledge at the moment of the query; fine-tuning bakes a pattern of behaviour into the model itself. That distinction decides which technique solves which problem.

If your issue is “the model doesn’t know our latest pricing,” retrieval fixes it instantly. If your issue is “the model keeps answering in the wrong tone, ignoring our output format, or missing fields in structured extraction,” that’s a behaviour problem, and fine-tuning is the right tool.

Real business cases where fine-tuning earns its cost:

• Consistent customer support tone. A model tuned on your past support transcripts stops sounding like a generic chatbot and starts sounding like your brand.
• Regulatory summary formatting. Compliance teams often need a fixed structure every time. Fine-tuning enforces that far more reliably than prompting alone.
• Structured JSON extraction from forms. Invoices, contracts, and application forms benefit from a model trained specifically on your document layouts.
• Distillation for cost and speed. Tuning a smaller open model on outputs from a frontier model gives near-equivalent quality at a fraction of the inference cost.

The limitations matter just as much. A fine-tuned model has no built-in way to cite where an answer came from, it will not reliably surface sources the way retrieval does, and its knowledge freezes at training time. For anything where facts change monthly, or where a wrong answer needs to be traceable, fine-tuning alone is the wrong tool.

Fine-tuning techniques, data scale, and rough cost brackets

Four techniques cover almost every business scenario, and they are not interchangeable.

• Full fine-tuning retrains all of a model’s parameters. It gives the deepest behavioural change but demands the most compute and the most data, so it rarely makes sense for an SME project.
• LoRA and QLoRA freeze most of the model and train small adapter layers instead. Parameter-efficient adapters cut compute cost dramatically compared with full fine-tuning, which is why most SME projects Done.lu has scoped use one of these two.
• Instruction tuning trains a model on example instructions and desired responses, useful when you want a model to follow a specific house style or workflow.
• RLHF (reinforcement learning from human feedback) aligns outputs with human preference rankings rather than fixed examples. It’s powerful but heavier to run, and most SMEs never need it.

Pro Tip: Start with a few hundred curated input to output examples before committing budget to thousands. If the small set doesn’t move quality metrics, a bigger dataset usually won’t fix the underlying problem either.

Data volume typically runs from a few hundred to a few thousand curated pairs, formatted as input paired with the exact desired output. Timelines depend more on dataset curation than on training itself: expect curation, validation, and at least one retrain cycle to dominate the schedule, not GPU time.

Distilling a frontier model’s outputs into a smaller open model is the pattern that pays off fastest for high-volume use cases, since it keeps quality close to the original while cutting the per-query cost that adds up at scale.

Preparing company data safely for fine-tuning

Company data used for training carries legal weight the moment it includes anything personal or confidential. Treat data preparation as a governance exercise first, a machine learning task second.

1. Inventory every source. List where the data lives (CRM exports, support tickets, contracts) and classify each source by sensitivity: personal data, trade secrets, or neither.
2. Strip or pseudonymise personal data. Remove names, emails, and identifiers before anything touches a training pipeline.
3. Deduplicate and clean. Repeated or contradictory examples confuse a model faster than they help it.
4. Build clear input to output pairs. Every example should show exactly what the model receives and exactly what you want back.
5. Keep an audit trail. Log what went into training, when, and who approved it.

Regulated sectors, legal, finance, healthcare, commonly need private or EU-only infrastructure rather than a generic public cloud endpoint, precisely because AI adoption introduces new organisational risks that leadership needs to weigh against data exposure. A Data Processing Agreement with any vendor, a documented retention policy, and a check on where the model actually runs are not optional extras here. They’re procurement requirements. Our guide to GDPR-compliant AI walks through the specifics in more depth.

RAG first, then fine-tune: a short decision checklist

Before committing to a fine-tuning project, run through four questions.

• Does the information change frequently? If yes, retrieval wins. Fine-tuning freezes knowledge at training time.
• Do you need citations or provenance? Fine-tuned models don’t reliably show their sources. Retrieval does.
• Is a consistent output format mandatory? This is where fine-tuning genuinely helps, especially for structured extraction or fixed-tone communication.
• Is latency or per-query cost a real constraint? If you’re running thousands of queries a day against an expensive frontier model, distillation into a smaller tuned model often solves the economics.

The sequence that works in practice: build a RAG prototype, test it against a real evaluation set, and try fixing failures with better prompts or pipeline changes first. Only fine-tune once retrieval and prompting have been tried and residual behaviour problems remain. Skipping straight to fine-tuning without that step almost always means paying for a solution to the wrong problem.

Evaluating and maintaining a fine-tuned model after launch

A fine-tuned model isn’t a one-off deliverable. It needs the same operational discipline as any other production system.

Track these metrics from day one:

• Format compliance rate, the percentage of outputs that match your required schema exactly.
• Human quality scores on a representative sample, reviewed on a fixed schedule.
• Precision and recall for extraction tasks, not just overall accuracy.
• Inference latency and cost per query, compared against the model it replaced.

Roll out changes with A/B tests and a canary phase before a full switch, with clear rollback conditions defined in advance. Monitoring should include periodic sample audits, drift detection as your business data evolves, and scheduled re-evaluation that triggers a retrain when performance slips.

Pro Tip: Set a confidence threshold that routes uncertain outputs back to retrieval or a human reviewer, rather than letting a fine-tuned model guess. That single guardrail catches most of the embarrassing failures before a customer sees them.

Practitioner view: Done.lu’s typical project flow for company-data fine-tuning

In our experience, every fine-tuning project that succeeds follows roughly the same shape. Audit the data and business need first, prototype with RAG, and only move to a curated training set once retrieval has shown exactly where it falls short.

We’ve seen this with clients in regulated sectors: the audit stage usually surfaces a data sovereignty question before it surfaces a model question. That’s why on-premise or EU-only deployment gets scoped early, not bolted on afterwards. A small pilot fine-tune, measured against agreed KPIs, tells you within weeks whether full deployment is worth the investment.

— Thomas

How Done.lu can help with GDPR-aware fine-tuning pilots

Done is the practical alternative to hiring a specialist AI team from scratch. Instead of a lengthy build cycle, you get a scoped pilot: an audit, a RAG prototype, and a decision on whether fine-tuning is even worth doing, before you commit to a full training project.

Our services cover the whole path: AI audits, RAG prototyping, curated fine-tuning where the data justifies it, and private or on-premise deployment for teams handling sensitive records. Sales and support teams often see the fastest wins, and partner resources like Sales Label Consulting’s guide to AI workflows show how far automation can go once the underlying model behaves consistently.

Pricing is transparent from the first conversation, with no hidden setup fees. If you want to know whether fine-tuning is worth it for your business specifically, start with our AI consulting services for SME growth and book a short audit before spending on a full build.

Sources

• RAG vs Fine-Tuning Explained: What They Actually Do and When to Use Each | Towards Data Science
• RAG vs Fine-Tuning in 2026: A Decision Framework for LLM Teams
• Tyrimas parodė, kad DI įmonėse kelia naujas rizikas: vadovus ragina atsižvelgti į dvi sritis | BNS Spaudos centras

Recommended

• GDPR AI compliance: a practical guide for European SMEs
• How to implement GDPR-compliant automation for SMEs
• AI and GDPR: A clear guide for European business owners
• AI data privacy: a practical guide for SME leaders