GCP Prep
AI

Grounding or Fine-Tuning? The Question Every AI Exam Asks

When a generative model gives a wrong answer, the fix depends entirely on why it was wrong. Here is the framework.

GCPGCP Prep EditorialPublished May 29, 2026Updated August 29, 20264 min read
Table of contents

The most common wrong answer in AI questions

A model does not know your company's refund policy, so someone proposes fine-tuning it on your support tickets. This is intuitive and it is wrong, and it is the single most frequently selected incorrect answer on generative AI questions.

Fine-tuning changes behaviour. It does not reliably install facts. A model tuned on support tickets learns to sound like your support team while still inventing the policy details.

The framework

What is wrongThe fixWhy
Format, tone or structure is offBetter prompting with examplesThe model can already do it; it needs clearer instruction.
It does not know your private or current informationGrounding / retrieval augmentationThe information was never in training data. Prompting cannot conjure it.
It consistently misses domain conventions despite good promptsFine-tuningBehaviour needs to change systematically, beyond what examples achieve.
The task exceeds the model's capabilityA more capable model, or a different approachNo technique compensates for a fundamental capability gap.
Identical calls give different answersLower temperature, constrain output formatSampling randomness is causing the variation.

Why grounding is usually the answer

In practice, most production failures of generative AI features come from the model not having the information, not from it being incapable. Retrieval fixes that directly: find the relevant documents, put them in the prompt, and the model answers from them.

  • It is cheaper than tuning, and far cheaper than training.
  • When the source documents change, the answers change immediately. Nothing needs retraining.
  • It produces citations, so users can verify claims.
  • It respects access control — you can filter retrieval to documents the user is allowed to see.

When fine-tuning genuinely is correct

It has real uses, and exam questions do include scenarios where it is the answer. The signal is a systematic behavioural requirement that prompting has already failed to achieve.

  • Output must follow a specific internal format consistently, and few-shot examples have not been reliable.
  • The domain uses vocabulary or conventions the model handles poorly.
  • You want a smaller, cheaper model to match a larger one's behaviour on a narrow task.
  • The instructions needed to get correct behaviour have grown so long that they dominate the context window on every call.

The order to try things

  1. 1Improve the prompt, with examples. Cheapest, fastest, and often sufficient.
  2. 2Add grounding if the model lacks information. This solves most real failures.
  3. 3Try a more capable model if the task is genuinely hard.
  4. 4Fine-tune only if behaviour still needs to change systematically.
  5. 5Train from scratch essentially never, outside of research contexts.

Exam questions frequently give you a scenario and four options corresponding to steps in this list. The correct answer is the earliest step that addresses the actual problem described.

Share this article