Generative AI

Generative models produce new content — text, images, code, audio — rather than classifying or scoring existing content. Large language models work by repeatedly predicting the most plausible continuation of a sequence, having learned patterns from an enormous training corpus.

This mechanism explains both the capability and the characteristic failure. The model produces plausible continuations. Plausible and true usually coincide, but when they do not, the model is equally fluent and equally confident.

Foundation models change the economics of building AI features. Where a custom model once required labelled data, training infrastructure and specialist expertise, many problems can now be addressed by calling an existing model with well-designed input.

The consequence is that the hard work moves. It is no longer mostly training; it is context, grounding, evaluation and knowing what to do when output is wrong.

• Foundation model — a large model trained on broad data, adaptable to many tasks without task-specific training.
• Token — the unit models process. Both input and output are billed in tokens, which is the basis of the cost model.
• Context window — how much text the model can consider at once. Large but finite, and the constraint that makes retrieval necessary.
• Prompt — the input, including instructions, examples and any supplied context.
• Few-shot prompting — including examples in the prompt to demonstrate the desired output format and style.
• Grounding — supplying authoritative source material so the model answers from it rather than from memory.
• Retrieval-augmented generation (RAG) — retrieving relevant documents at query time and including them in the prompt.
• Embedding — a numeric representation of meaning, used to find semantically similar content rather than keyword matches.
• Vector database — stores embeddings and finds nearest matches quickly. The retrieval half of RAG.
• Fine-tuning — further training on your own examples to adjust style, format or domain behaviour.
• Hallucination — fluent, confident output that is factually wrong. The defining failure mode.
• Temperature — controls randomness. Low for factual tasks, higher for creative ones.

This is the framework worth memorising, and the shape of a large proportion of exam questions on the subject. When output is wrong, the fix depends on why it is wrong.

Beyond accuracy, this gives you two things that matter in production: citations, so users can verify claims, and access control, because you can filter retrieval to documents the user is permitted to see.

• Build an evaluation set of representative inputs with known good outputs before you deploy, not after something goes wrong.
• Automated metrics are a screening tool; human review remains necessary for anything subjective.
• Keep a human in the loop wherever an error carries real consequence — medical, legal, financial or safety contexts.
• Be explicit with users about what is AI-generated and how confident they should be in it.
• Consider privacy before sending data to a model, including whether inputs are retained.
• Test for biased or harmful output deliberately; it will not surface on its own during ordinary development.

• Question answering grounded in a company's own documentation.
• Summarising long documents, transcripts and support conversations.
• Drafting content that a person reviews and edits before use.
• Extracting structured data from unstructured text and documents.
• Code generation, explanation and review assistance.
• Semantic search that finds relevant content without exact keyword matches.