2: The Art of the Prompt

What is prompting?

Prompts are the input instructions for AI tools to perform tasks. They can include text, data (such as tables) or images. The input of a prompt is converted into tokens by the AI tool. Tokens represent the prompt elements converted into a form that the tool understands. These are processed and converted back into words or other outputs. The tool prompt length is limited by the number of tokens allowed to be input. A prompt is better understood as a set of constraints than as an instruction. The model's knowledge was fixed during training and your prompt adds nothing to it; what the prompt does is force the model to draw from one particular region of what it already contains. A direct request with no examples narrows the enormous space of possible outputs down to the neighborhood you described. This is why small changes in wording produce large changes in output — you are not persuading the model, you are redrawing the boundary it works inside. Supplying one or more examples of the pattern you want narrows that boundary considerably further. This is commonly described as few-shot prompting, in contrast to zero-shot prompting where no examples are given. The examples do not teach the model a new skill; they identify which of its existing patterns to draw on and in what register. In practice, two or three well-chosen examples routinely outperform a paragraph describing what you want, because an example shows the target directly rather than asking the model to infer it from a description.

Components of a good prompt

Several frameworks exist for organizing a prompt, and they differ mainly in vocabulary. One compact version holds that a strong prompt has three parts: the task, meaning what you want the tool to do; the instructions, meaning how you want it done; and the context, meaning what you want the tool to know. Another common framework is the role–task–context–format framework.

^{Generated by Google's Nano Banana 2 image generation model}

Context is the component most often omitted and the one that most improves results. The reason follows directly from Chapter 1: the model predicts from the material in front of it. Deprived of context, it predicts the most generic document its training would suggest. Supplied with real context, it predicts something shaped like your actual situation. A short set of habits reliably improves output. Start simple and build through conversation rather than attempting one perfect prompt. Be specific. State what you want rather than what you want avoided, since naming the thing to avoid places those very words into play. Ask for several options rather than one. Fence off pasted material clearly so the tool can tell your instructions from your source text. Start a fresh conversation when a thread has accumulated dead ends, because the tool rereads the entire conversation each time and old mistakes continue to steer new answers. And when writing the prompt is itself the hard part, ask the tool to draft the prompt.

Types of Prompts

Why the Same Prompt Produces Different Answers

Students frequently notice that an identical question asked twice produces two different answers, and reasonably conclude that the tool is unreliable. The behavior is real, but it is a design property rather than a malfunction, and understanding it converts an irritation into a control. Unlike a calculator, a generative model does not produce one correct answer. At every word it holds a ranked set of plausible continuations with associated probabilities, and it selects from that set. Asked to complete "the capital of France is," the word Paris carries an overwhelming probability, but the alternatives are not at zero. This is precisely the property that allows the same tool to be useful for both factual summary and creative drafting. Most tools expose settings that govern this behavior. Temperature determines how far the model may stray from its highest-probability choice: a low setting keeps output conservative and predictable, a high setting permits more surprising and varied language. Two related settings, commonly labeled top-k (selecting the number of top-ranking responses to consider) and top-p (selecting only the group of potential responses whose combined probabilities reach a set threshold for consideration), determine how wide a field of candidates the model considers before choosing at all. Taken together, these settings define where the model may explore, while temperature governs how boldly it explores within that space . The same mechanism explains the failure mode that Chapter 4 treats at length. A hallucination — a confident statement that happens to be false — is the model selecting a statistically plausible continuation that is not factually correct. No separate malfunction is required to produce one. The practical implication is that a single answer should be read as one sample rather than as a verdict: asking again, or rephrasing, is a legitimate and often necessary move.

Beyond the Single Prompt: Standing Instructions and Supplied Documents

Everything so far concerns one request typed into an empty box. Two further techniques extend the same logic and remove most of the repetition from everyday use. The first is the standing instruction, often called a system prompt: instructions and context written once, stored in a designated place, and silently attached to every subsequent conversation. It might record who you are, how you prefer output formatted, and what the assistant is for. Nothing unusual is happening here; the model is still predicting from what is in front of it, and a standing instruction simply guarantees that certain material is always in front of it. A standing instruction should hold only what is stable — per-task detail belongs in the individual prompt. The second is retrieval, sometimes called retrieval-augmented generation (RAG). You supply a set of documents that matter to you; when you ask a question, the system searches those documents and quietly attaches the most relevant passages to your prompt before the model answers. This is not a different kind of machine. It is the same prediction system with an automated librarian in front of it. Two benefits follow: your own documents can be more current than the model's training data, and grounding answers in real material reduces, though it does not eliminate, invented content. The corresponding limitation is that an assistant built this way is a specialist whose competence extends exactly as far as the documents provided and no further. Both techniques rest on a capacity worth naming, because it also explains a common frustration. When a tool appears to remember earlier parts of a long exchange, what actually happens is that the entire conversation is re-sent to the model with each new message, up to a limit called the context window. Nothing is being stored; the material is being reread each time. Early tools had small context windows and would effectively lose the beginning of a conversation before reaching its end. Modern tools have much larger ones, which is what makes supplying whole documents practical.

Conclusion

Prompting is less a technical skill than clear thinking made explicit. The components of a good prompt — a defined task, specific instructions, and adequate context — are the same components of a good brief given to a colleague, which is why practice with a tool that never tires quietly improves how you delegate to people. Two ideas from this chapter carry forward. The first is that variation in output is structural rather than accidental, so one weak answer is a sample and not a judgment on the tool or on you. The second is that context is the lever with the largest effect and the one most often left unused. Chapter 3 takes up what happens when a single well-formed request is still not enough: how to direct a model across a sustained exchange, and how asking it to work through a problem in visible steps changes what it can reliably do.

Additional resources on prompt engineering

• Prompting 101 (YouTube 24m 52s) by Anthropic
• Prompt Engineering Guide by Learn Prompting
• Prompt Engineering Guide by DAIR.AI
• Columbia University: How to talk to AIs - Advanced Prompt Engineering

Want to learn more? Watch Four Methods of Prompt Engineering (YouTube, 12m 41s) by IBM Technology.