The Pegboard Model for AI Prompting ​

When explaining AI agents to developers new to the space, I find myself reaching for physical analogies. The one that's stuck with me is the pegboard—sometimes called a Galton board or bean machine. It's a surprisingly useful mental model for understanding prompt engineering and context management.

The Basic Setup ​

Imagine a vertical pegboard. You drop a ball at the top, and it bounces off pegs as it falls, eventually landing in one of several buckets at the bottom. Each time the ball hits a peg, it bounces either left or right. The final position depends on the cumulative effect of all those small random bounces.

This mirrors how language models generate text. At each step, the model is making a choice—picking the next token from a probability distribution. Like the ball bouncing off pegs, each choice is influenced by what came before, and the final output is the result of many small decisions compounding.

Three Levels of Influence ​

What makes this analogy useful is how it maps to the different ways we can shape AI outputs.

1. The Pegs: Model Architecture and Training ​

The pegs themselves represent the model's fundamental structure—the neural network architecture, the training data, the reinforcement learning from human feedback. These determine the shape of the possibility space. A model trained primarily on code will bounce differently than one trained on poetry.

You don't control the pegs. They're baked in when you choose which model to use.

2. The Walls: Safety Training and System Prompts ​

In the demo above, you can toggle "guardrails" that block certain regions of the board. These represent the safety training and system-level constraints that model providers build in.

These walls prevent the ball from landing in certain buckets entirely. No matter how you drop the ball or how you bias it, some outputs are simply unreachable. This is why you can't easily prompt a well-aligned model to produce certain categories of harmful content—those buckets are walled off at a level you don't control.

3. The Ball's Color: Your Context and Prompts ​

Here's where it gets interesting for developers. When you craft a prompt or build up context, you're essentially "coloring" the ball. A blue ball is more likely to land in blue territory. A red ball trends red.

In the demo, try adjusting the bias slider and dropping several balls. You'll notice the distribution shifts—not perfectly, not deterministically, but probabilistically. This is exactly how prompts work. A well-crafted prompt doesn't guarantee a specific output. It shifts the probability distribution toward the outputs you want.

What This Model Gets Right ​

Probabilistic, not deterministic. The same prompt can yield different outputs on different runs. The ball might still land on the "wrong" side occasionally. This sets appropriate expectations—prompt engineering is about shifting odds, not writing programs.

Layered influence. The three levels (pegs, walls, ball color) map cleanly to model training, safety constraints, and user prompts. Each layer builds on the one below it.

Compound effects. Small biases add up. If each peg bounce is slightly more likely to go right, the final position shifts significantly. Similarly, consistent signals throughout your context compound into meaningful shifts in output probability.

Some areas are simply unreachable. The walls represent hard constraints that no amount of prompting can overcome—at least not through normal use of the API.

Where the Analogy Breaks Down ​

Every model has limits, including mental ones.

Independence of bounces. In a real pegboard, each bounce is independent. In language models, each token generation is heavily influenced by all previous tokens. It's more like the pegboard reshaping itself after each bounce based on where the ball went.

Dimensionality. A pegboard is 2D with binary choices. The actual probability space of a language model is vastly higher-dimensional—tens of thousands of possible next tokens at each step, not just left or right.

The "attraction" mechanism. A blue ball being attracted to blue pegs isn't physics—I'm invoking something more like magnetism as a metaphor. Worth being explicit that this is metaphorical, not a physical simulation.

Practical Implications ​

If you buy this mental model, a few practical insights follow:

Context is cumulative. Each piece of context you add is another small bias. System prompts, conversation history, retrieved documents, examples—they all compound. This is why RAG (retrieval-augmented generation) and few-shot prompting work: you're adding many consistent biases pointing in the same direction.

Consistency beats intensity. Ten small nudges in the same direction beat one strong shove. Repeating a key instruction in different forms throughout your prompt is often more effective than stating it once emphatically.

You're working with probabilities. Sometimes the ball will still land where you don't expect. Good prompt engineering includes handling the cases where the model goes off-script—validation, retry logic, guardrails on your end.

Know what's walled off. Understanding what the model won't do (due to training or system prompts) saves you from wasting time trying to prompt-engineer your way past hard constraints.

The Refinement ​

One addition to the basic analogy: where the ball bounces on the first row changes the shape of the pegboard below it. This captures the autoregressive nature of language models—early tokens constrain the possibilities for later tokens. The first sentence of a response often commits the model to a particular direction, which is why the beginning of your prompt (and the beginning of the model's response) carries disproportionate weight.

This is why techniques like "chain of thought" prompting work. By having the model generate intermediate reasoning steps, you're letting it bounce through several rows in a controlled way before reaching the final answer. Each intermediate step reshapes the remaining pegboard.

Conclusion ​

The pegboard model isn't perfect, but it's useful. It builds intuition about the probabilistic nature of AI outputs, the layered nature of influence, and the importance of cumulative context. Most importantly, it sets appropriate expectations: you're shifting probability distributions, not writing deterministic programs.

Next time you're debugging a prompt that "usually works but sometimes doesn't," picture the ball bouncing through the pegs. You've biased it toward the right bucket, but there's always that one bounce that sends it the other way.