Moving the Center of Mass

Every LLM engineering technique is, at bottom, doing the same thing: conditioning a probability distribution, moving its center of mass toward human intent while compressing tail risk.

The abstraction starts by giving up the instinct to “approximate the correct answer.” That instinct presupposes a ground truth — which holds in classification tasks and almost never holds in language generation. The question to ask is “is the output appropriate for the current context.” Accept that premise, and the optimization target changes from approaching a fixed endpoint to continuously conditioning a dynamic distribution.

The best way to understand this is to follow a request from the moment it enters the model to the moment the output emerges, and watch where the engineering intervenes.

A token sequence enters the model; at each step the model samples the next token from a conditional probability distribution until the output is complete. That distribution is not fixed — it is determined jointly by the model’s parameters and the current context. Whatever the context is, that is what the distribution looks like. This mechanism forces every intervention to answer the same question: before, during, or after sampling, how do you bring the distribution closer to the shape you want?

Before inference starts, three kinds of intervention are available. The most direct changes the information density of the context: RAG injects relevant documents, memory systems carry in historical state, MCP data-fetching feeds the live state of external systems into the context. The closer the context is to the true state of the world, the more accurate the conditional probabilities at sampling time — the center of mass moves on its own. The second kind injects behavioral priors: rather than telling the model the answer, it presses an expert’s procedural knowledge into the context, constraining the paths the model tends to take rather than the destination itself. Fine-tuning is the radical version of the same logic. It burns the behavioral pattern straight into the weights. The third kind is blunter: JSON schemas or structured output truncate the output subspace directly, hard-excluding invalid token paths at the sampling stage. The first two change what the model tends to do; the third changes the set of options it can draw from.

During inference, tool calls introduce an intervention of a different nature. The model is no longer just sampling. It calls external tools, and the return values re-enter the context, conditioning every subsequent step of inference. This does more than supplement information: it moves uncertainty out of the model and into external systems. One root of hallucination is that the model is forced to fabricate facts it is unsure of from parametric memory; tool calls outsource that uncertainty.

After inference there is one last kind of intervention: leave the distribution alone and do rejection sampling on the output side. Run, evaluate, retry. The premise of this harness logic is that rather than tuning the distribution to perfection, you sample a few more times and filter with deterministic evaluation logic. The reflection loop is a variant: let the model see its own output and revise, turning a single sample into iterative conditioning.

Spread along this timeline are three engineering philosophies: give the model enough to condition on (before), keep it grounded (during), and if it fails, run it again (after). Each has its cost. A system that relies only on up-front intervention is brittle in the face of edge cases, and a system that relies only on post-hoc filtering loses control of its compute cost. Mature engineering practice is usually a combination of all three, and knowing where each intervention sits on the timeline is the precondition for combining them sensibly.

Now look back at the dizzying stream of new techniques — new agent frameworks, new context-management schemes, new evaluation tools — and nearly all of them find a place in this coordinate system. To judge whether a new one is worth following, work out where on the timeline it intervenes and which part of the distribution it changes.