|
1. Context engineering is a retrieval problem, not a writing problem
Prompting is what you write. Context engineering is what you choose to put in front of the model, and where in the window it lands.
That second part isn't a style preference. The “lost in the middle” research out of Stanford, UC Berkeley and Samaya AI found a U-shaped accuracy curve across long contexts: models do best when the decisive information sits near the beginning or the end, and measurably worse when it sits in the middle. Claude was among the models tested, and the effect held even for models built explicitly for long contexts.
The practical consequence is the opposite of what a large window invites you to do. Every irrelevant document you add is one more candidate for the model to anchor on. So rank before you fill, keep the load-bearing material at the edges rather than buried mid-prompt, and vary position when you test. The same content in a different slot is, as far as the model is concerned, a different input.
2. The moment you add tools, you own a distributed system
A prompt is a function call. A prompt with tools is a network of them, and every distributed-systems problem arrives along with it.
Timeouts need a defined behavior rather than a default: does the loop wait, retry, or continue with a hole in what it knows? Retries are safe on reads and dangerous on writes, so anything that mutates state needs an idempotency key — an agent that retries a payment is not a hypothetical. And partial failure is the case most teams skip: three of five tools answer, and the model still has to do something defensible about the other two.
The worst outcome isn't any of those, though. It's a tool that quietly returns stale or empty data, because the model will narrate it confidently and nothing downstream looks wrong. Make failure legible inside the tool response itself (an explicit error the model can reason about beats an empty array every time) and put a ceiling on the loop, in steps and in wall-clock, before it finds one for you.
3. Write the eval set before you build the demo
A demo selects for the happy path. You pick the inputs, so of course it works. The problem shows up one change later, when you can't tell whether a new prompt, a new model version, or a different retrieval strategy made the system better or worse.
A fixed set of real inputs with known-good outputs is what makes that question answerable. Not a public benchmark — a few dozen cases drawn from your own traffic, deliberately including the awkward ones: empty input, ambiguous request, something out of scope, something adversarial. The awkward cases are where the behavior you actually ship gets decided.
Once it exists, everything downstream gets cheaper. Model upgrades become a measurement instead of an argument. Prompt changes stop being a matter of taste. And you get to decline a change that demos beautifully and scores worse.
| 
