Context Engineering for AI Agents: Why More Context Is Not Better

Context engineering is the design of the information an AI agent sees before it makes a decision. It is larger than writing a clever prompt. It covers instructions, selected files, conversation history, tool output, memory, and the way completed work is compressed. Good context makes a capable model precise. Bad context can make it worse even when every necessary fact is technically present.

More context is not automatically better

The naive strategy is to give the agent everything: the repository, the entire conversation, and all available documentation. The problem is that context is constrained by more than a token limit. The model's effective attention is constrained too.

When a critical rule sits between thousands of log lines and three obsolete specifications, the agent may fail to use it. Anthropic describes this degradation as context rot. Selection and structure therefore matter more than simply buying a larger window.

The layers of an agent's context

A practical system usually combines:

• governing instructions and permission boundaries,
• the current objective and completion criteria,
• the smallest useful slice of code or documentation,
• recent tool results,
• durable decisions retrieved from agent memory,
• a compact summary of earlier work.

These layers have different lifetimes. A security boundary may be stable for months. A live API result may be reliable for minutes. Treating both as timeless text is a design error.

Start with the decision, then fetch material

Ask what decision the agent must make next. Implementing a function, reviewing security, and publishing an article require different evidence.

A simple process works well:

1. define the objective and observable result,
2. state the constraints the agent cannot violate,
3. retrieve a few relevant files or notes,
4. after each tool call, keep only the output needed for the next step,
5. compress completed phases into decisions and evidence.

This reduces cost, but the larger benefit is auditability. You can explain what the agent knew when it acted.

Compression without losing the state

Long-running work cannot preserve every message forever. It needs summaries. A weak summary retells the conversation. A useful one captures operational state:

• what was completed,
• what was verified,
• which decisions were made,
• what must remain unchanged,
• the next action,
• unresolved questions.

Separate facts from conclusions. “The test passed on commit SHA…” is evidence. “The change is safe” is a judgment that may require more evidence.

Tools produce context too

Agents often drown in command output rather than documentation. A full build log or an API response with hundreds of fields can consume most of the useful context. Tools should return decision-ready output: the relevant error, resource identifier, status, and a pointer to the full trace.

That connects context engineering to tool and API design for agents. A perfect input schema cannot compensate for an enormous, unstructured response.

How to evaluate context quality

Do not stop at “the answer looks good.” Vary the context and observe behavior:

• remove one critical constraint and check whether the evaluation catches it,
• add a plausible but obsolete document,
• increase irrelevant tool output,
• test whether the agent asks for a missing fact instead of inventing it,
• compare cost and latency at the same result quality.

If the workflow works only with a perfectly hand-arranged prompt, it is not robust. Context engineering should produce a repeatable information pipeline, not a one-off prompt trick.

A practical filter

Every active piece of context should answer one of three questions: what must the agent achieve, what boundary must it respect, or what evidence does it need for the next decision? If it answers none of them, it probably should not be active right now.

The best context is not the largest. It is current, structured, and sufficient.

Sources:

• Effective context engineering for AI agents
• Building effective agents