Short-Term Memory: Working State Without Context Bloat

The most common short-term memory system for an agent is also the laziest: keep the chat history and hope the model can find the plot.

It works beautifully for a while. The user asks, the agent answers, tools run, files change, approvals happen, and every turn gets stacked neatly behind the next one. Nothing is lost. Everything is technically available.

Then the task grows legs.

The transcript now contains three abandoned plans, a 900-line test failure, a tool result from before the refactor, a correction that mattered two hours ago, and the phrase “let’s keep this simple” buried under enough context to make a senior engineer stare into the middle distance.

The agent still has memory. It just has the kind of memory you get from keeping every browser tab open since Tuesday.

Short-term memory is the part of agent memory that carries active work while the current task, thread, or session is alive. In the load / carry / recall model from How Agent Memory Actually Works, this is the carry layer. The previous piece, Current Memory: The Agent’s Loaded Brain, covered what should be loaded before work begins. This one is about what should survive while work is underway.

Short-term memory should preserve working state, not worship the transcript.

The Workbench, Not The Warehouse

Think of short-term memory like a workbench.

When you are fixing a bug, the useful surface is not every note you have ever taken. It is the current failing test, the files you touched, the hypothesis you are testing, the constraint you must not break, and the next command you plan to run.

The same is true for agents. The model usually does not need every false start. It does need enough state to take the next coherent step:

• the current goal,
• active constraints,
• decisions already made,
• evidence behind those decisions,
• files, records, or systems touched,
• tool results that still matter,
• pending approvals or interruptions,
• unresolved questions,
• the next action.

Conversation history can contain all of that, but it also contains a lot more. Short-term memory design is the discipline of turning raw transcript into useful working state.

That distinction sounds small until you build a long-running workflow. A support agent needs the user’s current issue, plan tier, attempted fixes, and escalation state. It does not need every greeting. A coding agent needs the task objective, modified files, failed tests, and design decisions. It does not need the entire first attempt after that attempt has been replaced.

The transcript is evidence. The workbench is what the agent should actually use.

What Frameworks Are Really Giving You

Modern agent frameworks make short-term memory concrete.

OpenAI’s Agents SDK docs describe sessions as storage for conversation history tied to a specific session. The Python reference exposes lifecycle operations such as retrieving items, adding items, popping the most recent item, and clearing a session. The JavaScript sessions guide goes further into custom storage, resumable human-in-the-loop runs, and history compaction.

That is useful, but notice the shape of the feature: it is not “make the agent remember everything forever.” It is “give the current session a managed history with operations.”

LangGraph’s memory docs draw the boundary even more explicitly. Short-term memory is thread-scoped state, persisted with checkpoints so a thread can resume. Long-term memory lives in stores across threads. Its persistence docs describe checkpointers as the short-term mechanism for conversation continuity, human-in-the-loop workflows, time travel, and fault tolerance.

That split is the whole lesson hiding in plain sight:

Short-term memory: what this thread needs to continue.
Long-term memory: what future threads may need to recall.
Current memory: what the agent should load before it starts.

When those boundaries blur, agents get weird. Temporary debugging guesses become durable beliefs. Stable rules disappear after compaction. A thread-specific workaround leaks into future work. The agent is not being mysterious; the memory layers are filing things in the wrong drawer.

The Context Bloat Curve

Short-term memory fails slowly. First it is just convenient history. Then it becomes expensive history. Then it becomes distracting history.

LangGraph’s memory overview warns that long conversation histories can exceed context windows, and that even when they fit, models can perform worse over long contexts because stale or off-topic content competes for attention. Anthropic’s Claude Code agent-loop docs make the same pressure concrete for coding agents: conversation history accumulates prompts, responses, tool inputs, and tool outputs, while large tool outputs can consume significant context in a single turn.

This is why “bigger context window” is not the whole answer. A bigger desk still gets messy if you never clear it.

Short-term memory has four common failure modes:

1. Recency bias: the newest messages are kept even when an older constraint matters more.
2. Tool-output sludge: verbose logs, file reads, and command output crowd out decisions.
3. Summary drift: compaction preserves a conclusion but loses the evidence or caveat that made it safe.
4. Promotion leaks: temporary state gets written into long-term memory before anyone checks whether it is actually true.

The fix is not to delete aggressively and hope for the best. The fix is to decide what kind of memory operation the moment needs.

Trim, Summarize, Checkpoint, Delete

LangGraph’s memory management docs list the practical operations: trim messages, delete messages, summarize older history, manage checkpoints, and use custom filtering. Those are not just framework features. They are engineering moves.

Trim when old content is no longer relevant and recency is a good proxy. This works for ordinary chatty turns, repeated confirmations, and stale tool output. It is risky when the important constraint appeared early.

Summarize when the work still needs continuity but not the full transcript. A good summary preserves goals, constraints, decisions, evidence, file paths, test results, errors, approvals, and next actions. A bad summary says “we debugged the issue” and calmly throws the stack trace into the sea.

Checkpoint when the system may need to resume, inspect, rewind, or recover. Checkpoints are especially useful around approvals, multi-step workflows, and places where a human may ask, “How did we get here?”

Delete when state is wrong, sensitive, expired, or actively misleading. Keeping bad context because it once happened is how an agent becomes confidently haunted.

For coding agents, Anthropic’s compaction guidance adds one important warning: compaction can preserve recent exchanges and key decisions, but early instructions may not survive. Stable project rules belong in loaded current memory, not in the first message of a long session.

That is the difference between memory as continuity and memory as ceremony. The goal is not to retain everything. The goal is to retain what lets the next step be correct.

A Useful Working-State Summary

If you maintain an agent loop, a task file, or compaction instructions, this is the kind of state worth preserving:

## Current Goal
Fix the retry bug without changing the public client API.

## Active Constraints
- Keep Node 20 support.
- Do not alter response payload shapes.
- Publishing requires explicit approval.

## Decisions
- The bug is in retry scheduling, not auth.
- Preserve exponential backoff; add jitter only inside retries.

## Evidence
- `retry.test.ts` fails on the third retry path.
- Logs show duplicate timer scheduling after cancellation.

## Touched Artifacts
- `src/client/retry.ts`
- `tests/retry.test.ts`

## Pending
- Re-run retry tests after cancelling stale timers.
- Check whether cancellation affects streaming requests.

That is not a transcript. It is the live shape of the work.

This also makes human review easier. A human can scan the state and correct it. “No, auth is still in scope.” “That file path is wrong.” “The payload shape can change behind a version gate.” The memory becomes inspectable, not just accumulated.

What Not To Carry

Short-term memory should be temporary by default. That means some things deserve to leave the workbench.

Do not carry solved branches unless their reasoning still matters. Once a hypothesis is disproven, preserve the conclusion and evidence, not every exploratory step.

Do not carry raw verbose output when a structured observation is enough. Keep the failing assertion, error message, command, and relevant lines. Archive the full log if needed.

Do not carry stable rules only in thread history. Move them to current memory, rules files, permissions, hooks, tests, or configuration.

Do not carry durable lessons as transcript fragments. If the agent learned something future work should use, promote it deliberately into long-term memory with scope, provenance, and review.

And do not carry emotional support context for the model. “Be careful” feels comforting to write, but it does not beat a specific constraint, a failing test, or a permission boundary.

Barista’s Tip: Audit The Carry Layer

For any agent workflow that lasts more than a couple of turns, ask:

1. What is the current goal?
2. Which constraints must survive compaction?
3. What decisions have already been made?
4. What evidence justifies those decisions?
5. Which tool outputs still matter?
6. What can be trimmed, summarized, checkpointed, or deleted?
7. What should move to current memory or long-term memory instead?

This audit keeps short-term memory from becoming a junk drawer with timestamps.

It also makes agents more predictable. If the system exposes session state, summaries, checkpoints, and promoted memories, engineers can debug memory failures like ordinary software failures: wrong state, missing state, stale state, leaky state.

That is much better than shrugging and saying the agent “forgot.” Sometimes it forgot. Sometimes we handed it a warehouse and called it a workbench.

The Last Sip

Short-term memory is what the agent carries while the work is alive.

Use sessions, thread state, checkpoints, summaries, and compaction to preserve continuity. But design the carried state intentionally: goal, constraints, decisions, evidence, touched artifacts, pending approvals, unresolved questions, and next action.

The transcript matters. It is the raw record. But the agent should not have to reread the whole journey to know where it is standing.

Keep the workbench clear enough for the next useful move.

Sources On The Counter

• OpenAI Agents SDK Sessions is useful for session-backed history, custom storage, resumable human-in-the-loop runs, and history compaction.
• OpenAI Agents SDK Python Memory shows short-term memory as an explicit session interface with retrieve, add, pop, and clear operations.
• LangGraph’s memory overview draws the boundary between thread-scoped short-term memory and cross-thread long-term memory.
• LangGraph’s memory management guide covers trimming, deleting, summarizing, checkpointing, and filtering short-term histories.
• LangGraph persistence docs explain checkpointers for thread state and stores for durable cross-thread memory.
• Anthropic’s Claude Code agent-loop docs ground the compaction and context-growth tradeoffs for coding-agent workflows.