Original post · Original
New NVIDIA paper finds that AI agents barely lose accuracy when they keep contexts short by compacting their history often.
Each new token costs more to generate as the context grows. Serving open models at a 256K context window instead of 16K cost 3.6x to 7x more per generated token.
They had agents stop at a set limit, write a short handoff note, and restart with only the task and that note. For long documents, the agent searched files with Bash tools instead of reading everything at once.
On 6 long-horizon benchmarks, at most 5 of 22 compaction settings per model lost more than 2 points versus full history. Compacting after every action kept Claude Opus 4.5 at 70.2% on SWE-bench Verified, versus 70.0% with full history.
Before paying for long context, give agents a structured handoff note and file search tools, then check if short contexts match your accuracy.