Skip to content

Repository files navigation

Context Optimizer

npm license node python status platforms

Keep your coding agent's context small. When a session gets compacted, Context Optimizer reranks the relevant parts, drops duplicates, and compresses the rest with a local ML pipeline (LLMLingua-2 + Sentence Transformers) — so more of the window stays useful and fewer tokens get billed.

Highlights

  • Local — all ML runs on your machine.
  • Fails open — if Python or a model is missing, context is passed through untouched. It can't break a session.
  • Two platforms, one config — works with both OpenCode and Claude Code, sharing a single config and stats store.
  • Tunable or zero-config — every model and threshold is configurable per-model or globally, but the defaults just work.
  • GPU or CPU — uses CUDA when available, and falls back to lighter CPU models automatically.
  • Fully automatic — runs on every compaction, and optionally on every chat turn (OpenCode only). No (extra) need to execute /compact yourself.

Dedupe Rerank Compress

Both platforms compress the session on compaction; OpenCode additionally optimizes each chat turn live (Claude Code has no hook for that — see How it works per platform).

This plugin can help when you have long running sessions. If you often start short lived session then this plugin will not have much impact.

On the context that actually gets compacted, LLMLingua-2 at the default 0.6 rate keeps about 60% of the tokens it sees, on top of what rerank + dedup pruning already dropped. Whole-session savings depend on how much of the session is compactable — and the exact % saved is measured on every compaction and you can retrieve stats, so you never have to trust a headline number.

When the conversation being compacted is small, the bookkeeping around compaction costs more than the compaction saves. Only manually run a /compact after you have send multiple messages and want a fresh context.

Tip

If you want more information about how to manage your coding agent's context, you can check out Coding Agent Orchestration It will describe that you could add to your context so that your agent will have the right awnser faster using plugins like Graphify and Mempalace, Stop adding irrelevant information to the context and keep your context clean with context-mode, Dedupe, Rerank and Compress the context before sending it to the LLM with context-optimizer (this plugin) and ask your agent to resond with a compact response with Caveman

Reviews:

Quality can even improve in practice. If the plugin prevents Claude from reading lots of irrelevant files, the compacted context may actually be more focused on the task. But that depends on the nature of your work and whether the filtered information was truly irrelevant.

So my expectation would be: Smaller compacted context: likely yes, often. Same or better effective quality: often, if the plugin successfully removes noise.

For long Claude Code sessions I would actually expect this pipeline to outperform native /compact fairly often.

/compact will almost certainly operate on a substantially smaller context. In many coding sessions the resulting compact may actually be higher quality because noise is removed first. However, the process is inherently lossy and cannot guarantee equal quality in every case.

Requirements

Requirement Minimum How to check
Node.js 18+ node --version
Python 3.9+ python --version
Disk space ~3–5 GB free (models) —

Note

A CUDA GPU is strongly recommended. The optimizer runs CPU-only too, but compression is noticeably slower. On Windows, CPU-only machines, install the CPU PyTorch wheel before running the installer so it doesn't pull a large CUDA build: python -m pip install --index-url https://download.pytorch.org/whl/cpu torch

Installation

# auto-detect: installs into every environment it finds
npx @evermeer/context-optimizer install

# or force a specific target
npx @evermeer/context-optimizer install --opencode
npx @evermeer/context-optimizer install --claude

The installer:

  1. checks Node.js (18+) and Python (3.9+),
  2. installs the Python packages sentence-transformers and llmlingua (PyTorch comes with them),
  3. copies the Python bridge to ~/.context-optimizer/python/,
  4. downloads the models once (~3–5 GB, one-time — see Models),
  5. installs the OpenCode plugin and/or registers the Claude Code hooks.

Detection: OpenCode is recognized by ~/.config/opencode/ or an opencode binary on PATH; Claude Code by ~/.claude/ or a claude binary on PATH. Without --opencode/--claude flags, every detected environment is installed.

Useful flags:

  • --skip-deps — skip the pip install (already installed)
  • --skip-models — skip the model warm-up download

Check what would be detected: npx @evermeer/context-optimizer detect

How it works per platform

OpenCode

The installer copies a self-contained plugin to ~/.config/opencode/plugins/context-optimizer.js. OpenCode loads it automatically. On compaction, the plugin replaces the conversation that OpenCode sends to its summarizer LLM with the ## Optimized Context from the Python bridge. OpenCode's experimental.session.compacting hook doesn't receive the messages, so the plugin only flags the session there. The rewrite happens in the experimental.chat.messages.transform call that OpenCode makes right after, on the exact messages it serializes for the summarizer. The summarizer then works on the optimized context instead of the full conversation, which saves tokens on the compaction call itself. It fails open: if the optimizer fails or the context is below min_chars, the summarizer gets the original conversation. Unlike Claude Code's manual /compact, an LLM summary is still made, because OpenCode has no hook to skip it. In addition, experimental.chat.messages.transform runs live optimization on every chat turn (no Python round-trip, pure TS):

  • deduplication — tool calls with an identical tool name + parameters keep only the newest output; older duplicates are replaced with a short marker.
  • purge errors — string inputs of errored tool calls older than 4 user turns are replaced with a marker (the error output is kept). Mutating/planning tools (write, edit, task, todowrite, …) are protected and never optimized away.

Slash commands:

Command What it does
/context-optimizer Show help
/context-optimizer context Show the current session context breakdown
/context-optimizer stats Show cumulative pruning/compaction stats
/context-optimizer compact Run the optimizer on the current session and show the result, without changing the session. Tip: run /compact to actually compact; it runs natively and doesn't need an LLM to interpret the command.
/context-optimizer config [get|set|reset] Show or update safe settings (timeout_ms, min_chars, model_limits)

Claude Code

Claude Code hooks cannot rewrite the compaction context directly, so the adapter uses a two-phase hand-off, registered in ~/.claude/settings.json:

  1. PreCompact hook — before compaction runs, the transcript context is optimized via the Python bridge and the result is stored under ~/.context-optimizer/claude-sessions/.
  2. SessionStart hook (matcher compact|clear) — the stored optimized context is injected into the fresh session as additional context and the hand-off file is removed.
  3. PostCompact hook — only active in debug mode: captures Claude's native summary for comparison.

What happens next depends on how compaction was triggered:

  • Manual /compact — plugin-only compact. The hook blocks Claude's own compaction and tells you to run /clear. After /clear, the new session starts with only the optimized context, and Claude's LLM summary is skipped entirely. You have to type /clear yourself (hooks cannot run slash commands). A saved hand-off expires after 1 hour, so it never leaks into an unrelated /clear later.
  • Auto-compact. Claude's own compaction runs as usual on the full transcript; right after, the optimized context is added next to Claude's summary. The savings report (initial size, final size, % saved) appears as a message once the compacted session starts. It isn't added to Claude's context. It is not blocked, because blocking an auto-compact that fires at the context limit would fail the current request.

Both hooks fail open: if the optimizer fails or the context is below min_chars, nothing is blocked and Claude Code compacts as usual.

Claude Code has no hook that can rewrite the live conversation, so the per-turn optimization strategies from the OpenCode plugin run here at the PreCompact rewrite point instead, while parsing the transcript:

  • deduplication — identical tool calls (same tool + parameters) keep only the newest result.
  • purge errors — errored tool results older than 4 user turns are dropped.

Surviving tool outputs (capped per result) are fed to the optimizer alongside the prose, instead of being discarded wholesale.

Native /compact vs. /compact with context-optimizer installed

Claude Code's built-in /compact is a single LLM call: it summarizes the whole transcript into shorter prose, with no visibility into what got cut or how much was saved. Claude Code hooks cannot change what that call receives, so context-optimizer replaces it for manual compaction instead: /compact runs rerank, dedupe, and LLMLingua-2 compression (default compression_rate of 0.6) locally, blocks the LLM summary, and /clear continues with that result. No LLM call is spent on compaction. The exact % saved is recorded on every compaction and available on demand via /context-optimizer:stats instead of being a guess. Auto-compact still uses Claude's own summary, with the optimized context added alongside it. Quality-wise it's lossy compression, not paraphrasing — LLMLingua drops low-information tokens and favors keeping distinctive facts, names, and numbers over prose connectors; if a compaction ever trims something you needed, raise compression_rate or max_chunks (see Configuration).

Slash commands (installed as markdown commands in ~/.claude/commands/context-optimizer*):

Command What it does
/context-optimizer Show help
/context-optimizer:context Show the current session's context/token breakdown (estimated from the visible conversation)
/context-optimizer:stats Show cumulative pruning/compaction stats (context-optimizer stats)
/context-optimizer:compact Run one compaction pass on the current conversation (context-optimizer optimize)
/context-optimizer:config [get|set|reset] Show or update safe settings (context-optimizer config)
/context-optimizer:debugon Turn debug mode on (context-optimizer debug on)
/context-optimizer:debugoff Turn debug mode off, the default (context-optimizer debug off)
/context-optimizer:evaluate [session-id] Compare the newest debug compaction's plugin and native results and save an evaluation (run it in a new session)

These shell out to the same context-optimizer CLI (npx @evermeer/context-optimizer <cmd>) that backs the OpenCode commands, so config and stats are shared across both platforms.

Debug mode: plugin vs. native compact

Debug mode lets you check whether the plugin's context is actually better than Claude's own summary on your sessions. It is off by default. Turn it on with /context-optimizer:debugon and off with /context-optimizer:debugoff. You can also set the CONTEXT_OPTIMIZER_DEBUG=1 env var, which overrides the stored setting.

While debug mode is on, every compaction (manual /compact and auto-compact) runs both:

  1. PreCompact runs the optimizer as usual and saves its result, but blocks nothing and hands nothing off. Claude's native compaction is not canceled, and the session continues on the native summary alone, as if the plugin weren't installed. Debug runs are not counted in /context-optimizer:stats.
  2. PostCompact captures Claude's native summary, saves it next to the plugin result, and shows a report:
[context-optimizer debug] manual compaction: native and plugin results saved.
Native compact: 12,345 chars
Plugin result:  6,000 chars, 49% of the native size (51% smaller)
Original context: 80,000 chars
Files: ~/.context-optimizer/debug/<session-id>/2026-09-24T10-15-30-123Z
Compare them in a new session with /context-optimizer:evaluate <session-id>

Each compaction gets its own folder, so the three files that belong together sit side by side. A per-session index.md lists every compaction with sizes and links:

~/.context-optimizer/debug/<session-id>/
├── index.md                         one row per compaction: sizes, plugin-vs-native %, links
└── <timestamp>/
    ├── plugin.txt                   context-optimizer's result
    ├── native.txt                   Claude's own compact summary
    ├── evaluation.md                written by /context-optimizer:evaluate
    └── meta.json                    sizes and plugin status

To evaluate, start a new Claude Code session (so nothing else is in its context) and run /context-optimizer:evaluate, or /context-optimizer:evaluate <session-id> for a specific session. It picks the newest debug compaction and compares both files on task continuity, key facts, fidelity, noise and size. It writes a verdict with its reasoning to evaluation.md in the same folder. context-optimizer debug latest [session-id] prints that compaction's file paths and sizes as JSON.

Debug mode needs the PostCompact hook. Installs before this version only registered PreCompact and SessionStart, so re-run npx @evermeer/context-optimizer install --claude once.

Configuration

All state lives in ~/.context-optimizer/ (override with the CONTEXT_OPTIMIZER_HOME env var): config.json, stats.json, context-optimizer.log, the Python bridge, and the debug/ logs.

Environment variables win over config.json, which wins over defaults:

Setting Env var Default What it does
timeout_ms CONTEXT_OPTIMIZER_TIMEOUT_MS 300000 How long to wait for the Python bridge before failing open
min_chars CONTEXT_OPTIMIZER_MIN_CHARS 2000 Minimum context size before optimization runs
compression_rate CONTEXT_OPTIMIZER_COMPRESSION_RATE 0.6 Fraction of tokens LLMLingua keeps (0–1); higher keeps more detail
max_chunks CONTEXT_OPTIMIZER_MAX_CHUNKS 12 Max ranked chunks kept before compression (positive integer)
dedupe_threshold CONTEXT_OPTIMIZER_DEDUPE_THRESHOLD 0.9 Cosine similarity (0–1) above which a chunk is treated as a duplicate
total_prune_budget_chars CONTEXT_OPTIMIZER_PRUNE_BUDGET_CHARS 8000 Char budget of ranked+deduped context kept before compression (positive integer)
auto_compression_chars CONTEXT_OPTIMIZER_AUTO_COMPRESSION_CHARS 4000 Context size (chars) at which per-model model_limits overrides kick in
reranker_model CONTEXT_OPTIMIZER_RERANKER_MODEL BAAI/bge-reranker-large HuggingFace cross-encoder used to rank chunks by relevance
embed_model CONTEXT_OPTIMIZER_EMBED_MODEL all-MiniLM-L6-v2 HuggingFace embedding model used for deduplication
compressor_model CONTEXT_OPTIMIZER_COMPRESSOR_MODEL (auto by device) LLMLingua-2 model used for compression; unset lets the bridge pick by device (see below)
model_limits CONTEXT_OPTIMIZER_MODEL_LIMITS {} Per-model overrides, e.g. {"gpt-4.1": {"compression_rate": 0.65, "max_chunks": 8}}
— CONTEXT_OPTIMIZER_DEBUG off 1/true forces debug mode on, 0/false forces it off; unset uses /context-optimizer:debugon/debugoff
— CONTEXT_OPTIMIZER_PYTHON py -3 (Windows) / python3 Python interpreter used for the bridge
— CONTEXT_OPTIMIZER_CLI ~/.context-optimizer/python/context_optimizer_cli.py Path to the Python bridge script

compression_rate, max_chunks, dedupe_threshold, total_prune_budget_chars, reranker_model, embed_model, and compressor_model are global defaults; a matching key in model_limits overrides them per-model, and an explicit per-request option overrides both. If responses lose important detail, raise compression_rate or max_chunks; if prompts are still too large, lower them. (auto_compression_chars is a JS-side gate for model_limits, so it is not a per-model or per-request key.)

Models

The pipeline uses three models, each swappable via the matching config key. On first run the libraries download them from Hugging Face and cache them under ~/.cache/huggingface/; the installer warms this cache once so the first real compaction loads from disk. Set an alternative with context-optimizer config set <key> <model> (or the env var).

Any Hugging Face model that fits the role works — the lists below are tested, drop-in options. Larger models improve quality but cost memory and latency; smaller ones keep the optimizer responsive on CPU.

reranker_model — ranks chunks by relevance (CrossEncoder)

Model Notes
BAAI/bge-reranker-large Default. Best ranking quality; largest and slowest, heavy on CPU.
BAAI/bge-reranker-base Noticeably smaller/faster with a small quality drop — the go-to if the large model is slow or OOMs.
BAAI/bge-reranker-v2-m3 Strong multilingual reranking; larger, best on a GPU.
cross-encoder/ms-marco-MiniLM-L-6-v2 Tiny and very fast, English-only; lowest quality — for constrained CPUs.

embed_model — embeddings for deduplication (SentenceTransformer)

Model Notes
all-MiniLM-L6-v2 Default. Fast, small, solid general-purpose English embeddings.
all-mpnet-base-v2 Higher-quality English embeddings; ~3–4× larger and slower.
BAAI/bge-small-en-v1.5 Small, strong English embedder; good quality-for-size alternative to the default.
paraphrase-multilingual-MiniLM-L12-v2 Multilingual dedup for non-English or mixed-language context.

compressor_model — prompt compression (must be an LLMLingua-2 model)

Leave compressor_model unset to let the bridge auto-select by device. Only LLMLingua-2 checkpoints work here (the bridge runs with use_llmlingua2=True).

Model Notes
microsoft/llmlingua-2-xlm-roberta-large-meetingbank Auto-selected on CUDA. Larger, multilingual, higher-quality compression.
microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank Auto-selected on CPU. Smaller and faster; also a good explicit choice to force the lighter model on a GPU box.

Architecture

context-optimizer/
├── packages/
│   ├── core/          # business logic: payload building, config, stats,
│   │                  # Python bridge, environment detection, installer
│   ├── claude-code/   # Claude Code adapter (PreCompact + SessionStart hooks)
│   └── opencode/      # OpenCode adapter (compaction hook + slash commands)
├── python/            # ML core: rerank + dedupe + compress (SentenceTransformers, LLMLingua)
├── package.json
└── tsconfig.json

Everything is TypeScript compiled with tsup to dist/ (ES2022, ESM), except the ML core itself: reranking, dedup embeddings, and LLMLingua compression run in Python (PyTorch) and are called over a stdin/stdout JSON bridge. The adapters fail open — if Python or its dependencies are missing, your context is left untouched.

Development

npm install
npm run build     # tsup → dist/ (index.js, cli.js, opencode.js, claude-hook.js)
npm test          # build + node --test
py -3 -m unittest discover -s tests -p "test_*.py"   # Python core tests

Troubleshooting

Symptom Likely cause Fix
ModuleNotFoundError: sentence_transformers / llmlingua Packages installed into a different Python Re-run python -m pip install sentence-transformers llmlingua with the interpreter from CONTEXT_OPTIMIZER_PYTHON
First run hangs Models downloading from Hugging Face One-time; re-run the installer or raise CONTEXT_OPTIMIZER_TIMEOUT_MS
Nothing happens on compaction Context below min_chars, or adapter not loaded Check ~/.context-optimizer/context-optimizer.log for the skip reason
Warning instead of optimized context Bridge failed (fail-open) Read the warning in the log, fix the Python issue, retry
Out-of-memory / very slow CPU Reranker model is large Run context-optimizer config set reranker_model BAAI/bge-reranker-base (or set CONTEXT_OPTIMIZER_RERANKER_MODEL)
/context-optimizer stats shows zeros No successful optimization yet Stats accumulate in ~/.context-optimizer/stats.json after the first successful run

About

Coding Agent Context Optimizer Plugin

Resources

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages