LLMSlim + Hermes Agent
A Hermes Agent context engine that compresses older turns locally instead of asking a model for a summary, so prompt caches keep hitting.
LLMSlimContextEngine is a Hermes Agent ContextEngine that a plugin puts in place of the built-in context compressor. When the prompt nears the context window it compresses older tool results and turns locally instead of asking a model for a summary. There is no LLM call, the output is deterministic, and the start of the prompt is sent byte for byte the same, so the provider's prompt cache keeps hitting.
1. Installation and setup
There is no llmslim[hermes] extra: current Hermes releases are installed with Hermes's installer, Docker image or Nix (the hermes-agent package on PyPI is an old 0.19 release), and the engine needs nothing beyond llmslim. With llmslim in Hermes's environment (with uv as shown, or Nix extraPythonPackages), Hermes discovers llmslim's hermes_agent.plugins entry point. A managed Hermes install may rebuild its environment on hermes update and drop packages installed by hand.
# No extra: install llmslim into the Python environment Hermes runs in.
# Needs llmslim 0.9.0 or newer.
uv pip install --python <hermes venv>/bin/python llmslimOr use a directory plugin: create ~/.hermes/plugins/llmslim/ with two files, plugin.yaml and __init__.py. On Hermes 0.21 and newer, its package manager installs the declared python_dependencies when you enable the plugin and keeps them across updates; on older releases, install llmslim into Hermes's environment yourself.
name: llmslim
version: 1.0.0
description: Local context compression with LLMSlim (no LLM call)
python_dependencies:
- "llmslim>=0.9.0,<1"__init__.py, in the same directory:
from llmslim.integrations.hermes import register # noqa: F401Then enable the plugin and select the engine in ~/.hermes/config.yaml. hermes plugins enable llmslim does the first part.
plugins:
enabled: [llmslim]
context:
engine: llmslimOptional settings go under the plugin's entry in config.yaml:
plugins:
entries:
llmslim:
settings:
target_percent: 0.5 # compress to about half the context window
protect_last_n: 62. Architecture & execution flow
3. Runnable example and its output
import json
from llmslim import count_tokens
from llmslim.integrations.hermes import LLMSlimContextEngine
def test_output(suite):
lines = [f"{suite}/test_api.py::test_case_{i} PASSED" for i in range(300)]
lines.insert(200, f"{suite}/test_refund.py::test_rounding FAILED - 10.05 != 10.04")
return "\n".join(lines)
def tool_call(call_id, command):
arguments = json.dumps({"command": command})
return {
"role": "assistant",
"content": None,
"tool_calls": [
{"id": call_id, "type": "function", "function": {"name": "terminal", "arguments": arguments}}
],
}
messages = [
{"role": "system", "content": "You are Hermes, a coding agent."},
{"role": "user", "content": "Fix the failing refund test."},
tool_call("c1", "pytest -q tests"),
{"role": "tool", "tool_call_id": "c1", "content": test_output("tests")},
{"role": "assistant", "content": "test_rounding fails. Checking the API suite as well."},
tool_call("c2", "pytest -q api"),
{"role": "tool", "tool_call_id": "c2", "content": test_output("api")},
{"role": "assistant", "content": "Same failure there. Switching refund.py to Decimal."},
{"role": "user", "content": "Go ahead."},
]
# What Hermes does with a plugin engine: build it, tell it the model's window, then compress.
engine = LLMSlimContextEngine(protect_first_n=1, protect_last_n=2)
engine.update_model(model="any-model", context_length=8_000) # threshold 6,000, target 4,000
print("compress?", engine.should_compress(sum(count_tokens(m["content"] or "") for m in messages)))
compacted = engine.compress(messages, current_tokens=7_500) # Hermes's estimate incl. tool schemas
for before, after in zip(messages, compacted):
same = "same object" if before is after else "rewritten"
tokens = count_tokens(after["content"] or "")
print(f"{before['role']:9} {count_tokens(before['content'] or ''):5} -> {tokens:4} tokens {same}")
print(engine.last_fit.summary())
print()
print(compacted[6]["content"])Output (Python 3.14, Hermes Agent v0.21.7 from GitHub, token counts from tiktoken):
compress? True
system 8 -> 8 tokens same object
user 6 -> 6 tokens same object
assistant 0 -> 0 tokens same object
tool 3620 -> 80 tokens rewritten
assistant 12 -> 12 tokens same object
assistant 0 -> 0 tokens same object
tool 3620 -> 80 tokens rewritten
assistant 11 -> 11 tokens same object
user 3 -> 3 tokens same object
7,280 -> 200 tokens; budget 3,780; 2 compressed, 0 dropped
api/test_api.py::test_case_0 PASSED
... [198 similar lines]
api/test_api.py::test_case_199 PASSED
api/test_refund.py::test_rounding FAILED - 10.05 != 10.04
api/test_api.py::test_case_200 PASSED
... [98 similar lines]
api/test_api.py::test_case_299 PASSEDOnly the two tool results changed; every other message is the same object, and tool_calls and tool_call_id are untouched. The budget is 50% of the window minus the 220 tokens of the request that are not in the messages (Hermes's estimate of 7,500 minus the 7,280 counted here). The test output is a favourable case, because runs of near-identical lines collapse into a count; prose is compressed only as far as the budget requires.
4. Options
Keyword arguments, shown as name = default.