# LLMSlim + Hermes Agent

> A Hermes Agent context engine that compresses older turns locally instead of asking a model for a summary, so prompt caches keep hitting.

LLMSlimContextEngine is a Hermes Agent ContextEngine that a plugin puts in place of the built-in context compressor. When the prompt nears the context window it compresses older tool results and turns locally instead of asking a model for a summary. There is no LLM call, the output is deterministic, and the start of the prompt is sent byte for byte the same, so the provider's prompt cache keeps hitting.

Category: Framework

## Install

```bash
# No extra: install llmslim into the Python environment Hermes runs in.
# Needs llmslim 0.9.0 or newer.
uv pip install --python <hermes venv>/bin/python llmslim
```

There is no llmslim[hermes] extra: current Hermes releases are installed with Hermes's installer, Docker image or Nix (the hermes-agent package on PyPI is an old 0.19 release), and the engine needs nothing beyond llmslim. With llmslim in Hermes's environment (with uv as shown, or Nix extraPythonPackages), Hermes discovers llmslim's hermes_agent.plugins entry point. A managed Hermes install may rebuild its environment on hermes update and drop packages installed by hand.

Or use a directory plugin: create ~/.hermes/plugins/llmslim/ with two files, plugin.yaml and __init__.py. On Hermes 0.21 and newer, its package manager installs the declared python_dependencies when you enable the plugin and keeps them across updates; on older releases, install llmslim into Hermes's environment yourself.

```yaml
name: llmslim
version: 1.0.0
description: Local context compression with LLMSlim (no LLM call)
python_dependencies:
  - "llmslim>=0.9.0,<1"
```

__init__.py, in the same directory:

```python
from llmslim.integrations.hermes import register  # noqa: F401
```

Then enable the plugin and select the engine in ~/.hermes/config.yaml. hermes plugins enable llmslim does the first part.

```yaml
plugins:
  enabled: [llmslim]
context:
  engine: llmslim
```

Optional settings go under the plugin's entry in config.yaml:

```yaml
plugins:
  entries:
    llmslim:
      settings:
        target_percent: 0.5     # compress to about half the context window
        protect_last_n: 6
```

## How it fits

- Hermes loads the engine from llmslim's hermes_agent.plugins entry point or from the directory plugin, and uses it when config.yaml has context.engine: llmslim.
- Hermes builds the engine and tells it the model's context window. should_compress() is true once the prompt reaches threshold_percent (75%) of the window, using the token counts Hermes reports after each response.
- The system prompt, the first protect_first_n other messages and the user and assistant messages among the last protect_last_n are returned as the same objects. System messages and multimodal content are never rewritten.
- The text of the messages in between (tool results, user and assistant text) is compressed by one shared ratio until the prompt is about target_percent (50%) of the window. If that is not enough, the oldest are replaced by a one-line placeholder, and as a last resort tool results among the last protect_last_n are compressed too.
- No message is removed, so every tool call keeps its result. On the next compaction, messages that are exactly what the previous compaction returned are left alone as long as compressing the newer ones is enough, so the cached prefix is sent unchanged.

## Example

```python
import json

from llmslim import count_tokens
from llmslim.integrations.hermes import LLMSlimContextEngine


def test_output(suite):
    lines = [f"{suite}/test_api.py::test_case_{i} PASSED" for i in range(300)]
    lines.insert(200, f"{suite}/test_refund.py::test_rounding FAILED - 10.05 != 10.04")
    return "\n".join(lines)


def tool_call(call_id, command):
    arguments = json.dumps({"command": command})
    return {
        "role": "assistant",
        "content": None,
        "tool_calls": [
            {"id": call_id, "type": "function", "function": {"name": "terminal", "arguments": arguments}}
        ],
    }


messages = [
    {"role": "system", "content": "You are Hermes, a coding agent."},
    {"role": "user", "content": "Fix the failing refund test."},
    tool_call("c1", "pytest -q tests"),
    {"role": "tool", "tool_call_id": "c1", "content": test_output("tests")},
    {"role": "assistant", "content": "test_rounding fails. Checking the API suite as well."},
    tool_call("c2", "pytest -q api"),
    {"role": "tool", "tool_call_id": "c2", "content": test_output("api")},
    {"role": "assistant", "content": "Same failure there. Switching refund.py to Decimal."},
    {"role": "user", "content": "Go ahead."},
]

# What Hermes does with a plugin engine: build it, tell it the model's window, then compress.
engine = LLMSlimContextEngine(protect_first_n=1, protect_last_n=2)
engine.update_model(model="any-model", context_length=8_000)  # threshold 6,000, target 4,000
print("compress?", engine.should_compress(sum(count_tokens(m["content"] or "") for m in messages)))
compacted = engine.compress(messages, current_tokens=7_500)  # Hermes's estimate incl. tool schemas

for before, after in zip(messages, compacted):
    same = "same object" if before is after else "rewritten"
    tokens = count_tokens(after["content"] or "")
    print(f"{before['role']:9} {count_tokens(before['content'] or ''):5} -> {tokens:4} tokens  {same}")
print(engine.last_fit.summary())
print()
print(compacted[6]["content"])
```

Output (Python 3.14, Hermes Agent v0.21.7 from GitHub, token counts from tiktoken):

```text
compress? True
system        8 ->    8 tokens  same object
user          6 ->    6 tokens  same object
assistant     0 ->    0 tokens  same object
tool       3620 ->   80 tokens  rewritten
assistant    12 ->   12 tokens  same object
assistant     0 ->    0 tokens  same object
tool       3620 ->   80 tokens  rewritten
assistant    11 ->   11 tokens  same object
user          3 ->    3 tokens  same object
7,280 -> 200 tokens; budget 3,780; 2 compressed, 0 dropped

api/test_api.py::test_case_0 PASSED
... [198 similar lines]
api/test_api.py::test_case_199 PASSED
api/test_refund.py::test_rounding FAILED - 10.05 != 10.04
api/test_api.py::test_case_200 PASSED
... [98 similar lines]
api/test_api.py::test_case_299 PASSED
```

Only the two tool results changed; every other message is the same object, and tool_calls and tool_call_id are untouched. The budget is 50% of the window minus the 220 tokens of the request that are not in the messages (Hermes's estimate of 7,500 minus the 7,280 counted here). The test output is a favourable case, because runs of near-identical lines collapse into a count; prose is compressed only as far as the budget requires.

## Options

| Option | Default | Meaning |
|---|---|---|
| `threshold_percent` | `0.75` | Compress when the prompt reaches this share of the context window. |
| `target_percent` | `0.5` | Compress to about this share of the window (must be below threshold_percent). |
| `protect_first_n` | `3` | Messages after the system prompt kept verbatim. |
| `protect_last_n` | `6` | Most recent messages kept verbatim (tool results among them only as a last resort). |
| `min_tokens` | `200` | Messages with at most this many tokens are not compressed. |
| `relevance` | `"lexical"` | "lexical", "semantic" (needs llmslim[semantic]) or a callable scorer(query, texts) -> scores. |
| `additional_compress_kwargs` | `None` | Other llmslim.ContextCompressor options, for example mode. |

## Deployment

Install llmslim into the Python environment Hermes runs in, or use the directory plugin so that Hermes 0.21 and newer installs it and keeps it across updates. Enable the plugin with hermes plugins enable llmslim and select the engine with context.engine: llmslim in ~/.hermes/config.yaml. Options are constructor arguments or go under plugins.entries.llmslim.settings in config.yaml. Hermes gives every agent its own copy of the engine (copy.deepcopy); the engine holds only plain data.

## Tips

- Use /compress <topic> to focus a compaction: the topic is used as the query, so each message keeps the parts about that topic first.
- Keep target_percent below threshold_percent. The engine compresses once the prompt reaches threshold_percent of the window, down to about target_percent.
- Raise protect_first_n or protect_last_n to keep more of the head or the latest turns verbatim. Tool results among the last protect_last_n are compressed only as a last resort.
- Messages with at most min_tokens tokens are not compressed; raise it to leave more short messages untouched.

## Limitations

- Needs llmslim 0.9.0 or newer, the first release with this engine. The 0.9.0 release was tested with Hermes Agent 0.19.0 (PyPI) and v0.21.7 (GitHub).
- There is no llmslim[hermes] extra. A managed Hermes install may rebuild its environment on hermes update and drop packages installed by hand; the directory plugin keeps its python_dependencies across updates only on Hermes 0.21 and newer.
- The cached prefix is sent unchanged only while compressing the newer messages makes enough room. Otherwise the earlier messages are compressed again.
- When compression is not enough, the oldest messages are replaced by a one-line placeholder.

## FAQ

### How is it different from Hermes's built-in compressor?

It compresses older tool results and turns locally instead of asking a model for a summary. There is no LLM call and the output is deterministic.

### Does the provider's prompt cache keep hitting?

Yes, while compressing the newer messages is enough. The system prompt and the protected head are returned as the same objects, and on the next compaction messages that are exactly what the previous compaction returned are left alone. The cached prefix (system prompt, head and earlier compacted history) is then sent unchanged and only what follows it changes. If the newer messages cannot make enough room, the earlier ones are compressed again.

### Can a tool call lose its result?

No. No message is removed, so every tool call keeps its result, and tool_calls and tool_call_id are left untouched.

### Why is there no llmslim[hermes] extra?

Current Hermes releases are installed with Hermes's installer, Docker image or Nix, and the hermes-agent package on PyPI is an old 0.19 release. The engine needs nothing beyond llmslim, so install llmslim into Hermes's environment or use the directory plugin.

### Which Hermes versions was it tested with?

Hermes Agent 0.19.0 from PyPI and v0.21.7 from GitHub, with llmslim 0.9.0. On Hermes 0.21 and newer, the directory plugin's python_dependencies are installed when you enable the plugin.

## Troubleshooting

- **ImportError: llmslim.integrations.hermes needs Hermes Agent (agent.context_engine):** The engine imports from Hermes itself. Install llmslim into the Python environment Hermes runs in, not into a separate one.
- **Hermes still uses its built-in compressor:** Enable the plugin (plugins.enabled: [llmslim], or hermes plugins enable llmslim) and select the engine with context.engine: llmslim in ~/.hermes/config.yaml.
- **The engine is gone after hermes update:** A managed Hermes install may rebuild its environment and drop packages installed by hand. On Hermes 0.21 and newer, use the directory plugin, whose python_dependencies are kept across updates; on older releases, install llmslim into Hermes's environment again.
- **ValueError: need 0 < target_percent < threshold_percent <= 1:** target_percent must be below threshold_percent. Lower target_percent or raise threshold_percent.

## Further reading

- [Hermes Agent guide on GitHub](https://github.com/Thanatos9404/llmslim/blob/main/docs/integrations/HERMES.md)
- [Engine source: llmslim/integrations/hermes.py](https://github.com/Thanatos9404/llmslim/blob/main/llmslim/integrations/hermes.py)
- [Hermes Agent context compression developer guide](https://hermes-agent.nousresearch.com/docs/developer-guide/context-compression-and-caching)
- [Benchmarks](https://www.llmslim.app/benchmarks.md)

Canonical page: https://www.llmslim.app/integrations/hermes-agent