# LLMSlim + Pydantic AI

> A Pydantic AI history processor that fits the message history into a token budget before every model request. Tool calls and returns stay paired.

llmslim_history_processor(max_tokens=...) returns a Pydantic AI history processor. Before every model request it fits the message history into a token budget: older text is compressed by LLMSlim first, and older tool exchanges and turns are removed only when that is not enough. Tool calls and their returns are always kept or removed together, so the provider never sees an unpaired call or return. It runs locally, with no model call. Python 3.10+.

Category: Framework

## Install

```bash
pip install "llmslim[pydantic-ai]"   # pydantic-ai-slim 1.0 or newer, Python 3.10+; needs llmslim 0.9.0 or newer
```

The extra installs pydantic-ai-slim 1.0 or newer and needs Python 3.10 or newer. llmslim[all] includes it.

Pydantic AI 2.x (and recent 1.x releases) take the processor in the ProcessHistory capability; earlier 1.x releases take it in history_processors, which recent 1.x releases deprecate and 2.x removed:

```python
from pydantic_ai import Agent
from pydantic_ai.capabilities import ProcessHistory

from llmslim.integrations.pydantic_ai import llmslim_history_processor

fit = llmslim_history_processor(max_tokens=8_000)
agent = Agent("openai:gpt-5", capabilities=[ProcessHistory(fit)])

# Pydantic AI 1.x without pydantic_ai.capabilities:
agent = Agent("openai:gpt-5", history_processors=[fit])
```

## How it fits

- Pass it to an Agent in ProcessHistory(...), or in history_processors on earlier 1.x releases. It runs before each model request.
- System prompt parts, instructions, tool-availability changes and the most recent user prompt are never changed, and nothing is removed from the last keep_recent messages.
- Older text is compressed by one shared ratio: user prompts, model text and string tool results. Tool output and model text are untrusted, so imperative wording in them is never force-kept.
- If that is not enough, older tool exchanges are removed, oldest first (a model response with tool calls together with the request that returns them), then older turns whole, oldest first. The turn with the system prompt is never removed.
- As a last resort, the text in the last keep_recent messages is compressed, for example a very large tool result that just came back.
- A message is removed only together with every message that shares a tool call id with it, so the history still alternates request, response, request. Messages that do not change are passed through as the same objects; the input list is not modified.

## Example

```python
from pydantic_ai import Agent
from pydantic_ai.capabilities import ProcessHistory
from pydantic_ai.messages import ModelResponse, TextPart, ToolCallPart
from pydantic_ai.models.function import AgentInfo, FunctionModel

from llmslim.integrations.pydantic_ai import history_tokens, llmslim_history_processor


def run_tests(suite: str) -> str:
    """Run a test suite and return its output."""
    lines = [f"{suite}/test_api.py::test_case_{i} PASSED" for i in range(300)]
    lines.insert(200, f"{suite}/test_refund.py::test_rounding FAILED - 10.05 != 10.04")
    return "\n".join(lines)


def run(capabilities):
    sent = []  # history tokens in each request the model receives

    def fake_model(messages, info: AgentInfo) -> ModelResponse:
        """Stands in for an LLM: runs three test suites, then answers."""
        sent.append(history_tokens(messages))
        if len(sent) <= 3:
            return ModelResponse(parts=[ToolCallPart("run_tests", {"suite": f"suite{len(sent)}"})])
        return ModelResponse(parts=[TextPart("test_rounding fails in every suite.")])

    agent = Agent(
        FunctionModel(fake_model),
        system_prompt="You are a careful test runner.",
        tools=[run_tests],
        capabilities=capabilities,
    )
    result = agent.run_sync("Run the test suites and tell me what fails.")
    return result, sent


_, plain = run([])
result, slim = run([ProcessHistory(llmslim_history_processor(max_tokens=1_000))])
print(result.output)
print("history tokens per request without LLMSlim:", plain)
print("history tokens per request with LLMSlim:   ", slim)
print()
print(result.all_messages()[2].parts[0].content)
```

Output (Python 3.12 with pydantic-ai-slim 2.55.0, and Python 3.10 with 1.107.7 passing the processor in history_processors; token counts from tiktoken):

```text
test_rounding fails in every suite.
history tokens per request without LLMSlim: [17, 3946, 7875, 11804]
history tokens per request with LLMSlim:    [17, 110, 203, 296]

suite1/test_api.py::test_case_0 PASSED
... [198 similar lines]
suite1/test_api.py::test_case_199 PASSED
suite1/test_refund.py::test_rounding FAILED - 10.05 != 10.04
suite1/test_api.py::test_case_200 PASSED
... [98 similar lines]
suite1/test_api.py::test_case_299 PASSED
```

An agent runs three test suites, once without LLMSlim and once with it; FunctionModel stands in for the LLM, so the example runs offline. Every tool result stays in the history, paired with its call, and keeps the failure. The history ends far below the 1,000-token budget because runs of near-identical lines collapse into a count; this is a favourable case. Prose is compressed only as far as the budget requires.

## Options

| Option | Default | Meaning |
|---|---|---|
| `max_tokens` | `required` | Token budget for the history. |
| `keep_recent` | `2` | Most recent messages never removed and compressed only as a last resort (at least 1). |
| `min_tokens` | `50` | Parts with at most this many tokens are not compressed. |
| `relevance` | `"lexical"` | "lexical", "semantic" (needs llmslim[semantic]) or a callable scorer(query, texts) -> scores. |
| `additional_compress_kwargs` | `None` | Other llmslim.ContextCompressor options, for example mode. |

## Deployment

Pass the processor in ProcessHistory(...) on Pydantic AI 2.x and recent 1.x releases, or in history_processors on earlier 1.x releases. max_tokens counts the text of all message parts with llmslim.count_tokens; the instructions and tool definitions Pydantic AI adds to each request are not part of the history and are not counted. Pydantic AI stores the processed history, so result.all_messages() holds the compressed messages: keep your own copy if you need the originals.

## Tips

- Measure a history with history_tokens(messages); it gives the same count that max_tokens uses.
- Use fit_messages(messages, max_tokens, ...) for logging or tests: it does the same and also returns the token counts (HistoryFit).
- Raise keep_recent to protect more of the latest messages: they are never removed and are compressed only as a last resort.
- Parts with at most min_tokens tokens are not compressed; raise it to leave more short parts untouched.

## Limitations

- Needs llmslim 0.9.0 or newer, pydantic-ai-slim 1.0 or newer and Python 3.10 or newer. The 0.9.0 release was tested with pydantic-ai-slim 1.0.0, 1.107.7 and 2.55.0.
- max_tokens covers the message history only. The instructions and tool definitions Pydantic AI adds to each request are not counted.
- Pydantic AI stores the processed history, so result.all_messages() holds the compressed messages, not the originals.
- When compression is not enough, older tool exchanges and turns are removed whole. If the protected parts alone exceed the budget, the processor returns the smallest valid history it can rather than an invalid one.

## FAQ

### Can the provider see a tool call without its return?

No. Tool calls and their returns are always kept or removed together. A message is removed only together with every message that shares a tool call id with it, so a call answered several requests later is never orphaned, and the history still alternates request, response, request.

### ProcessHistory or history_processors?

Pydantic AI 2.x and recent 1.x releases take the processor in the ProcessHistory capability. Earlier 1.x releases, without pydantic_ai.capabilities, take it in history_processors, which recent 1.x releases deprecate and 2.x removed.

### What does max_tokens count?

The text of all message parts, counted with llmslim.count_tokens; history_tokens(messages) gives the same count. The instructions and tool definitions Pydantic AI adds to each request are not part of the history and are not counted.

### Does it call a model?

No. It runs locally, with no model call. In the example, FunctionModel stands in for the LLM, so the whole run is offline.

### Which Pydantic AI versions are supported?

pydantic-ai-slim 1.0 or newer, on Python 3.10 or newer. The 0.9.0 release was tested with pydantic-ai-slim 1.0.0, 1.107.7 and 2.55.0.

## Troubleshooting

- **ImportError: llmslim.integrations.pydantic_ai requires the optional extra:** Install the extra on Python 3.10 or newer: pip install "llmslim[pydantic-ai]". The module is llmslim.integrations.pydantic_ai; the extra is pydantic-ai.
- **No module named pydantic_ai.capabilities:** Earlier Pydantic AI 1.x releases have no ProcessHistory capability. Pass the processor in history_processors=[fit] instead.
- **history_processors is deprecated or no longer accepted:** Recent 1.x releases deprecate history_processors and 2.x removed it. Pass ProcessHistory(fit) in capabilities instead.
- **result.all_messages() returns compressed messages:** Pydantic AI stores the processed history. Keep your own copy of the messages if you need the originals.

## Further reading

- [Pydantic AI guide on GitHub](https://github.com/Thanatos9404/llmslim/blob/main/docs/integrations/PYDANTIC_AI.md)
- [Processor source: llmslim/integrations/pydantic_ai.py](https://github.com/Thanatos9404/llmslim/blob/main/llmslim/integrations/pydantic_ai.py)
- [Pydantic AI message history docs](https://pydantic.dev/docs/ai/core-concepts/message-history/)
- [Benchmarks](https://www.llmslim.app/benchmarks.md)

Canonical page: https://www.llmslim.app/integrations/pydantic-ai