# LLMSlim + LangChain

> A BaseDocumentCompressor for ContextualCompressionRetriever. Use it where you would use LLMLinguaCompressor.

LLMSlimCompressor is a LangChain BaseDocumentCompressor backed by llmslim.compress_documents. It keeps the documents that answer the query inside a token budget, runs locally with no model and no API call, and keeps each document's id and metadata.

Category: Framework

## Install

```bash
pip install "llmslim[langchain]"   # langchain-core; needs llmslim 0.8.1 or newer
pip install langchain-classic      # for ContextualCompressionRetriever (LangChain 1.x)
```

The extra installs langchain-core 0.3 or newer. ContextualCompressionRetriever lives in langchain_classic.retrievers.contextual_compression in LangChain 1.x and in langchain.retrievers.contextual_compression in LangChain 0.3.

## How it fits

- Use it as the base_compressor of a ContextualCompressionRetriever, or as one step of a DocumentCompressorPipeline.
- compress_documents(documents, query) calls llmslim.compress_documents on the documents' page_content.
- With a query, the documents share one budget (target_ratio of their combined tokens) and it goes where the query points: the most relevant documents stay whole and irrelevant ones are dropped.
- Without a query, each document is compressed to target_ratio on its own.
- Kept documents keep their order, id and metadata. Documents that are empty or compress to nothing are dropped. The input documents are not modified.

## Example

```python
from langchain_classic.retrievers.contextual_compression import ContextualCompressionRetriever
from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.vectorstores import InMemoryVectorStore

from llmslim.integrations.langchain import LLMSlimCompressor

docs = [
    Document(
        page_content="Standard shipping takes three to seven business days within the country. "
        "Express shipping arrives in one or two business days and costs fifteen dollars. "
        "We ship to forty countries, and international orders clear customs in about a week. "
        "Tracking numbers are emailed as soon as the parcel leaves the warehouse.",
        metadata={"source": "shipping.md"},
    ),
    Document(
        page_content="You can return any item within 30 days of delivery for a full refund. "
        "Refunds go back to the original payment method within five business days. "
        "Items must be unused and in their original packaging. "
        "Gift cards and final-sale items cannot be returned.",
        metadata={"source": "refunds.md"},
    ),
    Document(
        page_content="The company was founded in 2015 by two engineers in Pune. "
        "Our team of eighty people works across design, engineering and support. "
        "The office has a rooftop garden and a small library of design books. "
        "Every Friday afternoon the team hosts an open demo for customers.",
        metadata={"source": "about.md"},
    ),
]

# Any retriever works here. The fake embedding keeps the example offline.
vectorstore = InMemoryVectorStore.from_documents(docs, DeterministicFakeEmbedding(size=256))
retriever = ContextualCompressionRetriever(
    base_compressor=LLMSlimCompressor(target_ratio=0.25),
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 3}),
)

for doc in retriever.invoke("How many days do I have to return an item for a refund?"):
    print(doc.metadata)
    print(doc.page_content)
```

Output (Python 3.12, langchain-core 1.6.9, langchain-classic 1.0.8, token counts from tiktoken):

```text
{'source': 'refunds.md', 'llmslim_original_tokens': 50, 'llmslim_compressed_tokens': 40}
You can return any item within 30 days of delivery for a full refund. Refunds go back to the original payment method within five business days. Items must be unused and in their original packaging.
```

The three documents share one budget of 25% of their 158 tokens (40 tokens). The shipping and company pages do not answer the question and are dropped; the refund page keeps the answer and loses its sentence about gift cards.

## Options

| Option | Default | Meaning |
|---|---|---|
| `target_ratio` | `0.25` | Fraction of tokens to keep. |
| `relevance` | `"lexical"` | "lexical" (BM25, no model), "semantic" (cross-encoder, needs llmslim[semantic]) or a callable scorer(query, texts) -> scores. |
| `selection` | `"auto"` | "proportional" gives each document its own share, as in 0.7. |
| `context_role` | `"rag"` | Retrieved text is untrusted: imperative wording in it is never force-kept. |
| `record_token_counts` | `True` | Adds llmslim_original_tokens and llmslim_compressed_tokens to the metadata. |
| `additional_compress_kwargs` | `{}` | Other llmslim.compress_documents arguments, for example mode. |

## Deployment

Place the compressor between retrieval and the prompt, and pass the user's question as the query so the retrieved texts share one budget. Start at the default target_ratio of 0.25 and check answer quality on your own questions before going lower. The default lexical relevance needs no model; when questions and answers use different words, install llmslim[semantic] and pass relevance="semantic" to add the optional 22M-parameter cross-encoder.

## Tips

- Pass the user's question as the query. Without one, each text is compressed to target_ratio on its own instead of competing for one budget.
- Keep the lexical default when you need deterministic output and no model download; switch to relevance="semantic" when questions and answers share few words.
- Keep context_role at "rag" for retrieved text: imperative wording inside untrusted documents is never force-kept.
- Set record_token_counts=False if downstream code should not see the llmslim_original_tokens and llmslim_compressed_tokens metadata keys.

## Limitations

- Needs llmslim 0.8.1 or newer and langchain-core 0.3 or newer. The 0.8.1 release was tested with langchain-core 0.3.0 and 1.6.9.
- Extractive: it keeps or drops whole documents and sentences, copied from the input. It does not shorten or rewrite the sentences it keeps.
- Compression costs some accuracy. On NaturalQuestions at a quarter of the tokens, LLMSlim with its optional reranker scored 78.7% against 89.0% for the full context, read by Claude Haiku 5.5. Measure on your own questions before choosing a ratio.
- relevance="semantic" needs pip install "llmslim[semantic]" (sentence-transformers) and downloads a 22M-parameter cross-encoder from Hugging Face on first use.
- The example uses DeterministicFakeEmbedding to stay offline. Which documents reach the compressor depends on your real retriever.

## FAQ

### Can LLMSlimCompressor replace LLMLinguaCompressor?

Yes, in the same places. Both are LangChain BaseDocumentCompressor classes, so LLMSlimCompressor goes in the base_compressor slot of a ContextualCompressionRetriever or in a DocumentCompressorPipeline. Two differences: it takes a fraction of tokens to keep (target_ratio) rather than a token count, and it needs no language model by default. The LLMSlim vs LLMLingua page compares the two approaches on a public benchmark.

### Does it call a model or an API?

No. The default lexical relevance is BM25 in your process, with no model and no network call. relevance="semantic" runs a 22M-parameter cross-encoder locally after pip install "llmslim[semantic]". Your documents are not sent anywhere either way.

### Which LangChain versions are supported?

langchain-core 0.3 or newer. The 0.8.1 release was tested with langchain-core 0.3.0 and 1.6.9.

## Troubleshooting

- **ImportError: llmslim.integrations.langchain requires the optional extra:** Install the extra in the same environment: pip install "llmslim[langchain]". The adapter needs llmslim 0.8.1 or newer.
- **No module named langchain_classic:** On LangChain 1.x, pip install langchain-classic. On LangChain 0.3, import ContextualCompressionRetriever from langchain.retrievers.contextual_compression instead.
- **Every document comes back cut to the same fraction:** The query was empty, so each document was compressed to target_ratio on its own. Pass the user's question as the query so the documents share one budget and the irrelevant ones are dropped.

## Further reading

- [LangChain guide on GitHub](https://github.com/Thanatos9404/llmslim/blob/main/docs/integrations/LANGCHAIN.md)
- [Adapter source: llmslim/integrations/langchain.py](https://github.com/Thanatos9404/llmslim/blob/main/llmslim/integrations/langchain.py)
- [LLMSlim vs LLMLingua](https://www.llmslim.app/compare/llmlingua.md)
- [Benchmarks](https://www.llmslim.app/benchmarks.md)

Canonical page: https://www.llmslim.app/integrations/langchain