Skip to main content
LLMSlim
Try Studio
Back to integrations
LangChain
FrameworkDocument compressor

LLMSlim + LangChain

A BaseDocumentCompressor for ContextualCompressionRetriever. Use it where you would use LLMLinguaCompressor.

LLMSlimCompressor is a LangChain BaseDocumentCompressor backed by llmslim.compress_documents. It keeps the documents that answer the query inside a token budget, runs locally with no model and no API call, and keeps each document's id and metadata.

1. Package installation

The extra installs langchain-core 0.3 or newer. ContextualCompressionRetriever lives in langchain_classic.retrievers.contextual_compression in LangChain 1.x and in langchain.retrievers.contextual_compression in LangChain 0.3.

terminalbash
pip install "llmslim[langchain]"   # langchain-core; needs llmslim 0.8.1 or newer
pip install langchain-classic      # for ContextualCompressionRetriever (LangChain 1.x)

2. Architecture & execution flow

1

Use it as the base_compressor of a ContextualCompressionRetriever, or as one step of a DocumentCompressorPipeline.

2

compress_documents(documents, query) calls llmslim.compress_documents on the documents' page_content.

3

With a query, the documents share one budget (target_ratio of their combined tokens) and it goes where the query points: the most relevant documents stay whole and irrelevant ones are dropped.

4

Without a query, each document is compressed to target_ratio on its own.

5

Kept documents keep their order, id and metadata. Documents that are empty or compress to nothing are dropped. The input documents are not modified.

3. Runnable example and its output

langchain_retriever.pypython
from langchain_classic.retrievers.contextual_compression import ContextualCompressionRetriever
from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.vectorstores import InMemoryVectorStore

from llmslim.integrations.langchain import LLMSlimCompressor

docs = [
    Document(
        page_content="Standard shipping takes three to seven business days within the country. "
        "Express shipping arrives in one or two business days and costs fifteen dollars. "
        "We ship to forty countries, and international orders clear customs in about a week. "
        "Tracking numbers are emailed as soon as the parcel leaves the warehouse.",
        metadata={"source": "shipping.md"},
    ),
    Document(
        page_content="You can return any item within 30 days of delivery for a full refund. "
        "Refunds go back to the original payment method within five business days. "
        "Items must be unused and in their original packaging. "
        "Gift cards and final-sale items cannot be returned.",
        metadata={"source": "refunds.md"},
    ),
    Document(
        page_content="The company was founded in 2015 by two engineers in Pune. "
        "Our team of eighty people works across design, engineering and support. "
        "The office has a rooftop garden and a small library of design books. "
        "Every Friday afternoon the team hosts an open demo for customers.",
        metadata={"source": "about.md"},
    ),
]

# Any retriever works here. The fake embedding keeps the example offline.
vectorstore = InMemoryVectorStore.from_documents(docs, DeterministicFakeEmbedding(size=256))
retriever = ContextualCompressionRetriever(
    base_compressor=LLMSlimCompressor(target_ratio=0.25),
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 3}),
)

for doc in retriever.invoke("How many days do I have to return an item for a refund?"):
    print(doc.metadata)
    print(doc.page_content)

Output (Python 3.12, langchain-core 1.6.9, langchain-classic 1.0.8, token counts from tiktoken):

outputtext
{'source': 'refunds.md', 'llmslim_original_tokens': 50, 'llmslim_compressed_tokens': 40}
You can return any item within 30 days of delivery for a full refund. Refunds go back to the original payment method within five business days. Items must be unused and in their original packaging.

The three documents share one budget of 25% of their 158 tokens (40 tokens). The shipping and company pages do not answer the question and are dropped; the refund page keeps the answer and loses its sentence about gift cards.

4. Options

Keyword arguments, shown as name = default.

target_ratio = 0.25
Fraction of tokens to keep.
relevance = "lexical"
"lexical" (BM25, no model), "semantic" (cross-encoder, needs llmslim[semantic]) or a callable scorer(query, texts) -> scores.
selection = "auto"
"proportional" gives each document its own share, as in 0.7.
context_role = "rag"
Retrieved text is untrusted: imperative wording in it is never force-kept.
record_token_counts = True
Adds llmslim_original_tokens and llmslim_compressed_tokens to the metadata.
additional_compress_kwargs = {}
Other llmslim.compress_documents arguments, for example mode.

5. Deployment best practices

Place the compressor between retrieval and the prompt, and pass the user's question as the query so the retrieved texts share one budget. Start at the default target_ratio of 0.25 and check answer quality on your own questions before going lower. The default lexical relevance needs no model; when questions and answers use different words, install llmslim[semantic] and pass relevance="semantic" to add the optional 22M-parameter cross-encoder.

6. Key optimization tips

  • Pass the user's question as the query. Without one, each text is compressed to target_ratio on its own instead of competing for one budget.
  • Keep the lexical default when you need deterministic output and no model download; switch to relevance="semantic" when questions and answers share few words.
  • Keep context_role at "rag" for retrieved text: imperative wording inside untrusted documents is never force-kept.
  • Set record_token_counts=False if downstream code should not see the llmslim_original_tokens and llmslim_compressed_tokens metadata keys.

7. Limitations

  • Needs llmslim 0.8.1 or newer and langchain-core 0.3 or newer. The 0.8.1 release was tested with langchain-core 0.3.0 and 1.6.9.
  • Extractive: it keeps or drops whole documents and sentences, copied from the input. It does not shorten or rewrite the sentences it keeps.
  • Compression costs some accuracy. On NaturalQuestions at a quarter of the tokens, LLMSlim with its optional reranker scored 78.7% against 89.0% for the full context, read by Claude Haiku 5.5. Measure on your own questions before choosing a ratio.
  • relevance="semantic" needs pip install "llmslim[semantic]" (sentence-transformers) and downloads a 22M-parameter cross-encoder from Hugging Face on first use.
  • The example uses DeterministicFakeEmbedding to stay offline. Which documents reach the compressor depends on your real retriever.

Frequently asked questions

Troubleshooting

Further reading