Changes with consequences.
LLMSlim releases document what shipped, what changed, and where the boundary remains.
LangChain compressor
LLMSlimCompressor is a BaseDocumentCompressor for ContextualCompressionRetriever. Install llmslim[langchain].
LlamaIndex postprocessor and Haystack component
LLMSlimPostprocessor is a BaseNodePostprocessor (Python 3.10+, llmslim[llamaindex]); LLMSlimDocumentCompressor is a Haystack component with run(documents, query) (llmslim[haystack]).
Same rules as the core
Retrieved text stays untrusted; documents that compress to nothing are dropped; token counts are recorded in metadata and kept out of the prompt for LlamaIndex.
Query-focused selection
Structural units are ranked with BM25, light stemming, pseudo-relevance feedback and a document prior, kept whole in their original order, then topped up sentence by sentence. compress_documents() shares one budget across documents by relevance.
Code and logs keep their lines
Failures and tracebacks come first, repeated lines collapse into a count, the tail is kept, and code keeps its skeleton plus the most relevant bodies. Compressing a 1.3k-token file dropped from about 3.5 s to 7 ms.
Optional reranker
relevance="semantic" rescores the strongest candidates with a 22M-parameter cross-encoder (the semantic extra), on CPU or GPU. Any scorer(query, texts) callable works too. Without it, LLMSlim still runs no model.
Guardrails still win
Trusted instructions and preserve_patterns survive inside large units; untrusted retrieved, tool and assistant text still cannot reach the protected tier. Query-less prose behaves exactly as in 0.7.
Most relevant evidence is kept
After the budget allocation, a deterministic guard keeps the single most relevant, non-superseded document, memory or tool result whenever it fits by dropping or compressing lower-relevance evidence.
Superseded documents go first
Items whose metadata sets superseded, superseded_by, or a past valid_until are scaled down and dropped before current evidence, with a warning in the plan.
Untrusted wording earns no value
Imperative text inside retrieved, tool or assistant content no longer raises its planning value. It is still rendered as untrusted data; there is no injection detector.
Cache plans and diagnostics
Stable-prefix fingerprints, append-only reuse estimates, cache generations, invalidation reasons, and separate network-byte and token metrics are visible in Context Inspector.
Provider request builders
Execution-free OpenAI, Anthropic, Gemini, and vLLM builders expose supported cache controls. OpenAI state continuation and Gemini cached resources require explicit host opt-in.
Reported data stays distinct
Local prefix matches are estimates until a provider reports usage. Sarvam caching remains unverified; raw KV is confined to an explicit self-hosted process adapter.
Agent Context Runtime
ContextEnvelope, ContextGraph, ContextPolicy, progressive planning, quality reporting, and bounded sessions prepare a turn without taking over the agent loop or tool execution.
Context Inspector and adapters
The Studio shows before and after context, decision reasons, graph links, and a prompt-free trace. Generic, Sarvam, OpenAI-compatible, and text-only OpenAI Agents adapters return host-owned model inputs.
Quality and budget tradeoff
Across 112 offline cases, the new runtime retained all measured required facts and tool contracts, with 67.9% budget success and 18.7% context reduction. Provider-backed answer quality remains unmeasured.
Adaptive Context Planner
Multiple-choice constrained allocation selects safe representations for trusted instructions, chat, RAG, memory, tool results, and tool schemas, then explains each decision.
Failure is explicit
Required trusted content is never silently truncated. Impossible budgets produce an infeasible plan, provider candidates fail closed, and tool selection never grants execution authority.
Optional ecosystem adapters
Official-SDK Sarvam rewriting, bounded read-only Zoho sources, and PyMongo async memory remain optional extras; the base planner stays local and provider-neutral.
MCP catalog sources
Streamable HTTP and literal-argv stdio support bounded pagination, scoped caching, contract fingerprints, and stale-plan detection.
Execution belongs to the host
The optional Agents SDK bridge requires a host callback. Full and measure-only plans retain every schema; selective exposure remains research-only.
A new home for LLMSlim
A redesigned website with light and dark themes, a Sarvam integration guide, and a note on LLMSlim's acceptance into the Sarvam Startup Program.
Stable tool-contract APIs
The released tool surface supports copied schema normalization, deterministic canonical JSON, SHA-256 fingerprints, exact contract checks, and safe catalog optimization.
Retrieval remains research-only
Lexical, dense, hybrid retrieval, selective exposure, and lazy hydration are explicit experiments. They do not authorize or execute tools, and compress() keeps its established behavior.
Schema-tax evidence stays transparent
The v0.4.0 release gate records 489 passing tests, 90.93% coverage, and 375 generated tool schemas in 18 catalogs. The compact baseline produced 0.00% lossless reduction.
Provenance-aware priority locking
ContextRole distinguishes system, developer, user, assistant, tool, RAG, and general content. Untrusted RAG, tool, and assistant text cannot become hard-locked priority from imperative wording, safety patterns, or preserve patterns.
Nonce-protected rewrite template fences
User text containing a template-fence-like delimiter remains byte-for-byte unchanged while the template uses a content-preserving nonce fence, preventing fence breakout.
Context roles and token-counter telemetry
The public Python API exports ContextRole and records token_counter_used so callers can distinguish tiktoken from the documented heuristic fallback.
Inline code and CJK sentence boundaries
Inline backtick code spans no longer create false sentence boundaries, and the CJK ideographic stops 。, ！, and ？ are recognized by the sentence-splitting paths.
Pipelines preserve provenance
Document compression defaults retrieved context to RAG, while chat compression propagates supplied system, developer, user, assistant, and tool roles instead of flattening them.
Benchmark collection reports real pytest results
The release runner collects actual pass/fail reports rather than replaying stale tuples. The checked-in release record reports 432 passing tests, zero failures, 92.57% branch coverage, and Ruff passing.
Rewrite and hybrid strategies
compress() gained extractive, rewrite, and hybrid strategies. Rewrite-capable paths require a caller-supplied provider; the default extractive path remains offline and deterministic.
Provider and validation contracts
BaseRewriteProvider, CallableProvider, RewriteRequest, RewriteMetadata, versioned templates, and structural, instruction, entity, and similarity validation were added as public building blocks.
Rewrite work moved into a focused sub-package
Prompt construction, execution, and validation were separated into llmslim.rewrite modules, with strategy benchmarking and a CLI strategy flag added alongside them.
Instruction and entity retention
The release added explicit instruction prioritization plus regex and heuristic preservation for names, dates, financial metrics, technical identifiers, and URLs.
Chunking, embeddings, and cost tooling
It introduced semantic and hybrid chunking, optional sentence-transformers support, model cost estimation, an interactive/file-based CLI, and comprehensive benchmark tooling.
Richer ranking and result telemetry
TF-IDF plus PageRank-style sentence centrality, optional tiktoken accounting, and a standardized CompressionResult with detailed telemetry became part of the release.
Installable LLMSlim package
v0.1.0 published the llmslim package for Python 3.8+ with the public compress(), ContextCompressor, CompressionResult, token, pipeline, and cost utility exports.
Offline prompt compression foundation
The initial engine provided ratio-driven prompt compression using TF-IDF sentence scoring and thresholding, with optional semantic, fast-token, and NLTK extras.