Target: Provider-neutral model profilesBaseline: Raw, prefix, and fixed-ratio baselines

v0.6 Adaptive Context Planner

Frozen multilingual context planning with explicit quality and safety checks

The offline suite compares raw context, naive prefix truncation, fixed-ratio compression, and adaptive planning. It uses task-grounded string constraints rather than an LLM judge.

System environment

cpu

Environment-dependent; benchmark locally

ram

Environment-dependent

Cross-platform Python

python Version

Python 3.9+

package Version

llmslim v0.6.0 release candidate

tokenizer Version

tiktoken when available; otherwise documented heuristic

sample Size

28 frozen context-planning cases

iterations Per Sample

One deterministic plan per case and baseline; rerun for local latency

Empirical benchmark matrix

Method	Token reduction	Latency	Cost	Semantic retention	Instruction retention	Entity preservation
Raw full context	0.0%	Measurement only	Profile-dependent	100% required facts	100% hard constraints	71.4% budget success
Naive prefix	9.2%	2.21 ms*	Profile-dependent	100% required facts	100% in scoped checks	100% budget success
Fixed ratio	9.6%	26.55 ms*	Profile-dependent	94.0% required facts	96.4% hard constraints	78.6% budget success
Adaptive planner	11.4%	208.02 ms*	Estimated, labelled	100% required facts	100% hard constraints	96.4% budget success

Key insights

All required trusted instructions and complete selected tool contracts were retained in the checked-in run.
The one infeasible case is intentionally impossible and is reported explicitly rather than truncated.
SELECTIVE tool exposure remains experimental; FULL and MEASURE_ONLY remain production-safe.

Limitations and non-recommended workloads

The corpus is compact and synthetic; it is not a universal task-quality claim.
Latency is environment-specific and the checked-in run is not a throughput guarantee.
The live Sarvam harness requires explicit opt-in and credentials and is not part of the offline result.

Experimental methodology

Run the frozen 28-case corpus across chat, RAG, tools, mixed, external-source, and multilingual scenarios. Check budget fit, required facts, complete tool schemas, hard constraints, and stale-plan rejection.

Raw evaluation dataset

raw_dataset_sample.jsonjson

{
  "release": "v0.6.0-rc",
  "cases": 28,
  "categories": [
    "chat",
    "rag",
    "tools",
    "mixed",
    "external",
    "multilingual"
  ],
  "languages": 12,
  "live_sarvam": "NOT RUN without explicit opt-in and credentials"
}

Reproducible Python script

benchmark_reproducible.pypython

python benchmarks/v06_planner.py --output benchmarks/results/v0.6-planner-latest.json --report docs/planning/BENCHMARK_REPORT.md