Skip to main content
Back to Technical Papers Index
Context Engineering9 min read•Yashvardhan Thanvi (LLMSlim Author & Core Maintainer)•Published: July 15, 2026 (Updated: July 15, 2026)

Mitigating U-Shaped Attention Recall Decay in Long Context Prompts

Structuring Information Density to Overcome Position-Dependent Attention Loss

Mathematical Intuition & Formal Derivation

Transformers exhibit maximum retrieval fidelity at context boundaries (0-15% and 85-100%). Pruning redundant middle prose elevates critical sentences into higher attention regions.

Key Takeaways

  • 01.Information positioned in the middle 30-70% of long context prompts suffers from systematic recall degradation.
  • 02.Extracting central informational sentences reduces total prompt volume and moves key facts closer to instruction boundaries.
  • 03.Combines query relevance scoring with position preservation to maintain narrative coherence.

1. The Lost-in-the-Middle Phenomenon

Research by Liu et al. (2023) established that decoder language models exhibit a U-shaped performance curve when retrieving information from input documents: - **Head Bias**: High recall accuracy when target information resides near the initial system directives ($0-15\%$ of context). - **Tail Bias**: High recall accuracy when target information resides adjacent to the final prompt query ($85-100\%$ of context). - **Middle Degradation**: Statistically significant drop in recall performance when critical facts reside in the middle $30-70\%$ region. By identifying and removing non-essential filler sentences from document contexts, LLMSlim reduces overall sequence length, effectively moving mid-document facts closer to high-attention boundaries.

2. Query-Aware Document Compression Implementation

Using `compress_documents()` to extract relevant sentences across retrieved vector chunks:
rag_pruner.py
from llmslim import compress_documents

retrieved_chunks = [
    "Doc Chunk 1: Background corporate history founded in 2012...",
    "Doc Chunk 2: Q3 Financial Results: Net operating income reached $4.2M, representing a 14% YoY increase...",
    "Doc Chunk 3: Additional administrative overhead details and disclaimers..."
]

query = "What was the Q3 net operating income?"

# Compress documents with query-focused sentence scoring
compressed_docs = compress_documents(retrieved_chunks, query=query, target_ratio=0.4)

for idx, doc in enumerate(compressed_docs):
    print(f"--- Chunk {idx+1} ---")
    print(doc.compressed_text)

Academic Literature & Peer-Reviewed References