# Mitigating U-Shaped Attention Recall Decay in Long Context Prompts

> Structuring Information Density to Overcome Position-Dependent Attention Loss

By Yashvardhan Thanvi (LLMSlim Author & Core Maintainer) · Published July 15, 2026 · Updated July 15, 2026 · Context Engineering

## Abstract

An examination of empirical attention position bias in Transformer architectures and how sentence-level centrality ranking repositions core information relative to head and tail context bounds.

## Intuition

Transformers exhibit maximum retrieval fidelity at context boundaries (0-15% and 85-100%). Pruning redundant middle prose elevates critical sentences into higher attention regions.

## 1. The Lost-in-the-Middle Phenomenon

Research by Liu et al. (2023) established that decoder language models exhibit a U-shaped performance curve when retrieving information from input documents:

- **Head Bias**: High recall accuracy when target information resides near the initial system directives ($0-15\%$ of context).
- **Tail Bias**: High recall accuracy when target information resides adjacent to the final prompt query ($85-100\%$ of context).
- **Middle Degradation**: Statistically significant drop in recall performance when critical facts reside in the middle $30-70\%$ region.

By identifying and removing non-essential filler sentences from document contexts, LLMSlim reduces overall sequence length, effectively moving mid-document facts closer to high-attention boundaries.

## 2. Query-Aware Document Compression Implementation

Using `compress_documents()` to extract relevant sentences across retrieved vector chunks:

```python
from llmslim import compress_documents

retrieved_chunks = [
    "Doc Chunk 1: Background corporate history founded in 2012...",
    "Doc Chunk 2: Q3 Financial Results: Net operating income reached $4.2M, representing a 14% YoY increase...",
    "Doc Chunk 3: Additional administrative overhead details and disclaimers..."
]

query = "What was the Q3 net operating income?"

# Compress documents with query-focused sentence scoring
compressed_docs = compress_documents(retrieved_chunks, query=query, target_ratio=0.4)

for idx, doc in enumerate(compressed_docs):
    print(f"--- Chunk {idx+1} ---")
    print(doc.compressed_text)
```

## Key takeaways

- Information positioned in the middle 30-70% of long context prompts suffers from systematic recall degradation.
- Extracting central informational sentences reduces total prompt volume and moves key facts closer to instruction boundaries.
- Combines query relevance scoring with position preservation to maintain narrative coherence.

## References

- [Liu et al. (2023)] [Lost in the Middle: How Language Models Use Long Contexts (Transactions of the Association for Computational Linguistics)](https://arxiv.org/abs/2307.03172)

Canonical page: https://www.llmslim.app/articles/lost-in-the-middle-mitigation