The Notepad Mistake
When people first start working with AI models, they treat the context window like a notepad: dump everything in, ask a question at the bottom, hope for the best.
This works when inputs are short. It breaks down fast when they're not.
A 30-page contract. A 400-comment support thread. A year of product feedback. These aren't prompts — they're corpora. And how you organize them determines whether the model extracts signal or wanders.
After analyzing the highest-performing long-context workflows across Lumina's enterprise users, the patterns are consistent enough to document.
The Hierarchy Problem
AI models process context sequentially but attend non-uniformly. Content near the beginning and end of the context window generally receives more attention than content in the middle — a phenomenon sometimes called the "lost in the middle" problem.
This means the naive approach — paste all your content, then ask a question — is the worst possible structure for long inputs. Your question is at the very end. The material it's asking about is distributed across positions of varying attention weight.
The better structure inverts this:
[Instruction and question first]
[Most relevant excerpts next]
[Full context last, if needed]Lead with what you want. Put the highest-signal material early. Use the bulk content as reference, not as the primary input.
Chunking Strategies That Work
Hierarchical summarization for large documents
Don't send a 50-page document as a single context block. Chunk it into sections, summarize each section first, then send the collection of summaries with the full document appended as reference.
// Effective structure for a large document
DOCUMENT SUMMARY (generated in a prior step):
- Section 1 — Contract parties and effective date: ...
- Section 2 — Payment terms: net-30, late fees 1.5%/month...
- Section 3 — Liability cap: limited to 12 months of fees paid...
[...]
QUESTION: Does this contract include an automatic renewal clause?
FULL DOCUMENT:
[paste full text]This gives the model an orientation layer before it encounters the dense text. Outputs improve significantly.
Relevance filtering before submission
For knowledge base lookups and document search, never send the entire corpus. Retrieve the top-k relevant chunks first (using embeddings or keyword search), then submit only those chunks as context.
Sending 10 highly relevant paragraphs produces better results than sending a 100-page document and hoping the model finds the needle.
Explicit section labels
Label every distinct block of content with a clear header. Not markdown-style headings for aesthetics — explicit semantic labels that tell the model what type of information follows.
[COMPANY BACKGROUND — For context only, do not quote directly]
...
[CURRENT SUPPORT TICKET — Primary input]
...
[SIMILAR RESOLVED TICKETS — Reference examples]
...
[TASK]
Based on the current support ticket, draft a response using the resolved tickets as style reference.Models are excellent at following explicit structure. They are inconsistent at inferring it.
Information Hierarchy Principles
Three rules that hold across every long-context workflow we've analyzed:
Rule 1: Instructions before content, always. The model should know what it's supposed to do before it reads what it's supposed to do it to. Instructions after 10,000 tokens of content arrive at diminished attention.
Rule 2: Most specific → most general. If you're including both a specific question and general context, the specific question comes first. The model builds its attention frame from the beginning.
Rule 3: Separate roles explicitly. If different blocks of content play different roles (background, primary input, examples, constraints), label those roles explicitly. "This section is for context only and should not be quoted" is a legitimate instruction. Use it.
Practical Template
For any long-context workflow, this structure outperforms an unstructured dump:
ROLE: [Who the model is in this task]
TASK: [What you want produced, specifically]
OUTPUT FORMAT: [Exact structure of what you expect back]
CONSTRAINTS: [What to avoid, length limits, tone]
PRIMARY INPUT:
[The material that directly answers the task]
SUPPORTING CONTEXT:
[Background, reference material, examples]
CONFIRMATION: Perform the task above using only the provided inputs.The "CONFIRMATION" line at the end is not redundant. It re-anchors the model after reading a long context, reducing the chance of drift.
What This Changes in Practice
Teams that implement structured context organization typically see two improvements: better output quality on the first pass, and significantly lower re-prompting rates.
The re-prompting rate is the more important metric. If you're sending a prompt, reading the output, and then sending a correction 60% of the time, that's a structural failure in your prompt — not an intelligence failure in the model. Fixing the structure is almost always faster than engineering better corrections.
Treat your context window as a document, not a conversation. Structure it accordingly.