- Measured 2026-07-31 on Hetzner via OpenAI Chat Completions (
reasoning_effort: none, temperature 0) - Phase 1: Luna exact-cited a mid-document needle at ~8k → ~299,112 prompt tokens; Terra also passed ~300k mid-depth
- Phase 2: Luna passed depths 0.1 / 0.5 / 0.9 at 128k, 200k, and 300k, plus summarize-then-cite at 300k (11/11 scored cells correct)
- Viral “breaks at 300k” claim: not reproduced on this synthetic NIAH harness
- Limits: one needle, synthetic filler, short answers — not multi-hop RAG, not adversarial distraction, not a full 1M sweep
Sibling: Luna guide · Sol vs Terra vs Luna · cost per completed task.
Method (repeatable, cheap)
We plant a unique CLOUDYBOT_NEEDLE_… string inside a long synthetic inventory document, then ask the model to return only that code. Prompt size is taken from API usage.prompt_tokens. Script: sites/cloudaxis/content/gpt-5-6-luna-long-context-recall.mjs; logged results in gpt-5-6-luna-long-context-measurement.mjs.
- Models:
gpt-5.6-luna(primary),gpt-5.6-terra(anchors) - Depth: fraction of the document where the needle sits (0.1 early · 0.5 mid · 0.9 late)
- Extra probe: summarize in 2 sentences, then cite the needle (300k Luna)
- Cost control: short max completion tokens; no Sol 300k runs; no 500k–1M sweep in this pass
Results
Phase 1 — mid-depth Luna ladder: exact cite at every checkpoint through 299,112 prompt tokens (3734 ms). Terra mid-depth at ~300k also exact-cited (299,112 tokens).
| Model | Target | Prompt tokens | Result |
|---|---|---|---|
gpt-5.6-luna | ~8k | 7,798 | exact cite |
gpt-5.6-luna | ~32k | 31,104 | exact cite |
gpt-5.6-luna | ~64k | 63,110 | exact cite |
gpt-5.6-luna | ~128k | 127,122 | exact cite |
gpt-5.6-luna | ~200k | 199,116 | exact cite |
gpt-5.6-luna | ~300k | 299,112 | exact cite |
gpt-5.6-terra | ~128k | 127,122 | exact cite |
gpt-5.6-terra | ~300k | 299,112 | exact cite |
Phase 2 — depth sweep: Luna at 128k / 200k / 300k × depths 0.1, 0.5, 0.9 — all exact cites. Summarize-then-cite at 300k mid-depth: exact cite. Terra early-depth (0.1) at 300k: exact cite. Hard-sweep failures: 0.
What this does and does not prove
It proves that on this NIAH harness, Luna still exact-cites a single planted string through ~300k prompt tokens across early/mid/late depths — including after a forced short summary. It does not prove perfect usable recall across a full 1M window, multi-hop retrieval, or noisy real corpora.
- Secondary “summarizes instead of cites near 300k” reports may use different tasks, distractors, or evaluation rubrics — we are not calling those authors wrong without matching their harness
- We did not burn a full 500k–1M matrix here (token cost × variance); extend the script if you need that curve
- For product guidance: treat Luna’s 1M window as capacity with task-dependent usable recall — measure your own documents
Reproduce
On a host with OPENAI_API_KEY (we used the CloudyBot API container on Hetzner):
node sites/cloudaxis/content/gpt-5-6-luna-long-context-recall.mjs
# optional harder sweep: e1-hard-sweep.mjs (depth × summarize)
Related
Frequently asked questions
Does GPT-5.6 Luna really fail long-context recall at 300k?
Not on our needle-in-haystack harness dated in this post. Luna exact-cited through ~299k prompt tokens at early, mid, and late depths, and after a summarize-then-cite prompt. Other tasks may still degrade earlier.
Did you test the full 1M window?
No. This pass stops near 300k to control cost. The methodology script can be extended.
How is this different from the context window spec?
Context window is capacity. Recall is whether the model can use a fact buried in that capacity. Specs do not equal usable recall on every task.
Can I reproduce this?
Yes. The measurement scripts and JSON logs ship under sites/cloudaxis/content/. You need an OpenAI API key and the gpt-5.6-luna model id.