- Harness: local tool executor + OpenAI Chat Completions (
reasoning_effort: none) - Task: if ABC-9 inventory < 10 and price < 50 →
submit_reorder(ABC-9, 25)→ return confirmation JSON - Measured 2026-07-31: PASS · 3 tool calls · 1043 in / 99 out · $0.000327 · 3508 ms
- Why it matters: proves Luna agent loops on the same stack we host — not a screenshot of someone else’s Terminal-Bench
- Scope: one policy task, one run — extend the script for bake-offs; see also benchmark inflation
Why a tiny owned harness
METR and the SWE-bench Pro audit (see our inflation piece) are reminders that agent scores are harness-dependent. We already publish scheduling cadence stats from real CloudAxis duties (scheduling benchmark). This post adds a tool-loop microbench we can re-run after model or SDK changes.
Task and tools
Hidden world state: ABC-9 costs 42.5 with inventory 4. Tools: get_price, get_inventory, submit_reorder. The model must not invent numbers — only tool results count. Final answer must be JSON with the confirmation id.
Result
On 2026-07-31, gpt-5.6-luna completed the policy correctly:
- Success: PASS
- Tool calls: 3 (price → inventory → reorder)
- Tokens: 1043 prompt / 99 completion
- List cost: ~$0.000327 at Luna $0.20/$1.20
- Latency: 3508 ms wall time
- Final:
{"confirmation_id":"REORDER_MS88UX19","sku":"ABC-9","qty":25}
Log: sites/cloudaxis/content/gpt-5-6-luna-agentic-harness-measurement.mjs. Runner: gpt-5-6-luna-agentic-harness.mjs (needs OPENAI_API_KEY; we executed inside the CloudyBot API container on Hetzner).
What this is not
- Not a browser / Browserbase E2E (separate suite)
- Not SWE-bench or Terminal-Bench
- Not a multi-model bake-off yet — Luna-only pass to keep cost and variance low
- Not a claim Luna wins every agentic chart — only that this owned loop works at nano-ish price
Related
Frequently asked questions
Is this a full agentic leaderboard?
No. It is a lean, owned tool-loop task we can re-run cheaply after model changes. Extend the script for broader bake-offs.
Did you use the Responses API?
This pass used Chat Completions with function tools and reasoning_effort set to none — the path our lean agent forces for GPT-5.6 + tools.
How much did the run cost?
About $0.000327 at Luna list rates for 1043/99 tokens.
Can I reproduce it?
Yes. Run sites/cloudaxis/content/gpt-5-6-luna-agentic-harness.mjs with OPENAI_API_KEY on a host that can reach api.openai.com.