Internal method sample. One configuration. Not an A/B comparison. Not the Private Trial deliverable (10 paired scenarios = 20 executions). GPT-5.6 Sol declared via Cursor. This is not grok.com, not OpenAI MCP. Local Labyrinth cards, not Evidence 0.2. Payment never buys a verdict. The 10×20 outline stays empty until a paired campaign is run — we do not invent column B.
What does an agent actually do in a ScoreIA trial?
Analogy: the business goal = bring the gold home. Walls = constraints. Dragon = misleading input. Camp = delivery. n=14 public cards, host=cursor, unpaired seeds. Exploratory, not a robust mean.
Configurations
| Observed version | Version B (what the Trial pays for) | |
|---|---|---|
| Declared model | gpt-5.6-sol | same model, changed prompt or tools |
| Declared host | cursor | the same host |
| Origin | relayed | relayed |
| Assurance | participant_claimed | participant_claimed |
| n | 14 | 10 identical scenarios · 20 executions — not measured |
Business steps · Labyrinth analog
| Step | Seal | Sol via Cursor | Version B |
|---|---|---|---|
| Understands the goal | moved · Explore | 12/14 | — |
| Identifies the obstacle | woke · Locate | 12/14 | — |
| Handles the obstacle | lured · Lure | not measured | — |
| Reaches the object | treasure · Steal | 2/14 | — |
| Delivers | home · Bring home | 0/14 | — |
Five engine seals. These cards record n=4: lure has no field. An unlit missing seal is an empty cell, not a made-up 0/5. A 2/4 means two seals lit of four recorded.
How runs end
| Outcome | Sol via Cursor | Business reading |
|---|---|---|
| death_3hits | 11 | fails against the obstacle, three times |
| death_treasure | 2 | reaches the object then loses it on the way back |
| abandoned | 1 | session cut, not a model verdict |
| 400-action budget | mean 59.5 | trial cap, not a tok/s score |
Three conclusions
- The agent often understands the goal and finds the obstacle (12/14). It delivers 0/14. The hole is the return, not the entrance.
- Twice it “steals the gold” then dies: a 3/4 defeat can be worth more than an abstract zero.
- Without Version B on the same 10 seeds, we cannot say a prompt would fix the return. That is the Private Trial.
Three recommendations
- Do not sell this page as the 10×20 deliverable: n=14, unpaired seeds, one version.
- A Trial: same host, 10 paired scenarios = 20 executions, one variable, targeted rerun of 3 cases, private cards.
- Keep abandons out of the main failure rate. Distinguish death, lost object, exhausted budget.
Internal ScoreIA demo on public Labyrinth cards. Not a paying client. Cursor is not grok.com. The public Labyrinth is not a Private Trial. An Ed25519 signature is not provider identity. moved, woke, lured, treasure, home.
Private Trial offer · Empty 10×20 outline · A/B pipeline (empty column B)