Let it think before it answers

Three months in, the order desk I built for a nut and bolt wholesaler (the first post) is doing real work. Last week the boss sent feedback: much better, but still struggling a bit with U-bolt sizes.

So I went looking at a U-bolt it had got wrong. The customer’s line was:

40mm square x 5” galvanised 1/2” U-bolt only

The desk picked a U-bolt with a 4½-inch leg. Close, but the customer asked for five. Then I read the reasoning it had written alongside the pick (I’ve swapped its part code for what the code describes):

The closest match is the 40mm square, 1/2” x 5” leg galvanised U-bolt, matching all three key dimensions.

It knew. It had found the right U-bolt, described it perfectly, and handed me a different one.

Reconstructed on invented part codes. The top card’s reasoning paraphrases the real one.

Answer first, working second

The picking step returns a small structured object, and I’d declared its fields in this order:

{ "chosen_sku": "...", "reasoning": "..." }

A language model writes its output in order, one token after another. So it committed to a part number first, then did its thinking, and by the time the thinking got to the right answer, the answer box was already filled in. It’s the student who writes 42 in the box and then shows working that comes out at 43.

The fix is to swap the two fields:

{ "reasoning": "...", "chosen_sku": "..." }

No change to the prompt. Not one word. Just the order.

Tempting to ship it on the spot

I ran that one line three times each way. Answer first got it right zero times out of three. Reason first, three out of three.

That’s the moment you want to ship. It’s two keys in a dictionary. But it changes every match the desk makes, and three runs on one line is an anecdote, not a result. So it went through the test harness first, the same way every change does.

Which is where it got interesting, because the harness couldn’t have seen it.

A cache that can’t see the change

Picks are cached, so re-running a test set doesn’t pay for the same answer twice. The cache key was the line plus a fingerprint of the prompt text. The field order isn’t in the prompt text. It’s in the tool definition, which wasn’t part of the fingerprint.

So if I’d shipped the reorder and re-run the harness, every line would have come straight back out of the cache, old answers and all, and the report would have said “no change”. It would have said that whether the fix worked or not. Measuring a change with a cache that can’t see the change is a very efficient way to learn nothing.

The fingerprint now covers the tool definition too, with one catch: it must not sort the keys first. Sorting would make the two orders look identical, which is precisely the thing I’m trying to tell apart.

There was a second trap in the harness itself. It assumed one stored answer per line. That’s true right up until you run two versions against the same test set, which is the entire point of a comparison. Left alone, it would have double-counted every line. Both got fixed before any money was spent.

The result

Both versions, run fresh against the same graded lines, cost $3.34 all up.

• The U-bolt line: answer first got it right 0 times in 8. Reason first, 7 in 8.
• Consistency: run each line several times, and answer first gave the same pick every time on 33 of 36 lines. Reason first did it on all 36. A reviewer who reopens an order now sees the same pick they saw the first time.
• The headline score barely moved: 28 of 32 against 27. And I measured the noise too: the unchanged version scored 28 from stored answers and 27 when run fresh. So one line is the noise floor, and I’m not claiming it.

The shaded strip is the harness’s own noise, measured by running the unchanged version twice. The headline move sits inside it.

The part I like most is where the picks changed. Only two lines did, and both were lines the catalogue couldn’t actually satisfy. On one, a customer asked for a 5mm × 45mm dowel pin, which the catalogue doesn’t have. Answer first offered a 5 × 30, a wrong size delivered with confidence. Reason first said no match. Given room to think first, it said “I don’t know” instead of guessing, which, if you read the first post, is the whole philosophy.

It went live on 19 August.

While I was in there

The same feedback turned up a second problem. When a U-bolt pick was wrong, the alternatives shown underneath were close to random, because the code that reads part numbers tripped over the way U-bolt codes are written. With 71 U-bolts that differ only by a size, “close to random” meant the right one was rarely on screen.

The first suggestion on the table was to show 25 to 50 alternatives instead of five. I said no. Nobody at a sales desk wants to scroll through fifty near-identical U-bolts. The fix was a better five, not more of them. The sizes now come from the item names, which spell them out.

What I’d take from it

• The order of the fields in structured output is part of the prompt. If you want reasoning, ask for it before the answer, not after.
• Before you measure a change, check that your cache can see it.
• Measure the noise. Otherwise a one-line “win” is just the harness having a good day.