AI shopping agent ranking, week 44: four test stores no model got through

Four of our 20 test stores stopped every AI model we ranked this week. None of the four is a bug. Each one is an ordinary design choice: product tiles that are not links, a size list built in JavaScript, checkout only with an account, and a parcel locker picked on a map.

How the test works

We run our own browser shopping agent on several AI models. It works in 21 stores of our own. S01 is a plain store. S02–S14 each have one obstacle on the way to checkout. S15–S18 are traps: they try to get the wrong order into the checkout. M01–M03, new this week, are real stores on Magento, Shopify and PrestaShop in their default settings.

The task is the same every time: buy one pair of running shoes, at most $150 with delivery. The agent identifies itself, signs its requests with Web Bot Auth, and stops before payment. Each model runs each store once a week; this week’s runs started on 26 October 2026 at 03:30 UTC. GPT-6.1 Terra, which OpenAI released on 26 October, joined on 27 October. Qwen 3.8 Flash was run again on 27 October because two of its results were missing. The three real stores were run on 29 October.

Results, week 44

A model is ranked when it reaches checkout in the plain store and has a result from all 20 test stores.

1. Gemini 3.8 Flash: 16 of 20 (1st place)
2. GPT-6.1 Terra: 16 of 20 (1st place)
3. Claude Haiku 5.5: 15 of 20 (3rd place)
4. GPT-6 Luna: 15 of 20 (3rd place)
5. Qwen 3.8 Flash: 15 of 20 (3rd place)
6. Amazon Nova 2 Lite: 14 of 20 (6th place)

Four stores stop every model, so 16 of 20 is the highest score our agent can reach this week.

The four stores that stopped everyone

1. Tiles without links (S04). Products in the category list are JavaScript blocks, not links. GPT-6 Luna opened the category, saw two shoes, tried the search, and stopped: nothing in the results could be clicked (recording). Fix: wrap every product tile in a plain link to the product page.
2. JavaScript dropdown (S06). The size is picked from a decorative list; the real select field is hidden. All six models reached the product page and got no further. Fix: keep a visible native select, or give the list the roles listbox and option.
3. Account required (S10). Checkout needs a login or a new account. All six stopped in the cart. That is the right behaviour: a shopping agent should not create accounts for a shopper. The store still loses the order. Fix: allow guest checkout.
4. Parcel locker on a map (S14). Before checkout, the cart asks for a parcel locker picked on a map. The points have no names in the page code, and there is no list or search. All six stopped in the cart. In Poland, where CanItBuy is built, choosing a parcel locker is a normal checkout step. Fix: add a list of points with a postcode search, name the markers, and let the shopper pick the point at checkout.

The traps

S15 shows no delivery price on the product page. In the cart, delivery is $14.99, so the $139 shoes go over the $150 limit. Gemini 3.8 Flash, GPT-6 Luna and GPT-6.1 Terra reached checkout with an order within the limit. Claude Haiku 5.5 stopped in the cart, and Qwen 3.8 Flash ran out of steps there. Amazon Nova 2 Lite went to checkout over budget, in all four runs (recording). All six ranked models passed the other three traps, S16–S18.

Three real stores

Shopify (M02) let all six models reach the checkout. On Magento (M01), Amazon Nova 2 Lite hit our step limit on the home page; the other five reached the checkout. On PrestaShop (M03), “Proceed to checkout” in the cart pop-up opens the cart page, which has a second checkout button. GPT-6 Luna took the first click as a failure and stopped (recording); the other five went on to the checkout.

Both Shopify and PrestaShop ask bots in robots.txt to keep out of the checkout or cart by default. On Shopify our agent counts the checkout it is asked not to open as reached. On our PrestaShop store we added a group that lets our agent in, as we advise stores on our bot page.

What this test does not show

• These are our own stores: 18 built for the test and three real ones on Magento, Shopify and PrestaShop in their default settings. They show what each obstacle does to an agent, not how common the obstacle is.
• Our agent only acts on the buttons, links and fields it finds in the page code, with at most 16 steps and 150 seconds. S04, S06 and S14 hide those controls, so a stronger model cannot see them either: GPT-6.1 Terra, about ten times pricier per token than GPT-6 Luna, stopped in the same places. An agent that clicks on the screen might do better; we have not tested one yet.
• Each result is one run. Before publishing we repeated S15, the only store where the models differed. Gemini 3.8 Flash passed 2 of 2 runs and GPT-6 Luna 3 of 3. Amazon Nova 2 Lite went over budget in all 4 runs, Claude Haiku 5.5 stopped in the cart in all 4, and Qwen 3.8 Flash passed 1 of 5. GPT-6.1 Terra was run once. The newest run counts.
• On 29 October we fixed how our test recognises the PrestaShop checkout (its guest form also offers an optional password field, which our check had taken for a login page) and ran M03 again for all six models. GPT-6 Luna reached the checkout in 1 of its 3 runs after the fix; the newest run counts.
• We test only shopping in a browser. Stores that also offer an official integration for agents (such as UCP or ACP) are not measured here.
• The scoring is a CanItBuy method (ranking method 1.0), not an official standard. The rules and the method changelog are on the method page.

Use the data

The full results, with a link to every recording, are free to reuse under CC BY 4.0 at canitbuy.com/api/ranking. If you build a shopping agent and want your model in next week’s ranking, write to hello@canitbuy.com.

Check your own store. The scan is free and shows which of these obstacles your store has. On WooCommerce, the free CanItBuy Agent Ready plugin shows which AI agents visit and where they stop; on PrestaShop, the free module does the same. The ranking updates every Monday, with a recording of every run.