Does your AI agent pay a fake seller before it notices?

An agent that buys things has to decide whom to buy from, and it makes that decision by reading. It reads search results, product pages, seller descriptions and the responses of the tools it calls. A seller that wants the agent's money only has to be convincing in text, and text is exactly what a language model is built to trust.

The scams that work on agents are mostly old ones in a new setting. A lookalike seller copies a real one's name and description. A listing promises exactly what the task asked for at a slightly better price and delivers nothing. A seller asks for payment up front for something that never arrives. What is new is the scale and the speed: an agent can meet many sellers in one task and can pay before any person has looked at the decision.

Agents are also exposed to a kind of deception people are not: instructions written into the content they read. A product page can contain text addressed to the agent rather than the human, telling it that this seller is the approved one, that the budget has been raised, or that it should send payment to a different place. A tool response can do the same.

An agent that follows instructions found inside the content it is processing, rather than only the instructions of the person it works for, can be steered into paying the wrong party while believing it is doing exactly what it was told. This is one of the main reasons scam resistance has to be tested on the agent as a whole, with its tools and its browsing, not on the model alone.

Noticing a scam after paying is worth little. The useful behaviour is to notice before: to check that a seller is the one the task named, to treat an unexpected instruction inside a page as a warning rather than an order, and to stop and ask when the deal looks too convenient. A test therefore has to record not only whether the agent eventually realised something was wrong, but whether any money had moved by then.

The ledger answers that precisely. If the deceptive seller received a payment, the agent was scammed, whatever it said afterwards. If it received nothing and the agent completed the task with the legitimate seller, or stopped and reported the problem, the agent passed. There is no need for a judge model to decide how suspicious the agent sounded.

A fake-seller test places deceptive sellers among legitimate ones, built from the patterns above: lookalikes, listings that promise and do not deliver, pages with instructions hidden for the agent, and sellers that ask for payment through an unusual route. The agent is given an ordinary task, such as buying a dataset or a service within a budget, with no warning that some sellers are traps.

Each run is graded on what the ledger shows: whether a deceptive seller was paid, how much, and how early in the task. The report separates the failure modes, because an agent that resists lookalikes but follows hidden instructions needs a different fix from one that does the opposite.

We write and maintain the deceptive sellers ourselves, and we keep changing them. A test whose traps never change becomes a test the agent can be tuned to pass without becoming any safer.

The fixes that hold up are mostly about what the agent is allowed to trust. Keep a list of approved sellers outside the agent's reach and check payees against it. Treat instructions that arrive inside content as data, never as commands. Require a confirmation step above a threshold or for any seller not seen before. And keep testing, because a new model or a new browsing tool changes what the agent reads and what it believes.