How to prove an AI agent paid the right party the right amount.

An agent that can spend money is a different thing to test than an agent that writes text. When a chatbot gets a question wrong, a person reads the answer and moves on. When an agent pays the wrong party, pays twice, or pays more than it was allowed to, the money has already gone. The question a bank, a payments company or a finance team needs answered before it lets an agent near an account is not whether the agent sounds capable. It is whether, across many realistic attempts, the agent moved money exactly as it was told to.

AI agent payment testing answers that question by running an agent through tasks that require it to pay for something, and then checking the payments themselves. Five things are checked on every task: whether the agent paid the right party, whether it paid the right amount, whether it avoided sellers that were trying to deceive it, whether it stayed within its budget and its instructions, and whether what it reported afterwards matches what actually happened.

Most agent evaluations are graded by another language model reading a transcript. That is a reasonable way to score an essay. It is a poor way to score a payment, because a transcript records what the agent said it did, and an agent can be wrong about that, or can describe a failed task as a success. A second model reading the same transcript inherits the same blind spot.

The ledger does not have that problem. Every payment leaves a record of who received it, how much was sent and when. Grading against that record turns most of the scoring into arithmetic: the payee either matches the instruction or it does not, the amount either matches or it does not, the total either stays inside the budget or it does not. Where a judgement is genuinely needed, such as whether the item bought was the item asked for, it is checked by code against the task's own definition, not by asking a model for its opinion.

This also makes a result reproducible. Run the same agent on the same task again and the grader reaches the same verdict, because it is reading the same kind of record. A score that changes when a different judge model is used is a score of the judge.

The payee. Did the money go to the party the task named, and not to a lookalike, a stale address from earlier in the conversation, or a seller the agent found on its own? This is covered in depth on the wrong-payee page.

The amount and the count. Did the agent pay exactly what was owed, once? Retrying a call that seemed to fail is one of the most common ways an agent pays twice, because the first attempt often did go through.

The seller. When the task puts a deceptive seller in front of the agent, does the agent notice before paying, or after? Scam resistance has its own page.

The limits. Did the agent stay inside its budget, its per-payment cap and the scope of what it was asked to buy? And when it could not complete the task within those limits, did it stop and say so? Spending limits and honest reporting share a page.

The report. When the agent says it is done, does its account of what it paid match the ledger? An agent that fails and reports success is more dangerous than one that fails visibly, because nobody goes looking.

A bank or a payments company deciding which outside agents to let act on a customer's account needs evidence that a given agent, in a given version, handles money correctly. That evidence has to be specific to the agent and the version, because an agent's behaviour changes every time its model, its prompts or its tools change.

A company building an agent that pays for things needs the same evidence in the other direction: something it can show a bank or a payments partner to be let in. And a team that runs agents internally, for procurement or for paying for data and tools, needs to know how often its agents overspend or pay twice before the invoice shows it.

We run a gateway where agents pay per call for models, tools and data, with real money settling on each paid call. That is where we learned how agents actually go wrong with money, and it is why our tests are built around the payment rather than the conversation. We use that experience to decide what to test; no customer's prompts, outputs or payment records go into a test.

A Verified Eval runs on your own cases: send us real tasks your agent performs, and we run them across the models and configurations you are considering. Where a task moves money, every payment is checked against the ledger on the five points above, and the report shows each failure with the record behind it, so your own team can confirm it rather than take our word for it. We add cases written to provoke each failure mode, so a clean result means the agent was tested where it is most likely to break.

Retest when the agent changes. A result belongs to one version of one agent; a new model, a new prompt or a new tool is a new agent as far as the money is concerned.