Fresh off a week at ACFE Global in Boston (which contained significantly more games of Uno than you’d expect), Hailey Windham sat down with Chen Zamir (Head of Fraud Strategy), Tom Todd (Applied AI Lead), and Ryan McCormack (Director of Engineering) for a deep dive into agentic fraud ops. These Sardines covered everything from a client's ATO attack that was using its own blocks as a mutation signal to the infrastructure gaps that only became visible once deployment had already been automated.
We highly recommend checking out the full webinar for yourself but in the meantime, here’s a quick rundown of what we found most interesting.
Machine-speed fraud attacks: An ATO attack that learned from every block
Chen opened with a story from a client dealing with account takeover. Every time the client adjusted a rule, the attack came back slightly different, and the turnaround was fast enough that the team started looking at the timing.
"Whenever the attack was blocked, the denial was actually a signal that the attacker used to mutate and change their behavior. We literally saw these changes happening in milliseconds, and clearly that wasn't a human behind it."
He was quick to say this is still a small minority of what we see. But the direction is clear, and he was perfectly candid: "I wish I had good news, just buy Sardine and integrate us, and that's it. It's never like that." Getting a fraud program to machine speed means rethinking the org and the processes, not just the tooling.
The reaction cycle KPI: Where a week of reaction time actually hides
If attacks are adapting in milliseconds, the question becomes which "speed" you're actually measuring. A rule fires in real time, but the process behind changing that rule (spotting the problem, scoping it, designing a fix, testing it, and finally getting it into production) can easily take a week. Chen calls this the reaction cycle, and he's written about it on our blog as the KPI most fraud teams should be watching.
Our engineering team had spent years compressing the deployment end of that cycle: a rule engine analysts deploy from directly, automated model pipelines, testing baked in. That work paid off and we'd do it again. But once deployment was fast, the overall cycle time barely moved, which meant the week was hiding somewhere earlier in the process.
"Really, the bottleneck we find, and it's a much more challenging problem, is at the seam of investigations and testing, maybe not necessarily deployment and detection."
Sometimes the data you need was never captured, or a vendor integration quietly stopped firing months ago and nobody noticed. Labels are another problem entirely: chargebacks take weeks, scam reports may never arrive, and your own blocking decisions censor the record because a blocked transaction generates no outcome to learn from. And then there's leakage, where a rule backtests well, ships, and falls apart in production. "Those problems can become quite large and problematic and hard to reverse and even detect in some cases."
Fraud investigation at Scale: Device fingerprinting and the 150,000-account ring
Tom showed what the other side of this looks like when the infrastructure is in place. Agents running pattern hunts in the background, an analyst opening their day to a proposed rule with the reasoning already attached, the ability to chat with the agent and push back before anything ships.
Then he walked through the investigation we've written about before: a client flagged some suspicious signups, we pulled the shared device fingerprint, noticed the IP said US while the users were actually in Germany and the UAE, and fed the whole thing to our data analyst agent. The agent mapped the connected accounts and surfaced the common signals, and the scope kept growing. 150,000 accounts, one coordinated ring, and the investigation from first flag to full picture took minutes.
The clip below covers Tom walking through it step by step.
Four pitfalls of AI agent deployment in fraud operations
That investigation worked because the data infrastructure behind it was already in place. We learned what happens when it isn't early in our own agentic work.
We handed an agent our device ID field without explaining that device IDs rely on clearable browser storage, that people regularly clear their cookies and some browsers do it automatically. The agent had no way to know that: "It was constantly flagging, this is a bad actor, they have 10 devices, this is a bad actor, they have 10 devices." One field, one missing piece of context. Scale that to 10,000 features without attached semantics and the agent is producing noise at volume.
LLMs also arrive with built-in hunches about which countries look suspicious, and a business operating globally has to override those with evidence from its own data rather than defaulting to the model's priors.
Human analyst decisions sound like a solid accuracy baseline until you actually audit how consistent those decisions are from case to case and against the SOP. We couldn't hit the accuracy targets we wanted until we curated the baseline itself.
And giving an agent unscoped access to your full knowledge base sounds efficient, but the knowledge base contains assumptions that used to be true and quietly aren't anymore, and those steer the agent's recommendations without anyone realizing it.
AI agent governance: Skill, trust, and sequence
Chen laid out a rollout framework during the webinar that's worth expanding on here.
Your team's AI proficiency takes real time to build and can't be shortcut. Rollout timelines need to budget for that ramp, not just for the technical integration. Chen said he knows for a fact he's a better AI user than he was a year ago, and that learning curve applies to every analyst on a fraud team too.
The headcount question came up, as it always does. Teams adopt AI and headcount stays flat. That's not necessarily a failure, but it does suggest a trust gap. The fix isn't mandating adoption. It's transparency: clear goals, visible monitoring, reporting that shows what agents are actually doing. Without that, teams keep doing the work manually alongside the agent instead of shifting their effort.
Chen's argument is that agents get governed through data and monitoring, which is analytics work, and most fraud orgs don't have those functions staffed for it yet. Those roles need to grow before anyone talks about shrinking operations.
The order you build in matters too. Deploying agents to write rules faster doesn't help much if your labels are stale, because the rules are training on bad ground truth. If you can't label events in near-real time first, faster rule-writing agents are building on a weak foundation.
Where to start: A fraud ops audit framework
Hailey asked each panelist for one thing to take back to your desk.
Tom's was the most foundational: before you buy or build anything, document your fraud ops SOPs across every task your team does. When you review chargebacks, what are the actual steps? Where does the most time go? Clients come to Sardine wanting AI agents before they've mapped what they'd want the agent to do, and that mapping is free.
Ryan's was about what comes after: decide how you'd evaluate and audit the system before you build it. How would you judge whether it's working? How would you catch issues? That audit capacity is easier to design from the start than to bolt on later.
Chen's was simple: Your data environment has to be ready. Assuming your data is already ordered and ready to be consumed by AI is almost always wrong if it's an assumption rather than something you've verified. And on top of that, assuming agents can handle fragmented data on their own is equally dangerous.
The full recording is available on demand, and the whitepaper the panel kept referencing, Chen's recommended "light reading before bedtime," is here.
