Sardine named a Leader in The Forrester Wave™: Financial Crime Management Solutions, Q3 2026

Learn More

Fireside chat recap: What we learned putting TypeSafe AI’s Jev into our fraud decision model stack

Brittany Geronimo
Brittany Geronimo
9 min read
bg-image
bg-image
Isometric graphic by Sardine with floating digital windows labeled LM, LLM, RLCD, and RLHF surrounding text about putting a fast decision model into a fraud stack.
Subscribe to newsletter
Share

When a customer swipes their card or signs up for an account, a fraud team has a few hundred milliseconds to decide whether to let them through. They've traditionally built a model fast enough for that call by training it on a large set of their own labeled fraud data, which plenty of companies starting out just don't have yet.

Our CEO and co-founder, Soups Ranjan, dug into that problem at a fireside chat we hosted with TypeSafe AI. He sat down with Erik Gafni, TypeSafe AI's CTO and co-founder, while Yoko Li, a partner at a16z Infra, moderated. The two of them spent the conversation on Jev, TypeSafe AI's System 1 model, and on what happened when we plugged it into Sardine Flow. TypeSafe AI borrowed the label from Daniel Kahneman's term for fast, intuitive thinking.

How TypeSafe AI's Jev model makes fast, calibrated AI decisions

Jev makes one kind of call very well, returning a fast, calibrated decision that slots into ordinary software as a building block. Erik's team designed it to compose with the rest of a codebase, so an application calls the model the way it would call any other function.

Any software engineer can now build a classifier on Jev in seconds. A few years ago, a company needed their own team of experienced machine learning engineers to fine-tune a BERT model on a mountain of labeled examples for the same result. They'd end up with something narrow that drifted whenever the data changed. Erik argues a classifier built on Jev generalizes further, so it keeps working when fraud patterns shift.

Using Jev for name-to-email matching and KYB decisions

Soups walked through two problems from our own evaluation of Jev, starting with name-to-email matching. He used his own address as an example, since soups@sardine.ai looks nothing like his full name. Rules engines tend to miss a match like that, while large language models (LLMs) only catch it after several seconds of reasoning. Jev got it right in one shot, quickly enough for a live payment flow.

A fraud team waiting two to four seconds on an LLM has already missed their window on a customer's card swipe or signup.

Soups then turned to know your business (KYB). In our KYB product, we pull from 20 to 30 data sources, including Google Maps and Yelp reviews, to work out who owns a business and what they actually do, down to their true industry classification. Customers used to hand-tune their own score that turned all that evidence into an onboard-or-decline decision. Our team spent plenty of hours helping them get it right.

Those customers can now hand that last step to Jev with no tuning at all. Jev also runs as a node in Sardine Flow, our workflow builder for regulated industries.

Automating sanctions screening starts with a written SOP

Soups argued that a company with a written standard operating procedure (SOP) rarely needs an AI agent to rediscover their decision boundaries. He'd rather see teams build their workflow in a tool like Sardine Flow, n8n, or LangChain and drop a decision model in at the steps that call for judgment. He used sanctions screening to show why, pointing to a prominent fintech that handles their sanctions alerts with a 10-page SOP. Soups would encode all ten pages in a workflow, which runs faster and costs less than agents improvising their way through.

Plenty of banks and fintechs never wrote theirs down. Their senior analysts carry the real procedure around in their heads. Erik asked whether a setup with no written SOP could ever run without a human reviewing every decision, and Soups pointed out how many companies run that way right now. Those teams have to write the SOP down before they can put a model to work on it.

Once they do, they can start automating a big share of their review work. Soups estimates that around 30% of a bank's workforce does manual review. Many of those analysts spend their days clearing false positives on sanctions lists. An analyst reviewing a customer named Osama, for example, has to rule out Osama bin Laden by checking the customer's date of birth, birthplace, and address before clearing the match.

We built an AI engine that encodes a bank's SOP for exactly this kind of work. At some banks, it now automates about 90% of sanctions reviews, and we've tracked 100% accuracy across six months of running it. Deloitte’s Center for Financial Services has tracked similar automation gains at tier-one banks deploying ML-driven sanctions review. We got there by putting a human on every case at first, then tapering review from 100% of cases down to 10% as the engine proved itself.

Training a fraud classifier without labeled data

In real deployments, we run into missing labels more than any other obstacle. If a client could tell Soups which of their onboarded businesses turned out to be fraudulent, he could train a classifier on that history. A client starting fresh has nothing like that to hand over. Soups would reach for a System 1 model there, since that client still has to make decisions and would rather not put their data scientists on weight-tuning duty.

An audience member asked how a team could catch fraudulent campaigns in aggregate without any labels. The panel described a feedback loop. Jev makes the first call. A human operator then reviews it with a thumbs-up or thumbs-down, which feeds back into retraining. Over time, the operator builds up the labels the client never had.

Why stateless models make fraud decisions consistent and explainable

Jev is stateless, so engineers pass it the full context on every call. Soups tested Jev on a real fraud ring event, where fraudsters ran several accounts on shared cards, addresses, and phone numbers. He gave Claude Opus and Jev the ring's graph and transaction history. Jev flagged every transaction in the sequence as fraud, a conclusion Opus never reached from the same data.

Soups was quick to point out that this was one test on one event. The result still lines up with what financial services teams need from a model. Banks want a cheap, fast answer they can reproduce the next time they ask, and Erik spent much of the chat on that reproducibility.

Erik pointed out that a small LLM asked the same question ten times will often flip its answer or sound far too sure of itself. He traces the problem to reinforcement learning from human feedback (RLHF), which rewards a model for giving answers people like. TypeSafe AI trains Jev for calibration with a method they call RLCD. Erik's team also dropped the chat interface, which takes away a lot of the pressure that pushes variance up.

Engineers building on a stateless model take on the context engineering themselves. Erik recommends breaking their system into small decisions that each run locally, with the software acting as a state machine around the model. He'd rather inspect those small decisions than read a long reasoning trace, which can even contradict the model's own final decision.

Erik pictures each decision as a circuit of micro-decisions. An engineer can inspect the question at each node, change a threshold in their code, and replay historical data to see how editing a single character shifts the probabilities. Erik used replay to answer an audience question about drift, too. Cardholders can file chargebacks weeks after a transaction, so engineers can't wait on that data to catch their model drifting. They can replay history against an adjusted node and check their change right away instead.

TypeSafe AI also trains Jev so the state and rules in the input win whenever a client's instructions conflict with what Jev picked up from the internet during training.

Catching AI agent fraud and fake accounts by the tells they leave

Soups argued that every adversary leaves a tell eventually. He pointed to the AI shopping agents behind agentic commerce, which read a merchant's page structure and click the exact center of each field without moving the mouse at all. Fraudsters who add mouse movement to their agents still produce motion without the natural variance of a human hand. Soups wants mimicking a human to get exponentially expensive for a bad actor. The ones who keep trying eventually slip and give away the tell. For agentic commerce, our No Human protocol verifies the human identity behind an AI shopping agent.

TypeSafe AI ran into fraud of their own at launch, when they offered free credits and fraudsters started creating fake accounts to claim them. Erik's team shut the offer off, since free access is exactly what someone needs to distill a frontier model into a copy of their own. We're watching fraudsters run the same play against other AI-native companies, driving a spike in token fraud.

Soups sees the same tactics in Sardine's own hiring, where 75% of applicants for AI and machine learning roles use fake profiles, VPNs, VOIP numbers, or deepfakes. Someone in the audience picked up on the recruiting angle and asked whether either company could have stopped the Binance hack, which started with a recruitment scheme and malware. Soups made clear that while Sardine doesn't do intrusion detection, we do offer candidate fraud verification, which tackles the fake-applicant side of the problem. Erik pointed out that engineers can now analyze their access and log files cheaply and at scale to find suspicious patterns.

Putting decision models to work in your fraud stack

If you're bringing a decision model like Jev into your fraud stack, start with your SOP and write down whatever still lives in your analysts' heads. From there, you can break each decision into small questions to inspect, tune, and replay against history. Jev fits best at the questions with no labels and a tight latency budget. Your reviewers can check every call at first, then step back as the model proves accurate.

Erik closed by pointing out how hard it is to run LLM agents reliably in the background millions of times. Even as someone who builds AI for a living, he'd decompose the problem into deterministic paths and hand the model as little of it as possible.

Watch the Sardine and TypeSafe AI fireside chat on demand

Watch the full conversation with Soups, Erik, and Yoko on demand here.

Frequently asked questions

What is a fraud decision model, and where does it fit in a fraud decision model stack?

A fraud decision model returns a fast, calibrated decision that an application can call like any other function. In a fraud decision model stack it handles the steps that call for judgment, while the rest of the workflow stays deterministic. A rules engine tends to miss matches such as name-to-email matching, which Jev handled quickly enough for a live payment flow.

Why do stateless fraud decisions stay consistent and explainable?

A stateless model keeps no memory between calls, and engineers pass it the full context each time, which puts context engineering in their hands. Erik Gafni, TypeSafe AI’s CTO, recommends building the surrounding software as a state machine of micro-decisions that each run locally. Teams can inspect each one and replay historical data after changing a threshold, and he prefers that to reading a long reasoning trace.

Should a written SOP or an AI agent handle sanctions screening?

Soups Ranjan argues that a team with a written standard operating procedure (SOP) already has its decision boundaries on paper and can encode the procedure in a workflow, adding a decision model only at the steps that call for judgment. He says that runs faster and costs less than agents improvising, and he used a fintech’s 10-page sanctions SOP as the example. Teams with no written SOP have to write the procedure down first.

What is a System 1 model for fraud, and why is Jev called one?

A System 1 model is built to make one kind of call quickly and return a calibrated decision that other software can use directly. TypeSafe AI borrowed the label from Daniel Kahneman’s term for fast, intuitive thinking. Gafni says a small LLM asked the same question repeatedly can flip its answer or sound overconfident, which he traces to reinforcement learning from human feedback (RLHF) rewarding answers people like, and TypeSafe AI trains Jev for calibration instead.

How do you build a fraud classifier without labeled data?

Missing labels are the obstacle Sardine meets most often in deployments, and a team with no confirmed fraud history can start with a System 1 model that makes the first call. A human operator then reviews each call with a thumbs up or thumbs down, and those judgments feed retraining. Over time they build the labels the team never had.