Boltの内幕：グレーゾーンで不正と闘う

Wow. Jaanus Uudmäe (0:06): You have somebody, you know, the CEO's friend gets blocked or uh or doesn't get a discount and then you get a very direct feedback that rules or not work. Let's agree that you will take this loss on yourself then and then there immediately start the stop. It's like okay explain to me again what will happen. Processers always manage to surprise you and come up with something different. So it's better to allow for some fraud to come in and then

What's up, everybody? Welcome to another episode of the Saturday Fraud Strategist. With me today, Jaanus Uudmäe from Bolt. Jaanus, welcome to the show. Jaanus Uudmäe (00:42): Hello, good to be here.

That's awesome. Pleasure to have you on the show. Jaanus, I've seen your post popping up on my LinkedIn feed for probably the better part of two or three years now. And you always have very interesting takes on fraud and also specifically on managing fraud at Bolt, which, if you guys are not familiar with Bolt, is on-demand marketplace, headquartered in Estonia. But Before we get there, Jaanus, maybe you can tell us a bit about yourself, about your career, what brought you to this to this role that you have right now at at Bolt. Jaanus Uudmäe (01:22): Sure. Like first of all, thank you for following me on LinkedIn. So that's always nice to hear that actually the stories that I write resonate with people. So that's good. But yeah, about myself. I got started in this fraud space already a long time ago, like 20 years ago, when in 2007 I joined Skype

Wow. Jaanus Uudmäe (01:43): In Estonia as a software developer. So straight out of college. I just graduated from University of Arkansas in the United States. And I came back to Estonia, I was looking for a job. And a good friend of mine, my old classmate, he was working at Skype, said it's a pretty cool place to join. And by accident, I ended up in the fraud team, not knowing anything about fraud. Like most people in this space, you don't really study anti-fraud or fraud fighting in the university, but yeah, I ended up as a developer of fraud tools in Skype. Very cool experience, work there. For several years doing the internal tools building and things related to that. And in the process, I got to know more about fraud and all the interesting and cool schemes. And over time, as the team grew, I became more involved on the product side of fraud and became the product manager there. And ended up spending 10 years in Skype. And then Skype was acquired by Microsoft at Microsoft as well. And then in 2017, I believe, yes, this company called Taxify, which later got renamed to Bolt, approached me and and said that hey, we kind of have a fraud problem as well popping up that would you be interested in joining? And I thought that yeah, it's interesting company, so so why not give it a try? Kind of restart of my career because I went from product management to kind of doing hands-on analytics. At the time. So we had just two people working on the fraud department at that point. So this is how I ended up at Bolt. And over time here, the team has grown. So right now I have 14 people reporting to me. And Bolt operates in like 50 plus countries. We have many different business lines. So we have the kind of taxi business, food delivery business, micromobility, which is scooters and e-bikes, then car rentals. And a business product so in different geographical regions, different product lines, mostly in Europe and Africa, but also Asia, South America, Canada. So we see a variety of different customers and different fosters. So life is very interesting here at Vault.

Yeah, yeah, yeah, yeah, I bet. It it was interesting because when when we had our first call, and I found out that he worked at Skype, I figured that at a certain point we were kind of like colleagues because I was working from 2009 at PayPal, and weirdly enough, if you know the history, PayPal and Skype separately were bought by eBay around about the same time. Which is very weird to think about it nowadays that these two companies ever lived under the same roof. But yeah, that's super cool. Bolt, I would say, I mean, for our American listeners, you cannot really compare it to any kind of single company in the US, right? Like in my mind it was always kind of like Uber, but it's not really like Uber, right? Or not only. Jaanus Uudmäe (04:56): It is. Yeah, well it is it yeah, well it's like Uber but better. Right. That's what we would like to think. But in many ways, actually, we are very similar to Uber and we operate in many of the same markets, right? We were not in the US, but Europe, Africa, we're on the same markets, especially on the ride hailing side, and then Uber has it's Uber Eats, and we have our own Bolt food. So a lot of them business lines are very similar, the products are similar. The services that we offer are similar and speaking to the Uber's fraud people, obviously fraudsters and the fraud schemes are very similar as well.

Yeah. So so let's talk a bit about that because I think that marketplaces are, you know, it's a known issue. And you know, when you look at the retailer side of marketplaces, obviously your attack or sorry, the surface of attack is is wider and there are all sorts of different schemes and let's say fraud patterns that can cause you damage. I do think that it actually gets worse when you're dealing with kind of like the on-demand economy economy marketplaces. So tell us a bit about that. Like I'm guessing that you're seeing like extremely weird stuff when it comes to fraud and to a lot of extent your definition of fraud. Is probably quite different than how a lot of fraud fighters who are listening to us right now would think about it. Jaanus Uudmäe (06:39): Uh sure. I think pretty much every company has its own challenges when it comes to fraud. It so much depends on what you're selling, what your what your services are. But for us, I think we see every every piece of it pretty much like that. The standard fraud that most companies that sell anything online deal with is this issue with chargebacks. So basically stolen credit cards that somebody uses to purchase services on your system. So this is a very standard thing. This is what kind of industry standard, what the Visa and MasterCard measure. We gotta make sure that we are below those fraud thresholds that they offer. So this is kind of yeah when you speak to other companies, other fraud fighters, everybody sees this part that how do you deal with those people who'd come with stolen cards and then try to purchase your services. But being on the marketplace we also have Decide that we don't really control the services and the and and what's offered. So we can have issues with drivers that are colluding with the riders, and we can have issues with potentially restaurants. So this bec creates a lot of complexity. And in food delivery, especially, that you have this three sided marketplace that you have the person ordering the food that we call the eater, and then you have the restaurant preparing the food, and you have the courier in play. So if and when something goes wrong. And there is some accusation of fraud, it's sometimes very hard to figure out actually what's going on. Is somebody colluding with somebody, or is it just a misunderstanding? So so we have those issues, and then also a lot of our attention nowadays goes to things which are not easily that measurable. So, for example, I think what what we call campaign fraud, or some companies call bonus uh

Yeah. Jaanus Uudmäe (08:32): Abuse or bonus fraud. Is obviously, you want to get new customers to the platform. So we offer quite good discounts for new customers. Obviously, some people like fraudsters they think that, well, why would why would I ever pay a full price for a service when if I create new accounts, I can just continue getting stuff at the discount? And so we're trying to prevent that. And that's very hard to fight against because you don't have a good way of understanding like was it now a good customer that used a discount and later just churned for whatever reason? Or was it the fraudster who who didn't ever intend to pay the full price? So this is this is one of the big challenges for us in an ongoing fight.

Yeah, I think especially the like the three party model is very interesting because just thinking about collusion from a single vector, which is like buyer and seller, you suddenly have three. You have buyer and seller, and you have seller and driver, and you have driver and buyer. And I'm just thinking how much just adding this that this third party adds to the surface of attack and what kind of very weird things probably you've seen along the years. And and I'm guessing that this is a lot of the times this is the challenge. Just the just managing the complexity and thinking about the like the fraud stories themselves and trying to stay ahead of the fraud stories especially. And I'm guessing that this is also an issue that Bolt has, what at least five different product lines, and you keep shipping more and new features. So also the product landscape keeps changing under your feet. Jaanus Uudmäe (10:25): Correct, yeah. So it never becomes boring, let's put it this way. Yeah, with all the new features and new services. And just to be clear, like most of our customers are good customers. So nothing wrong with them. So just that because we have so many of them, we see all sorts of weird patterns that sometimes are not even fraud, they're just issues with the platform, and then trying to distinguish between an issue and somebody intentionally trying to abuse your platform, that's difficult. Like let's take a simple example of refund abuse. So, what what I mean when I say refund abuse is that somebody intentionally lies to you about quality of services in order to get a refund. Just kind of initially pays for the service, but then comes back and says that they're not happy to to get the money back effectively. And it's this is very common in the delivery business because with food delivery it's very hard for us to prove what was the quality of the food, or did it arrive in in a nice way, or did the courier like mess it up and and shake the pizza in the package? And was it all a big big just chunk, right? So so there when we see a customer complaint coming in, like we have a lot of data about the customers, about the restaurants, the courier. We can see the route that they drove and all that all the times and everything, but we're not there to assess the quality of the food. So to understand whether somebody is telling the truth or not about some missing item, it's it's almost impossible. Like we don't have any hard evidence. So we have to look at other data to kind of understand like, could there be that somebody's lying? Like we can see that does the customer have an issue, like history of complaining a lot. Like every time they order, they they always say that something is missing. That's obviously a clear sign that they're not a good customer. Or does the restaurant have a history of not putting the items in the bag? Or does the courier always do something weird? So we have those data points. But it's not always so clear-cut because even for good customers, like stuff like this happens. Good restaurant as well, sometimes they forgot to add an item into the into the bag or they put a wrong wrong item in there. So you can't immediately say that based on one case like that somebody's at fault.

I'm guessing that it's also Jaanus Uudmäe (12:55): Issue. Yeah.

Yeah, I'm guessing it's also very difficult when it's the first transaction of a new buyer and you both don't have the history and also you don't know you know, because is it a good customer that is now in a real danger to churn because they had such a shitty experience? Or is it a fraudster that is clearly defrauding? Jaanus Uudmäe (13:21): Of course, yes. The first time customers, those are the hardest because if you already have history about the customer, it's quite straightforward in most of the cases. But the new ones, that's the trickiest. Because in the fraud team we don't want to you know, prevent hundred percent of the fraud. It's we know it's not possible. What we want is we want to grow the business but in a sustainable way so we don't lose too much money. But yeah, the first time customers it's always the trickiest because if they have a bad experience, that can mean that we lose them as a customer forever. So we want to make sure that we take good care of them, but at the same time we want to make sure that those are not fraudsters. So the way we approach it is that we look at the data. So if we on the fraud side can somehow prove that this customer is actually a fraudster. And the way we do it is that we can look at data items that we see, like for example, if they use the same email address then a previous customer who has a history of asking for a refund, we can quite safely say that, okay, those two accounts are linked. This is the same person just trying to abuse our refund policy. Or if you have some other data items which are hint strongly suggesting that this is the same customer, like IP addresses or or some device information, then we can say safely say that yeah, this is not a real customer or this is not the customer that we want, and they will they don't deserve a discount. But on if we can't prove it, then we just default to the standard store policy that Bolt has, like whatever it is in a particular market, what's the standard refund refund amount and and how many times you can get a refund. So this is the way we work it. But it's it's always very tricky. And we work together with the business teams as well to kind of show that, okay, this is what we see. Like this this is what a normal customer behaves, and we can show that if a customer, you know, three times in a row, they always complain about something. It's a sign that they're a bad they have bad intentions, that that they never just do a normal order and stop complaining.

Do you do you also w when you say that you're working about that you're working on it with the business team, what what does that actually entail when when you're outline your outlining your fraud policy and how you treat first time customers? Jaanus Uudmäe (15:46): Mm-hmm. So by business team I might mean mostly our customer service department who is in charge of like handling those flows and who owns the policy of like what's the okay if you complain that your pizza is gold, like what's the refund amount that you should get? So they own the defaults and then we work with them to show the data to also see that we don't just want to block everybody from getting a refund, but with the data we show that like we think that this is reasonable, explain those fraud scenarios at high level to them as well. And it's like a test for us. If we can prove to them that that makes sense, then it's good. Then we we can implement the policy. That does not mean that we we need to run every rule change by them. It's more at the high level that this is this is the type of fraud that we want to tackle. These are the kind of the data points, and this is this is the amount that we can kind of cover or protect for the company. 'Cause it also means that like if if that money goes out to the fraudsters as a refund, that's potentially a refund amount that some legitimate customer won't get.

Yeah, yeah, yeah. So but I think that's the interesting part, right? That you're working against a budget, but like a lost budget, and you're trying to make sure that you're spending this lost budget on the right folks. That makes sense. Tell me, I mean, this sounds obviously like a very hard problem. Uh and it sounds like like I wonder without having the evidence, and as you said, like it's very hard to know for sure what transpired. And it's definitely hard to know for sure if you decide not to give a refund or if you decide even not to accept a transaction in the first place. How do you how do you actually manage to automate and build rules or even models around these kind of problems? Jaanus Uudmäe (17:46): Good question. So I'll start with a simple case, like if we're talking about chargebacks, stolen cards. Uh for that, it's actually possible to measure it in a reasonable way. So there we we can run an A/B test to say that you know there are certain transactions that our system flags as high risk that we would normally stop or where we would do a 3DS challenge. So we can take a small percentage of those transactions and let them go on without any friction. And then we can measure what happens. So if that transaction results in a chargeback, then we know that our rule that triggered was correct. Because indeed there was a fraudulent signal in the end. So our rule was correct, and we can be happy about it that our system is working as normal. But if it turns out that we flag it as high risk, but if we allow the transaction to happen and there were then nothing bad happened, it was a good case, then we know that it's a false positive. And that's that's the F-word that in the fraud team everybody is very concerned about false positives, right?

Yeah, yeah, yeah. Jaanus Uudmäe (19:15): But but that works really well for for labeled data. So in in in this in the case of chargebacks, the data is labeled. You know what is good, what is bad, and you can target it. But now let's say, yeah, that that's the that's the easy part. But even there, the it's it's it's not super easy. There are complications there as well, but it's relatively easy. But then if you're talking about, let's say, this campaign campaign abuse and like which customers should get the campaign codes and which which accounts are basically created in order to just milk that money. It's a much harder problem because nobody comes back to you and says that, you know, I was able to trick you and and get this discount for for a hundred different accounts that I created. So so the data is not labeled. So it is our job in the fraud team to somehow put rules in place to label it so we can see that somehow say that these are this is the money that went to fraudsters and this is for good customers. The simplistic, you know, approach that we can take and where we can look at the total opportunities. Like let's look at all the customers where as a first time buyer they get some discount, let's say 30% off. And then after that discount period, did they make any transactions at normal rate. And you can say that if they made any transactions with a full price, you can say that these are actually good customers. That's one way of doing it. And if they stop making any transactions, does it mean that they're fraudsters? No, maybe there's just customers who who wanted to try the product, but for some reason didn't want to pay the full price for it. That doesn't mean that they're fraudsters, it's just customers who try it. It's it's quite normal. Right. So so it's up up to us to kind of label the data and say, like, at which point do we draw the line and say that this is fraud, this is not. And to make it even more interesting, that we have seen fraudsters who are kind of call them opportunistic fraudsters. So what that means is that you kinda you behave as a normal customer under normal circumstances, but then let's say your friend tells you that, there is a way to get some money back from Bolt or a little trick to to get a discount, which is not completely okay by the rules, but you know, this is how you do it, then they will try it. So this customer they will try to cheat the system a bit. So if we allow them to do it, they will continue doing that. And we as a company lose money. But if our rules step in and stop that behavior, like stop that refund or don't give them the discount code, then this this customer see that the hack doesn't work basically. So I will just go back to using the product as a good customer. So does it mean that this people are now fraudsters or not. They're more like on this gray area. So building any A/B tests is very tricky for them because they can behave as fraudsters or as good customers. So when you're trying to measure it in an A/B test like are they good or bad, it completely like messes up your data.

Yeah, and even if they cause you loss on a single transaction, it might be that they are still net positive when you look at their lifetime value. But I I want to ask you a different question because what you describe is I think definitely the right approach, which means in the end, you need to manufacture your own labels. There is no way around it if you want to to do automation. But the risk in doing that is that it's very easy to be tempted, let's say it like that, or to fall into the trap of recursive logic. Meaning I label trans accounts by saying, Yeah, they didn't transact for a full price after their discount period, and then and this creates my labels, and then I write a rule that you know, like uses somehow the same logic to make decisions. And if my rule logic and my labor logic is the same. I mean, obviously I will have good results, but it can also be that I was wrong in my assumption. And even though my manufactured numbers look good, I create a lot of false positives. How do you make sure that you don't you know, like fall into these kind of like biases? Jaanus Uudmäe (23:46): Well, we do have guardrails for that. Like we measure, for example, customer service tickets related to that issue. So obviously customers get good customers get very angry when they don't get the refund when they rightfully deserve one right. So we this is kinda proxy that we see that if the reef if the customer service ticket amount goes up, that maybe something is wrong. It might be that the fraudsters are actually being very vocal about it as well. But at least it's it's a sign that hey, we should look more deeper into it. Like maybe there is something in it. And then, like the in the worst case or in the most interesting case, it's you have somebody, you know, the CEO's friend gets blocked or or doesn't get a discount, and then you get a very direct feedback that, hey, your rules are good, rules are not working, and then the then you need to explain yourself. So that's a proxy way, right? But it it is very tricky, especially when you create your own labels. Like how do you know that it's it's yeah, like you said, it's a trap that you can say that I'm right because I blocked it and I blocked it because I'm right. And they automatically say that that this is the way. But for that we we we are playing it actually quite conservatively in our company. I think that usually when we don't give a discount or say that somebody is behaving as a abuser or fraudster. I want it to be in a very clear way that I can explain it to anybody in the company or even in my family. Like this is the reason. Like these are these are very s that's very strong evidence that this is not a good customer with not good intentions. But yeah, even then, mistakes do happen. And we're always balancing this side that you can see that maybe we're blocking too much in some markets, then we scale back the rules a bit. And there is yeah, there is no right or wrong answer. There there is no like state where you can say now the rules are ready, they're perfect, you you cannot make them better. And it's always a fluctuation that we see that you can optimize the rules first to stop the losses. Usually that's how we do it. Like when you're bleeding a lot of money, you need to stop the bleeding. And then once the once the situation has stabilized, then you start looking at okay, where are my false positives here? How can I make the rules more accurate to allow more good customers to go through? And And yeah, but but it it's a proxy way of CS, just anecdotal evidence from from friends and family and just common sense.

And and these proxy labels or proxy actually counter labels, do you measure them against your rules or also against your label sources? I'm guessing like the labeling itself is not done manually. I'm guessing that you have kind of like batch processes that just go through whatever, through the entire data every night and and kind of like mark what you think is is bad. Do you measure like both only the rules, only the labels? How does it how does it happen? Jaanus Uudmäe (26:49): We at like automatically we measure it at the whole set, at the high level. Not not at the rule level, but yeah especially with those labels that we create ourselves, then it's it's more trickier. But then you see that it's the health of the system overall. If everything is fine, no reason to dig any deeper. But if you see that something is trending in a weird way, then you can manually kind of go deeper and see like well, well, which rules are triggering the most. Is it because of fraud? Is it because of maybe some data issue? Because yeah, like a rule triggering more than usual doesn't mean that you have all sudden all of a sudden false positives. It might be that there is pressure from fraudsters and you actually want this rule to trigger. So you need to look more into it to to get to get the data. And the problem is also that with those labels that the time it takes for the label to appear, we call it maturing, it can be quite long because you make a decision today about one transaction, but in order to assess whether that customer is a good or a bad customer, you need to wait for the next transaction, next order, and maybe a few more. And that can take in some cases weeks before you have sufficient data because some of our customers, you know, order multiple times per day, then you have quick labels. But a lot of them it's just you make an order a couple of times a month and it's then very like you you're waiting during that period and you don't know whether you make made the right or wrong decision. So there is also this inertia in changing the rules. That what you change today has an effect that you can observe maybe next month. [Ad Break (28:31): Hey folks, I want to take a quick break to speak about today's sponsor, me. If you're finding this useful, do me a small favor, like and subscribe. It really helps with the algorithm. And if you're not already on my newsletter, the Saturday fraud strategist, you should definitely check it out. Every Saturday, I break down fraud trends, real cases, and strategies from my own experience of building and operating fraud prevention systems. Whether you're new to the space or a season practitioner, I'm sure you'll find it interesting. Check the link in the description. Takes 2 seconds. You'll thank yourself next Saturday. Now, back to the video.]

I I wonder with the risk of, you know, like going on a tangent with AI, here I said the word, with AI, there like I see two I don't know if to call them risks here or enablers. Let's call it in enabling factors. One, we know that today to send, you know, a picture of a rotten burger or an upside down pizza is very, very easy. Yeah. It's like one prompt away from you. And secondly, like literally in the last like month or two, we have all of these agent assistants, right? The instincts, the muses. And now you also don't need to like you know interact with customer support yourself. You can just you know like manufacture with a prompt, your image, tell your assistant through WhatsApp to get a refund, and that's it. You're pretty much done with it. I wonder like how have you seen AI shape the kind of like first party fraud scene, both on the opportunistic side, if you can say, and also on the on the fraud side. Jaanus Uudmäe (30:20): Uh it's definitely yeah, like altering the pictures or coming up with with pictures is definitely a thing. But I would say that this is not so prominent. At least I have not seen it yet. Or maybe maybe they're so good that I've not seen it. But in some cases for us, like for fraudsters, they know the right kind of things to go after as well. So if you say that you have a missing item, you don't even need a fake photo because there is no Like we don't ask for a photo if the item is missing. You take a picture of nothing. So you don't even need to generate the picture. So that's that's the easier way. You don't burn any tokens. What they can do, yeah. Like you can write maybe a really good, sad story about like describing your problem as well. But to counter that on our side, we we have also like machine learning models looking at what what are you writing about like like what's the actual issue that you're describing and then we can assess from fraud side that does it make sense or not. And then we can look at the data. Like you can fake a picture, but it's usually it is harder to to fake all the other elements that we see in the background about the order. So we have to rely on that.

Do you think though it helps more the opportunistic I I don't want to call them fraudsters, but kind of like the the legitimate accounts that opportunistically commit refund fraud because it's not only relatively easy in their mind, but also it's a bit more far removed from them. It's like it's not them doing that, right? It's their agent doing that. I don't know if you've seen this effect. Jaanus Uudmäe (32:03): It could, I I've not seen like usually these agents and these more sophisticated schemes, these are for professional fraudsters. The opportunistic people are more like, hey, I heard my friend did it. So he or she got a discount, so why am I the stupid one, you know, paying the full price always? Like let me do it as well. But usually then people try to do it manually. And and yeah, they don't go to agents at at this stage yet. So maybe some do, but in the end for us the methodology of detecting it is still the same. Like and and to making sure that we we kinda manage the risk there.

Got it. Very interesting. I I wonder okay, so we talked a bit about the labeling part and how you how you can make decisions in this kind of like uncertain environment where it's plagued with first party fraud and also like three parties within this business model. I wonder how like how do you actually measure your effectiveness here? Because, you know, with chargebacks, again, it's very easy. I I get chargebacks, I write a rule, I get less chargebacks. And here, you know, I I don't know how much like I I would say differently. It probably you probably always have refunds. And I'm I would assume that, you know, like certain attacks don't really change your refund rate tremendously. I might be wrong here, but it's probably very hard for you to A get a sense of how how many refunds you actually saved on one hand. And on the other hand, I don't know if you if it's that easy to understand like what do the false positive cost you at the same time. Jaanus Uudmäe (34:06): Exactly. Yeah, yeah, you're absolutely right. It is a difficult problem. And talking to my industry peers as well, like there is there is no easy way to do it. Like yeah, like one way indeed, like to measure how much we have potentially saved, that's quite straightforward because we can say that you know, we need we deny the refund to X amount of cases and the refund value would have been you multiply it whatever whatever the amount is, like you get to a number. So this is potentially what you have saved for the company. But what you don't know indeed there is that the false positive aspect of how how many of those were now good customers that got really pissed off at you because they they got a bad service and then you denied them a refund potentially and you lose their business going forward, right? So this is the tricky part. And to measure that, like we discussed, it's to measure that accurately is is quite difficult. It's just because you always have refunds and you're gonna always have people who who can lie about it depending on their mood or whatnot. So it is a constant constant struggle, let's put it this way, that we kind of know, yeah, like how much we can prevent. And and then you rely on the stories of what somebody picks up manually that they I see a weird pattern of somebody requesting a lot of refunds, and for some reason our fraud rules don't pick it up. So we can look into that, like what's the pattern, and you make an assessment: does it look like fraud or not? But yeah, the false positives are even trickier because a lot of the customers, good customers, don't come and complain to you. So you don't even have this proxy of CS. And it therefore it's also very hard to set targets. Because w w what is your and how do you how do you kind of measure that your team is doing a good job or not? Because there are external factors, like the amount of refunds depends on quality of the restaurants, the quality of the couriers, maybe even the overall economic situation when the economy is not doing so well, people are more tempted, good people are more tempted to ask for a refund to game the system a bit. So there are external factors playing as well. So we kind of try to establish some sort of range where we say that okay, we we think that our refund fraud is at this level based on the data and what we can prove. So let's try to kind of make sure that we we roughly save that amount of money. And if it goes up, then we need to explain it or goes down, like what are we doing? And the false positives as well, like the the proxy metrics or the cardrails with CS tickets and and those cases are are there to keep us honest and and to keep us from making too many mistakes. But it's very hard to say like where is the sweet spot. How do you know that now the team is performing really well and are we doing better than last year, for example? Very hard to say.

So I I think I I mean I I would kind of expect this answer, but then my my next assumption would be that actually the biggest challenge is facing this kind of like downward pressure from leadership. Because if you cannot really say, we're doing great, leave us alone, and cousin of the CEO now got declined or banned or whatever, and there aren't really set goals. It's very easy for the business to kind of like say, look, we need to grow faster. We like we just entered this market. And you know, and and our numbers like are way worse than our other other markets. And and you, you know, you can be like, Yeah, because the market is way worse. What can I do? And fraudsters are early adopters. How do you how do you manage? Because I think this is like the biggest challenge. How do you manage your own business leadership around this uncertainty and around this inability to kind of like, you know, like give them a confident measure of how well you're doing. Jaanus Uudmäe (38:19): Mm-hmm. Very good question. Like first of all, I would I I think that our leadership is quite good and and and smart. So they understand the challenge. And I'm not saying it only because they pay my salary, but because I truly see that as well. That they they do understand the struggle. Like there is it's possible to explain it to them, so that's the starting point. And then the second what we have done is that we don't overreact or over control it when we enter a new market. So usually we go in with just two new products, new new areas with just a kind of reasonable set of controls that we know that need to be there for sure, like chargeback controls and things like this. So kind of go in quite light on the rules and then observe what the market actually does. It's it's good for two things. Like first it enables expansion because that's like when you go to a new market that's the key that we we want to get reasonable market share we want to grow as fast as possible and you can say that the fraud cost is just the cost that you are paying as part of the expansion cost. That we're willing to take a bit more losses. But then once you have the losses then it's much easier for for me to go and show that hey based on the data we can see that this is this is going on like guys you can see I can pull up some concrete example accounts, like this is the type of fraud that's happening. Like obviously, everybody understands that we we want to stop it. So then it's much easier for us to kind of prove that we need these rules and we will add some friction, we will do the things that we need to do. And smart people will also understand that ultimately. Unfortunately, that there will always be false positives as well. So so so then it, yeah, exactly. So I I remember back in the day when our Bolt food delivery business was just starting and we we didn't have really fraud for a long time, like chargebacks in that. And we had just expanded to Latvia and in Latvia we operated for a couple of months without any issues as well. And then we got hit by fraud. And and at one point I think the fraud rate was like five percent of the overall revenue. And that was with like reasonable fraud controls. It was it's not that we were running without any controls. We had quite okayish rules in place, but it was five percent of the revenue. So that's like wow. Very easy to explain to the local teams and to the management that what like why we need these these rules, these these endpoints, this data, like it was a no-brainer. But but yeah, before that, like even when you look back, like what rules could we have built to prevent this five percent loss? No matter how hard you think or how how many tokens you burn nowadays with AI to try to come up with the rules for the schemes, like fraudsters always managed to surprise you. And come up with something different. So it's better to allow for some fraud to come in and then actually deal with the actual problem, not just just some hypothetical thing and be wrong.

I I I love it because I think it touches two two points. One, which is I I like see as a recurrent theme in how you guys are working, is really to to to like plainly budget it. And I and I and I liked how you positioned going into a new market, you need to kind of like budget fraud as part of your expansion. I was always thinking about it, you you basically you buy data. I I think your expansion expansion tag price sorry price tag is is is even better. Secondly, I think like playing it cool, and just giving the business what it wants and let them from time to time to actually fall into fraud. Many times, even though it sounds very how should I say it, very Machiavellistic, but many times it is actually much easier. To manage both the business and fraud, then going out very, very with a very low risk appetite, with very high constraint, and with this kind of like, you know, this this fear that you might get a lot of fraud, that actually many times get the opposite, the opposite result. So yeah, I I. I I really love how you how you frame this, especially around market expansion. That was very interesting. Jaanus Uudmäe (43:15): That that it that is true indeed. And it of course we're we're being reasonable. Like when we see that there is a product that like is has so many loopholes, like then we strongly advise the business like guys, like look, like guys and girls, like this doesn't make sense. Like we know from experience that this is a bad idea, we should put some controls in place. Like we do strongly warn them. Very rarely do we like put the foot down and say like no way we're going with this, like what we say that okay, if we're going, like let's agree that you will take this loss on yourself, then.

Exactly Jaanus Uudmäe (43:52): And then they're immediately start that the stop is like, okay, explain to me again what will happen. Like before that, they're not even listening that much, but then it's like, okay, now the loss is on you. Then they will listen and and usually they're yeah, they they will understand. But we have had some cases like last year as well, where we indeed launched a product knowingly that that it it had some major issues and we were lacking fraud controls and we did get hit hard, harder than we even expected. And it was it was a good lesson for everybody in the company that hey, well these guys kinda warned us and and these things actually do happen that there there are frauds out there that will take advantage of the of the weaknesses that you built into your products.

Absolutely. What can I say? I've been there. I want to switch gear a bit. And you know, we we talked a lot about fraud and how we deal with fraud, especially first party fraud. I I wonder, you know, like going a bit more to the side of defenses and how you guys work. I think we we talked very little about it when we had our prep call. But it it does sound like your team A are relatively technical. I mean, in the end, you guys do very little manual review, right? Most of what you do is automation with rules, with models, with whatever. I wonder how how much you guys use AI. And and also in what ways you used AI. What's working, what's not so much working. Jaanus Uudmäe (45:32): Yeah, maybe before even talking about AI, a quick overview of our kind of fraud defenses or how it's built up built up is that yeah, like you said, it's the team is technical and we don't spend a lot of time on manual reviews. And main reason being that our average transaction value is relatively low. Like a ride can cost in an African market, just a few Euros. Maybe even in UK we're talking about 20, 30 euros. So to review one case at a time, it doesn't really make sense. There are certain aspects where we must do human reviews, like when we suspect that the driver is actually involved in the scheme, then by regulations that sort of decision needs to be done by a human. Or when we're talking about do we pay out some big amount to a driver or restaurant, then human is involved. But mostly the rules are based on data and analytics, that like plainly, like if there is a new scheme, especially what we have is a fraud analyst going to the database, pulling some data, trying to find the patterns and doing this sort of data extraction and data analysis, right? This is what we have done from day one since I joined Bolt. But on top of that, what we have built as a next layer as well is the kind of, I would call it the traditional machine learning. So, machine learning models that learn from labeled data, that you have scoring each new transaction, each new activity, and then making the decision of like do we allow it, do we step up the challenge or not? So, this is the kind of the automation and and machine learning. And it works really well in known for known fraud patterns. So, this machine learning is it's very accurate, false positives are low, it's super good. So, the way we have it right now is that machine learning covers most of the core fraud, and then the human rules are there to address new and emerging trends that the machine learning doesn't yet know about.

Exactly. Yes. Yeah. Jaanus Uudmäe (47:40): And it's doesn't have the training data. So the human can build a rule, deploy it really fast in a matter of you know hours if needed, make changes to stop the bleeding. So and then eventually machine learning will also like the model will be retrained, it will know about it and then hopefully we can get rid of the manual rule. So this is the traditional way. And now with AI, like where where does AI come in? AI definitely helps us in this analysis part that the human does. Because as I explained, we have so many different countries, so many product lines, so many features. Like there is even in when you think about it in our ride hailing product, when you when you ride like sure. In the beginning of the ride you get an estimated price. We can book that amount from your credit card, but then stuff can start happening while you're ride. You can have a destination change event. You can have a paid wait time fee. You can have just a route can change. And at the end of the trip you can decide to leave a tip and then it's up to us to decide like do we allow you to leave a tip? Do we allow you to add a destination change waypoint? Which changes the price drastically, like what do we do in those steps? And at the database level, all of those data points are stored in various tables and the architecture keeps changing because you know the the feature teams are optimizing for it, building like re-engineering the work. So to know all of this context is quite difficult. Even for a seasoned analyst who has worked on that field for a long time, there is always gotchas that oh, but for this payment method, the data is stored a little bit differently, and what does this thing actually mean? So for us, this conversational analytics is really a game changer there. That now we're storing all of this knowledge that used to be in some analyst head who was looking at it. We're storing it in context files. So all of this works that hey, if you have a destination change initiated by the driver, then you should look at this data. And what does this GPS point actually mean? Is it the GPS point of the that you get from the rider app or the driver app? And all of these little details get that can make a difference. And when you're doing the analysis as a human, you might get it wrong or you might have a wrong assumption. But working with AI, then you store this information gets stored in a context file, and next time somebody runs an analysis, hopefully it pulls the right context, it it makes the right decision of how to interpret this data. And then as an analyst, the beauty is that when you discover that, actually the assumption was wrong that I I know better, that actually you should also incorporate this information. Then you can just tell AI that hey, this is what you do, this is this is the stuff that you're missing, and it will go and update the context. So we're kind of building this knowledge base, this living documentation and kind of making it better and better. So it's kind of our collective brain at this point of how we understand the complex of data structures.

Yeah, I mean that that's super fascinating because I I was sure that you were going to go on a whole different direction. Because usually when you're speaking about AI in the context of data analysis, you're speaking about AI doing the data analysis for you. Which is in a technical team, A, you would say, okay, how much time does it really save you? And B, you would also say, Okay, there there are enough guardrails because it's a technical team and they know SQL and they understand the quirks and they understand like where the AI can can go wrong and so on. But what I got from you is something else completely AI is actually how you use it is to create this like context layer, which might be missing for the analyst. We and I find this statement very interesting because usually we say that it is AI that is missing context. That you know, if we give AI a data set and it only has this data set and it has no business context, then you know it can make decisions that might look good on the data, but might not have like the the business sense behind them. And I really love that you expose that that has nothing to do with AI. That is just you know poor data analysis in general. And and looking just at the data without bringing it in context is is is just you know poor poor analytics. And it's very interesting that the way that you approach like solving that is with AI. That's the first time I ever heard about the idea. How how how I'm I'm wondering how do you see the effect of it? Like, like is it helpful? Is it successful? How do you know it's successful? Jaanus Uudmäe (52:43): Yeah, good question. Like, how do you measure the effect of it? Like subjectively, or what you can kind of measure is that how long it even takes somebody to write a certain script, or how much of their work day do they spend sitting behind a screen typing in SQL versus actually looking at the numbers and making a business decision? So right now, assuming that the concept context is relatively well described like this writing the SQL becomes quite trivial and automated. So that's already a big win. And it's for me personally, it's a huge win because when I started as a data analyst in this team, then yeah, I I knew the in and ins and outs of all the tables that were available. But as a team lead nowadays I have no idea. Like I just know the main tables, but I don't need to know them anymore. I can just go and ask and get quite good answers to my questions from AI. So it has enabled me to get stuff done on my own. And I'm I don't have to spend time and effort on like writing the correct SQL and debugging some SQL issues. Like that's done by AI, and I can focus already on the outcomes, on the numbers, on the next next part validating them, making sense of them. So that AI has definitely enabled it. And it all the dashboards that my team is building now. For ourselves as well, that we're moving away from this traditional BI-based dashboards and we're actually generating dashboards which are very specific to our product and features that enable us to look into okay, in this market, what are the trends? How many of the payments missing authorizations or we failed to capture, and how many of them are charge chargebacks? And you can easily then ask the next question. And hey, I've very helpful in like discovering these new opportunities. And it's what's also awesome is it allows us to quickly kind of expand outside of our sandbox. Because usually in our team and most fraud teams, kind of you optimize for fraud. So you optimize fraud losses and false positives. Because that that's what you know. That you can only as a human focus on that much. But now it's very easy for us to add to the context, for example, hey, what about payment costs? Because we operate with many payment services providers. And now it becomes a very interesting exercise that with some providers you have certain features available on the payment side, which enables you to tackle fraud better. So you can get the fraud rates down, but at the same time, just the payment processing cost is a bit higher on those providers. And you can with AI it's quite trivial to add that into the equation as well. You don't have to know the all the details about it. The payments team helps us to build that context, but we can put it together so we can see that maybe what we're building it helps to save on the fraud side, but actually on the payment side the cost goes up, so it's a net zero. So why why bother doing it? Or vice versa, that without sacrificing on the fraud rates, maybe we can change something and help to bring the payment cost down even further. So I think that this AI is not taking our job, but it's enabling us to be much more powerful and have a bigger effect for the company as a whole, because we can just look at so many things at the same time and and to make the right decision.

That's interesting. So, okay, so you guys do use AI to also write the queries themselves and also at the context layer, especially outside of your own team's context, which is interesting. How do you how do you see? I have an opinion about it. I want to hear yours. How do you think about non-technical team members that don't have SQL knowledge? Or are not expert on your data scheme. And having them work with AI for data analysis, where do you draw the line? Jaanus Uudmäe (57:09): So that needs to be managed carefully. That's it's a tricky part because I can see that like the fraud conversational analytics context that we have built is already very good. It keeps improving every day, but still it it makes mistakes. And sometimes the mistakes can have can result in really weird out outcomes in the analysis. So So you still need that human filter in the loop for kind of making this drastic or very deep drawing deep conclusions which will which will change the logic completely. Because usually when something looks really promising, you're missing something, or like a how can you like cut the fraud by fifty percent? It's it's probably unrealistic that you're missing some vital piece of information. So a person without any SQL knowledge, or worst of all no domain knowledge can take ask the AI some question and get results that look very good, but they have no way of validating them. So, therefore, yeah, having this ability to at least read SQL and to understand that ask, okay, what the SQL or how do you get the data? And then look at SQL to see that does it make sense at all? And then knowing your fraud domain and business domain overall as well, it helps you to validate because hallucinations do happen with AI. And sometimes it it's it's also very good at giving you the data in a way that sounds super convincing. That is like very well presented, uses very fancy words, backed up with statistics and data science, and it's very can be very misleading. It's it sounds very confident. And then at the same time, as soon as you push back, you're like, yes, you're absolutely right, it's the totally opposite. So the human at least today is needed. But at the same time, I can see that the co as the context gets better that we're building and putting the guard rails in place, the answers are more and more correct. But yeah, like I would not take the human out of the loop completely yet.

That's interesting. Yeah. Yeah, I think I think it's not like where I'm going with that is not whether we should or can take the human out of the loop, but really is the fact that AI can write SQL, does that mean that we don't need these skills anymore? We don't need like the analytical skills and AI can just take that. And you know we can give it to CS reps and they can run the the SQL queries. Or do you still need to have all of this knowledge and technically understand what's going on and also the mindset to sanity check and to challenge results that maybe look too promising? Which is kind of like, you know, if it doesn't happen to you every day, what are you doing as a fraud fighter, right? Or or as a fraud analyst? AI's job is not necessarily to come in and give you an ability that you didn't have before, but just to maybe just efficiency, just to save time, specifically in this use case. I'm not saying all use cases. Jaanus Uudmäe (01:00:32): I think that yeah, out of those, the analytical mindset, and the especially the fraud kind of skeptical analytical mindset is the key part of it. Because if I can, I don't know, draw some parallels that in early days of computing, you know, the programming happened at with ones and zeros. Very like down to the hardware, and then it was with assembler language also very close to the computer hardware. Like you really could control every registry and everything that was happening. It was super slow. And then you move to the higher level languages like C and C++ and and SQL itself, like that you're writing queries, it's also a higher-level language. You kind of describe what you want. Underneath, you still have you know bits and pieces moving on on a file system. So maybe this conversational analytics is just another layer that you're you're building on top of the SQL that that maybe writing SQL is kind of becoming a more niche thing and it's not absolutely necessary for an analyst. I think today it's still very good just to validate what you have, sanity check-in. But but I I can see from my speaking to my team as well. Like people are actually saying that they are losing this ability to write the SQL fast that since most of the s things are written by AI, that it does a good enough, very good job of writing the core of it that you don't need to go in and write it yourself manually. It becomes more of a hobby than that, I'm today I feel special, I'm gonna write something by hand. But analytical mindset is something that that adds the most value. Being able to ask the right question. And that's the difference between, anyway, a regular analyst or business analyst and a fraud analyst. Like you can take somebody who has really awesome SQL and Python skills and data visualization skills, but has no idea of fraud, and you put them in a fraud team. Initially, it will take them quite some time to come up with something meaningful. Even if you give them the task that they build with this, do this analysis, they are lacking the context and ability to kind of address the fraud. So it takes some time to get them up to speed. So yeah, the the analytical mindset is still very needed. This critical approach, challenging in the right places, and in our team as well. What what's super important is that sometimes your analysis, you do everything right, you come up with very good ideas. But in order to implement them, it's not just that you can write the fraud rule and implement it. You need sometimes you need the business to change something in their interface, something in the flow of somebody signing up to be a business user with us or a driver. And for that, you need to go talk to that team, need to present the data, convince them, have a conversation. Maybe they will push back really hard with valid points, then you need to go back and and a little bit adjust your system as well. So this analyst a lot of times are acting as product managers as well. That they're they're talking to other teams. And this is not just purely analytics and the AI agent or the conversational analytics agent today cannot do that because on the other side you all you have also humans.

Yeah. I I I agree. And I think, you know, in the end, the in fraud analytics, I think the difference between that and like classical analytics is that you're not only like crunching data and presenting graphs, right? You're releasing products, you're releasing models, you're releasing rules, like the solutions that you that you release, these are products. So I definitely agree with the product manager kind of like half-hat that that you need to wear. And I would wonder, can you actually develop this mindset, this DNA, without doing it, you know, to a degree with your own hands? And we still don't know, right? It will be a few years until the first data analyst that would grow grow up and be trained from scratch with AI, and and we'll see if it's possible or not. But I think today we don't know. Awesome. Jaanus, thank you very much for this. I I learned a lot. Super interesting. I want to kind of like go over a couple of the takeaways that I noted down as we were speaking. Because I think some of these things are super or can be super, super helpful for teams that encounter similar similar challenges. And I think a lot of the teams out there are encountering similar challenges, are encountering similar challenges.

I think one thing that we talked about is how you deal with on one hand making uncertain decisions that are very hard to argue for, even after the fact. And doing that especially around, let's call it business-sensitive populations. And I think we talked about two kinds. The first is very new customers or first-time customers, where you know, any any friction that you bring into the system can cost you a lot of lifetime revenue. And the second type of population that we talked about is a bit more kind of like high level, is basically going into a new market. And I think what I got from you is is that your team's approach to it, and and I fully agree with it, is actually two pronged. On one hand, you have a bit of a higher bar, a higher standard for actioning and for introducing friction. You talked about it several times when you talked about in the sense of proving that this account is a fraudulent account through history if you have it, through linking if you have it, through what whatever kind of like heuristics that are kind of like very, very safe and very high confident. But on the other hand, also to have some sort of budget that is negotiated, and maybe negotiated is a big word here, but negotiate between the fraud team and let's say the commercial team. The commercial team can be CS, the commercial team can be product, the commercial team can be, you know, the business unit of the new market you expand to. Where you're trying to get a feel, or actually not a feel, but an agreement of how much we're willing to lose here for this white glove white glove treatment that we're giving this population. And really to kind of like define it as part of the budget, whether this is like customer acquisition cost and part of that is acquiring fraudulent customers. Or whether this is part of your acquisition, sorry, not acquisition, your expansion, your market expansion cost. And having higher losses in a new market, that's that's part of expansion. So I think in general, switching from like a KPI slash goal-oriented measurement to a budget, an agreed budget, and starting to optimize in around optimizing your decisions around these kind of like budgets. This is definitely, I think, the right approach. I really love it. And I really like how you kind of like how this was really like the red line through a lot of the topics that we talked about. That was really, really good. And I think this is like proper, proper lesson that a lot of teams can can and should imitate. We also talked a bit, yeah. Jaanus Uudmäe (01:08:22): Maybe to clarify that well, we don't have like an official budget line for it. So so it's not like officially part of the budget that yeah, we'll go there and this is how much we spend on fraud. But that's just kind of mindset that we have that kind of unwritten agreement with the business teams as well, that especially new mark new markets or new feature launches initially will have higher fraud costs.

Yes. Yes, no, no, definitely, definitely. Jaanus Uudmäe (1:08:49): That that we will deal with a problem. Yeah, so we we're not stopping the business, we we let the business get started and then we optimize it later. So it's to kind of treat it treat it as a budget line without actually that being in the budget.

Yeah, or at least maybe hidden in the marketing cost or something like that, because you don't show your board. Yeah, anyways. Another thing that we talked about, which I really liked, it was in the context of first party fraud. But I I can guess that it can also apply to completely other vectors. By the way, we didn't talk about it today. I'm sure it's relevant also to both, but scams. Also an attack vector where labels are very you know uncertain and there's like underreporting and how you basically need as the fraud team to manufacture your own labels. And how in turn that can also lead to very adverse effects if you don't control it. If you go into this kind of like recursive logic of as you as you put it, you were right because you blocked, and you blocked because you're right, you were right. And I think here like I really like the approach that you gave around measuring not only your solutions, but also the labels themselves and making sure that on the base layer you're making the right decisions. Because in the end, if you haven't automated process that labels accounts for whatever reason, that is also a solution. Even if it doesn't act on on accounts or on transactions. So I think that that was also like a very interesting piece of of of of advice. And lastly, we talked about AI and specifically how you use AI for data analysis. I think I like like two things that you said. So yes, you're you're using it as an SQL monkey to save you time and to to make sure that your SQL is kind of like efficient and and you don't like you know forget the the right commas in the right places. But I think two things that you mentioned that were very interesting. One is to actually use AI to manage the context layer and how that not only helps AI itself to produce better results, but it helps the analysts, the humans to produce better results. Especially when you're working in a high change, high complexity environment like yours. Where the product always changes and because of that the data scheme always changes and there are you know there are multiple products and and so on. I think that bit was very, very interesting. And I think on top of that, and I'm guessing that is kind of like you know like a a subpart of of of of that point is that this also helps the fraud team to gain context from other teams. In your case, the payment team. And I think I need to do like a whole entire podcast on that and how like payment teams and fraud teams need to kind of like cooperate and how to how to optimize both of these verticals together. Because you gave the the right example. It might be that you you know you choose to go to a different vendor because you will have better results, but then that's some of it because of different interchange costs or whatever that is completely invisible to fraud is is the same or high. So I think also like getting the context of outside teams and understanding what's your real incremental value. Not how good this rule is, but really how does it affect the entire system and how does it affect the like bottom line profit margin? And using AI for that is, I think, really clever. I think both of these things are the first time that I heard about at least in how you use AI to to to to kind of like increase your not efficiency, but really productivity and the value that you bring. That was super interesting. So, Jaanus, thank you very much for all of these insights. Jaanus Uudmäe (01:13:20): It was a pleasure. Very interesting discussion, yes.

Of course, yes. Well, I I was assuming with a business like Bolt that will have a lot of interesting topics to to talk about. And yeah, I I I I was proven right once again. It's always it's always a good feeling. No, Jaanus, thank you very much. It was a pleasure having you on the show. Super interesting stuff. And for all of you guys, see you next Saturday. YouTube Description Inside Bolt: Fighting Fraud in the Gray Zone October 24, 2026 Episode Guest: Jaanus Uudmäe, Head of Fraud at Bolt For a couple of years now, Jaanus Uudmäe's LinkedIn posts have kept showing up in my feed, and they always carry a practical take on what fraud looks like at Bolt. Bolt is an on-demand marketplace headquartered in Estonia that operates in more than 50 countries, with ride hailing, food delivery, scooters, e-bikes, and car rentals. That means his team sees almost every kind of customer and almost every kind of abuse. Jaanus started as a software developer at Skype in 2007 and landed on the fraud team by accident. He joined Bolt in 2017 as a hands-on analyst when the fraud team had two people, and today he has 14 reporting to him. In this conversation, we spend most of our time in what I'd call the gray zone. Gray zone fraud is behavior where you can't tell whether you're looking at a fraudster, a frustrated customer, or an ordinary person who tried a trick a friend told them about. We talk about refund abuse and bonus abuse, why first time customers are the hardest to judge, and how to build labels when no one hands them to you. We also get into how to explain losses to the business when the numbers aren't clean, and how Jaanus's team uses AI in a way that surprised me. What you'll hear in this episode: ● How Jaanus went from a software developer at Skype to leading a 14 person fraud team at Bolt ● Why marketplace fraud prevention gets harder in a three sided marketplace with riders, drivers, restaurants, and couriers ● What gray zone first party fraud looks like in practice, from refund abuse to bonus abuse ● Why first time customer fraud is the hardest call, and how Bolt decides when to act and when to fall back on standard policy ● The difference between opportunistic vs professional fraud, and why opportunistic customers make A/B testing unreliable ● How fraud label manufacturing works when nobody hands you the answers, and how to avoid recursive fraud labeling ● Why label maturity matters, and which proxy metrics keep a team honest ● How Bolt thinks about fraud risk appetite and fraud expansion cost when entering a new market ● How AI works as a context layer for analysts, and why the fraud analyst mindset still matters You should listen to this episode if you: ● Deal with refund abuse or bonus abuse and can't prove intent ● Work at a marketplace or on-demand platform with more than two sides ● Are launching in a new market and need a way to talk about fraud losses with the business ● Create your own fraud labels and worry about fooling yourself ● Want to see how a technical fraud team uses AI for analysis without removing human judgment Episode notes & key takeaways: Gray zone fraud doesn't fit the chargeback playbook Chargebacks are the easier problem, because stolen cards leave a clear label and you can run an A/B test to see whether a rule was right. A marketplace adds more problems, since drivers can collude with riders and a food delivery order involves an eater, a restaurant, and a courier. The harder cases are bonus abuse and refund abuse, where someone lies about quality or a missing item and there is no hard evidence either way. Jaanus's team looks at the history of the customer, the restaurant, and the courier, and links accounts through email, IP address, or device. When they can't prove anything, they default to the standard policy for that market. Opportunistic customers make this harder, because they act like good customers until a friend shares a trick, and they go back to normal when a rule stops it. My takeaway is that in the gray zone, you don't need to catch everyone. You need to act where the evidence is strong and let policy handle the rest. Manufacturing labels without fooling yourself Nobody tells you when someone abused a discount. The fraud team has to create the labels. One simple version is to call a new customer good if they later transact at full price, though stopping after the discount doesn't prove fraud. I raised the risk of recursive logic, where the label and the rule share the same assumption. The results look right because they were built to. Jaanus's guardrails are customer service ticket volume as a proxy, anecdotal feedback such as a CEO's friend getting blocked, and a conservative bar, which means only blocking when he could explain the reason to anyone, even his family. He also monitors the system as a whole instead of rule by rule, and he accepts that labels mature slowly, sometimes over weeks. Measuring refund fraud is just as tricky, because denied refunds give you an estimate of savings while false positives stay hidden, since good customers rarely complain. His team works with a range and explains it when the number moves. If you manufacture the labels, you own their quality too. Treat fraud in a new market as part of the cost of expansion When Bolt enters a new market, it goes in light, with a reasonable set of controls like chargeback protection, and then watches what the market does. The losses become part of the expansion cost, and they also give Jaanus concrete accounts to show the business why more rules are needed. When Bolt Food started in Latvia, fraud reached about 5% of revenue even with decent controls, which made the conversation easy. His team warns strongly when a product has loopholes and rarely puts its foot down, but it asks the business to accept the loss if they go ahead anyway. There is no official budget line for this, only an unwritten agreement. I love that it swaps a performance target for a loss budget, and it reminds me that trying to predict every scheme in advance is a losing bet. Rules for what's new, models for what's known Bolt's fraud team is technical and does little manual review, because transaction values are low, a few euros for a ride in some African markets and 20 to 30 euros in the UK. Humans step in when regulations require it, for example when a driver may be involved, or when a large payout is at stake. Machine learning covers the core fraud patterns accurately, and analysts write rules for new trends that the model hasn't seen, sometimes deploying them within hours. Later the model is retrained and the manual rule can be retired. That is a healthy way to think about rules, as a bridge until something sturdier is ready. AI as the context layer for the analyst I expected Jaanus to tell me that AI writes his team's analysis. He described something different. His team uses AI to store knowledge in context files, like the difference between a GPS point from the rider app and one from the driver app. That kind of detail used to live only in an analyst's head. Writing SQL has become almost trivial. Jaanus now spends his time on the numbers and the decision. His team also adds context from other teams. Payment costs are one example. They show whether a change that reduces fraud simply raises processing fees. He still wants a human in the loop. AI can sound convincing while being wrong. Someone without SQL or fraud knowledge can't validate the answer. What matters most is the analytical, skeptical mindset, along with the ability to convince other teams to change a flow. AI gives the team more reach. It doesn't replace the judgment that makes someone a fraud analyst. Final takeaway: The most useful idea from this conversation is that uncertainty is something to plan for. You can't prove every case in the gray zone. Instead, you set a high bar for action, agree on how much loss the business can accept, and check your own labels as carefully as your rules. That makes the fraud team an easier partner for the business, because everyone knows the trade being made. Whether you run a marketplace or a payments product, the same approach applies. Be clear about what you can prove, honest about what you can't, and ready to explain both. Accordion dropdown close Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Jaanus Uudmäe | LinkedIn Head of Fraud at Bolt Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: Are You Using These 3 Fraud Rule Types? Meta Title: Fraud Rule Types: Strategy, Behavioral, and Trend Rules Meta Description: Not all fraud rules are built the same. Breaking down the three fraud rule types, who should own each one, and how often each needs to be monitored closely... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/fraud-rule-types Keywords Primary Keyword: fraud rule types Secondary Keywords: velocity rules fraud, threshold fraud rules, list based fraud rules, behavioral rules fraud, ML score fraud rules, compound fraud rules, network graph fraud rules, fraud rule deployment, fraud rule prioritization, fraud rule overfitting, blocklist fraud prevention, allowlist fraud prevention, how to write fraud rules, fraud rule lifecycle, device fingerprint fraud rules, fraud rule complexity, consortium fraud rules velocity rules fraud threshold fraud rules list based fraud rules behavioral rules fraud ML score fraud rules compound fraud rules network graph fraud rules fraud rule deployment fraud rule prioritization fraud rule overfitting blocklist fraud prevention allowlist fraud prevention how to write fraud rules fraud rule lifecycle device fingerprint fraud rules fraud rule complexity consortium fraud rules

We talk about fraud rules all the time, but what do we actually mean when we say rules? What are rules for? Who writes them? How should they be monitored? Here's the thing. During my career, I found out that there are actually three distinct categories of rules. But most teams treat them all the same. And what are these categories? Strategy rules, behavioral rules, and trend rules. Each designed differently, written by different teams, and with completely different roles to play in your fraud defenses. Mixing them up means you'll either over monitor some while under monitor others, assign ownership to the wrong teams, or worse, expect them to perform in ways they were never designed to. So, in this video, I'll break down each category and show you exactly how to handle them. Let's get into it.

Strategy rules segment your population and define the overall design of your fraud defense. Which segments get approved? Which segments get blocked? Which segments go through a step up like authentication or manual review? In most cases, these rules are written around the score calculated by a machine learning model, an AI agent, or if your organization is still living in the 80s, a scorecard that tells the result of other rules. A good strategy segments the population according to risk bands and ideally other general risk factors like payment method, transaction amount or user history. It usually looks something like this. If score is above 80 and the amount is higher than $100, then decline the payment. Keep in mind that strategy rules are usually interconnected, especially if you manage multiple segments. Changing a threshold in one rule would dictate mirroring changes in other rules in the same set. For example, if in that previous example I had a rule that sends to manual review payments with scores between 50 and 80 and I raise the upper limit to 85, I now need to change my decline rule we just saw to only start declining above 85. This also means you may want to measure these interconnected rules as a rule set and not only individually. Performance-wise as these are high level rules that operate on the entire population and mostly translate a score into a deterministic action. Accuracy tends to be quite low. However, this also means they're more resilient to degradation. Machine learning models tend to degrade much more slowly than rules. So you don't need to monitor them daily or weekly. A monthly cadence is usually more than enough. And from an ownership perspective, since strategy rules are tightly coupled to a model score distribution, it makes sense that they'd be designed, reviewed, and deployed as part of the model release process. Should a data science team write them? Perhaps. If it makes sense within your team's context. But even if fraud analysts write them, they should work very closely with the data science team to do so. And what if the model you use is supplied by a vendor? Then ideally you should request them to support you with the required data analysis and segmentation.

Behavioral rules detect typologies that bypass your fraud model and strategy rules. This can happen for a few reasons. That typology is new and didn't appear in the model's training set. It's too intricate or small to describe with a model. Or you're targeting false positive behaviors that models aren't really designed to catch. Regardless of the reason, the point of these rules is to describe suspicious, or legitimate behaviors, and not a particular fraud attack. And by doing so, your rules can catch different rings that have similar MOS without the need to constantly tweak them and overfeit them to one particular attack. A good example would be velocity rules. They describe a certain typology that can manifest in different attacks and by different fraudsters. Here's an example. If domain risk is high, and the device is new, and the IP country mismatches the account country, then block the account. Let's talk about performance for a second. Ideally, behavioral rules should be quite accurate. And in all cases more accurate than the strategy rules themselves. Otherwise, why bother with them? And while behavioral rules degrade faster than models, because they describe general typologies rather than specific patterns, they don't degrade that fast. Even if a fraud ring leaves your platform, the rule would still catch new fraudsters attacking you in similar ways. So from a monitoring perspective, you want to review your behavioral rules on a weekly or monthly cadence. And in terms of ownership, behavioral rules sit squarely with the fraud analytics team. As they require both deep domain expertise and data literacy.

Trend rules are designed to stop specific trends, attacks, or fraud rings. When nothing else works, when fraudsters bypass your model, strategy rules, and fancy behavioral rules, you need to bring the hammer down. These rules focus on a specific narrow pattern to maximize the effectiveness of your defenses. They usually look like this. If the email domain is tempemail24.com or gorilla.info or 10minute.net net, and the card bin is specific to this trend, and the shipping address has 1247 harbor somewhere in it, and the amount is higher than 50 bucks, then decline the payment. From a performance standpoint, trend rules tend to start off being very accurate as they're tailored to a specific attack. That means that on paper they should be your most accurate rules and by a wide margin, but they also degrade fast, very fast. It's often easy for fraudsters to tweak something in their behavior and bypass a trend rule. This means these rules should be monitored daily to weekly, especially if they're part of stopping an ongoing attack. And when it comes to the ownership question, I personally believe that trend rules should be within the tool set frauds teams have at their disposal when dealing with fraud attacks. However, you should consider your trend rules as a stop gap solution. The idea is they buy you time to deploy more substantial defenses that degrade more slowly, and that scale better. Allowing you to eventually get rid of them completely. Because if they are left unchecked over time they can really cause havoc by accommodating false positives.

Understand the differences between the three types of rules isn't just academic, it's operational. Because when you treat all rules the same, you waste resources monitoring things that don't need daily attention while missing degradation in rules that do. You assign ownership to the wrong teams and you harm your overall effectiveness by mismanaging the success criteria for each role category. But get this right and your fraud defense would be more resilient and easier to scale. Anyway, that's all for today. I'll see you next Saturday. YT thumbnail: Ep #26, Solo: 3 fraud rule types 1 · Wrong Thumbnail text: You manage / fraud rules [wrong] (navy on "wrong" only) Title: Fraud rule management: strategy, behavioral and trend rules need different owners and cadences 2 · Misbehave Thumbnail text: Why your fraud / rules [misbehave] (navy on "misbehave" only) Title: Fraud rule types: why rules underperform when you expect a job they weren't built for Episode Title: Scaling Agentic AI: Lessons From the Field Meta Title: Scaling Agentic AI: Lessons From the Field Meta Description: Scaling agentic AI means more than automation. A lesson from the field on how one risk team built agents that grew revenue while cutting loss rates... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/scaling-agentic-ai Keywords Primary Keyword: scaling agentic AI Secondary Keywords: agentic AI infrastructure, AI agent adoption fintech, build vs buy AI agents, residual risk AI agent, agentic AI model retraining, AI agent quality of life, centralized AI platform fintech, AI agent shared skills repository, non-engineer AI agent development ,AI agent false positive reduction, agentic AI efficient frontier, AI agent certainty measurement, AI agent explainability risk, how to scale AI agents in risk, AI agent deployment risk team, agentic AI underwriting optimization, AI fraud rule generation, agentic AI data infrastructure, AI agent hiccups production, out-of-time testing AI agents, AI agent personalization fintech, AI agent build prerequisites agentic AI infrastructure AI agent adoption fintech build vs buy AI agents residual risk AI agent agentic AI model retraining AI agent quality of life centralized AI platform fintech AI agent shared skills repository non-engineer AI agent development AI agent false positive reduction agentic AI efficient frontier AI agent certainty measurement AI agent explainability risk how to scale AI agents in risk AI agent deployment risk team agentic AI underwriting optimization AI fraud rule generation agentic AI data infrastructure AI agent hiccups production out-of-time testing AI agents AI agent personalization fintech AI agent build prerequisites

We have entirely rebuilt our risk systems to be geared around AI.

Did you guys actually downsize the size of the team?

That's not sort of been our philosophy around this. The quality of the life actually uh on the team is one thing that you know has improved so much. Are actually saving us hundreds of thousands of dollars in fraud losses. And I think that is actually you know 80% of the game, is to

Welcome everybody to another episode of The Saturday Fraud Strategist. With me today is Lalitha Rao. Uh Lalitha, you're the chief risk officer at Imprint, correct?

Yes, that's correct. Excited to be here and to join you today.

Yeah, the pleasure is all mine. Uh Lalitha, we had a call a couple of weeks ago. We talked a lot about agentic AI, which will be the topic of our conversation today. Got me very excited. We geeked over it for about 30 minutes. Today we have a bit longer than that. So, I'm very excited about our uh conversation today. Before we dive into it, maybe you want to share a bit about yourself and your background and what led you to to Imprint.

Yeah, absolutely. Uh I've been in the risk field now for about 16 years. Uh spent a bulk of my career at large banks like Capital One. And more recently at fintechs like Stripe and Square. Uh joined Imprint about uh now 19 months ago to lead their risk function. Uh at Imprint we are a loyalty and payment platform for brands to deepen engagement. We work with a number of large brands. Uh like H-E-B Booking.com, Crate & Barrel, and Rakuten just to name a few. And really my remit on the risk team is to enable safe and resilient growth. Uh to do that while staying within our risk appetite. And to create best-in-class member experiences to really deepening engagement for our partners.

Yeah. Well, and you have like uh very small partners with almost no fraud, right? That uh that's how it sounds like.

Fraud is definitely you know a a place that is evolving at uh such a rapid clip. And uh you know I'm excited that we have a really strong team and a really strong Data driven approach uh to continue to make strengthen our fraud system so we can really serve our customers our customers with the least amount of friction as they interact with our experience.

Before we deep dive into it, we had Brian Davis from Safeguard on the Pod a couple of weeks ago. And Brian and I talked a lot about his framework for uh the transformation journey, the AI transformation journey that he sees uh in mainly FinTech and I'm guessing banks as well. And I think that conversation was you know very theoretical. And very kind of like you know like high level framework doctrine methodology. And I think you know first of all if you haven't listened to it yet, go and check it out. But I think if you did listen to it and you liked it I think this conversation is exactly the next step. Because when we talk Lalitha um you know you your experience with rolling out agentic frameworks at Imprint is super interesting. The easiest way to start I'm guessing would be to just plainly ask what did you guys build? So today when you're looking at your kind of like agentic landscape what do you guys have in place?

Yeah, I was going to say I definitely tuned in for uh the podcast with uh Brian. Really enjoyed the episode. I think you know such a great framework uh that he shared. That you know especially for uh you know as uh different companies banks and fintechs are evaluating how you know they go on this journey. I think it was just like such a great session. Uh so uh you know excited to kind of um continue to build on that as well. Uh so uh you know I'm this is actually such a topic that's like just so near and dear to my heart. Because we have entirely rebuilt our risk systems to be geared around AI. Uh the day in the life of a risk analyst today looks so different from it from what it looked even 6 months ago. If you think about it, uh there's, you know, a lot of time spent on, you know, creating reporting, uh, aggregating data from different data sources, getting reporting together, creating, uh, reporting around SLAs's around ensuring that uh, we have the right governance and model validation documentation. And uh, now we've been able to just create so much operational efficiencies around a lot of that work. That it just frees up the team's time to do really strategic work. That's actually moving the needle uh for our members and our partners as well. Uh so it's been really exciting. Uh if you think about uh you know what we've built, uh I I want to also call out that you know one of the reasons we've been successful at uh you know really leveraging AI in a manner that is I think becoming a very durable and compounding advantage for us over time is because our you know chief technology officer and our AI team. Really created an operating platform form across the company. That enabled you know the teams to be able to use the AI in a manner, where you know they had access to standardized data. Uh they had access to standardized tools through an MCP connection. You you name it it was like snowflake, notion, uh you know linear, slack. Uh and as well as you know documentation around data lineage. So that when uh you your agent is looking for uh definitions and data, they know exactly where to go. And there is a canonical source of truth for where to find information. And I think that is actually you know 80% of the gam, is to have that platform and structure in place. And I think that really enabled, the you know different teams and the risk team to go deep on their domain main expertise. Uh so like I said, you know, we've really changed our workflows across credit, fraud, collections, disputes, uh you name it. And it looks very different today. Uh we've also built a suite of very specialized agents for our highest leverage work, uh that actually frankly are doing work that it would take teams of uh data scientists to do. Uh so excited to tell you more about that.

Yeah, please do. I mean, first of all, it's super interesting uh to just hear you echoing, you know, one of the main things that Brian said in in our conversation. That, you know, it starts with setting up the infrastructure and setting up the connections and the data flows. Um and it and it definitely sounds like, you know, this is not at Imprint at least that's not like a risk thing. That's technology across the board. Uh so super interesting. Um but yeah, I I would definitely love to hear more about what you've built. And understand like is it more around trying to streamline specific use cases or workflows? Or is it more about trying to just pick up the repetitive work and try to automate that away across the board? Like how did you how did you guys approach it?

Yeah. No, I think that's uh a really good question. And you know as you know when we started on this journey I thought you know that's where a bulk of the opportunity was. Uh which was you know being able to gain a lot of efficiencies, automating a lot of you know repeatable work that can be tedious. And we have definitely done that. But you know what we found is uh that the gains go so much beyond that. Uh and like I said we've built you know some very specialized agents, that are helping us do some of our highest leverage work. So I'll give you an example of this. We've built an agent that we uh have named Aria. This is our acquisition risk intelligence agent. Uh so think about Aria as an autonomous credit policy analyst. And what uh this agent enables us to do is it scour our portfolio, looks at performance based on honest data, but is also looking at data from the bureaus. To find early signals of deterioration, that have not yet manifested on our side in the form of delinquencies. And it comes up with recommendations for policy changes that uh bring us back to the most optimal point on our efficient frontier. And it runs this in parallel, across all of our partner programs. In the old world, you know, this would take an analyst, you know, several days to do for a single program. And now Aria runs these recommendations. Uh gives us a full audit trail. You know we are able to see the reasoning, the logic, we can see the source of truth in the form of the SQL scripts, the data sources. Uh understand you know the really the uh the reasoning behind the recommendation. And we surface it to a human who's then empowered to make the decision, on whether this is a you know a change that we should deploy or not. And it has dramatically improved the velocity with which we're able to rate on you know early signals that we are seeing in our portfolio. And continue to improve our underwriting. Uh in fact we've already seen the results from this in terms of being able to expand our approval rates, expand access to credit to more of our members. So like like I said, I think the biggest insight for me is just you know, uh the value here is in terms of like you know not just moving to a more optimal point on that efficient frontier. But really pushing that frontier out. And you know we've seen several examples of this. I think this is one example of that uh that uh you know that kind of work.

That that's super interesting. I I wonder you know what was when you look at Aria as a use case why was an AI agent the solution here and not for example I don't know um an SQL query, or some sort of a tool. Or some sort of machine learning model? Like how did an AI agent unlock potential that you couldn't unlock with other means?

Yeah I think what the agent here does is you know it has all the context of our uh of Imprint embedded and Imprint risk management embedded. And it can actually use all that context. So it actually un knows exactly what our current credit policy is. It has access to you know the data sources around performance as well. It understands you know what are our overall guard rails in terms of our risk appetite. What are our you know partner commitments as well. And it's able to run that optimization, uh and that tradeoff between uh you know risk and uh reward. Uh in a in a continuous manner, across all of our programs. I think the other reason this was especially impactful in our context is, you know we have brands across so many different verticals. You know we have very large partners in the grocery vertical. We also have very large partners in the travel vertical. And all of that means we have to customize our uh uh credit and fraud underwriting to the context of uh our members in those verticals. And given that you know Aria is actually able to do all these optimizations with all that unique context in parallel, and you know generate a report. Like I said we were we were doing the same things we were just doing them much slower. And it's allowed us to you know iterate so much more rapidly uh continue to increase the data sources we are using as well. And just drive better outcomes for our portfolio.

I think that you touched a very important point here and I'm trying to kind of like generalize it also to the audience. Because they are probably thinking about like very different use cases. And it goes back to something that you said before. That you know you you think about AI and you think first about automation. But then you find out that it's actually not about automation. And I think here if if I get you correctly, the the main benefit of using AI or the thing that agentic AI unlocks rather. And um or or as opposed to other technologies is the fact that you can query multiple different data sources in different environments and in different formats and digest them all into like one coherent repeatable

Yes

Workflow.

I I think that was very well articulated. Uh you know I'll also say that you know we we found this kind of leverage uh on in a few different areas as well. You know one of the things at Imprint that you know we are very excited about is, we sit very uniquely at the intersection of both uh the customer and the merchant. And what that means is we have access to very interesting data, that is uh you know very unique and proprietary here in terms of understanding uh for instance if you're looking at an application from a Rakuten customer, we can actually see their history with Rakuten, how long have they been a member with Rakuten. How have they been you know transacting over time. What has been their trajectory what does you know the basket composition look like, and that is such great signal that is just completely orthogonal to the information that's available on a traditional credit bureau report. Uh you know it's like a fraud team's uh dream uh data right, because you can steal an identity uh but it's hard to like really mirror a full digital footprint with a brand. So you know we use uh this proprietary data to build custom models uh across uh you know many different use cases, be it you know uh risk. But even outside of risk, spend. Attrition. And so on, and we what we've all done is we've built an agent that we call our residual mining agent. What this agent does is it is continuously looking at the errors in the model. Where is the model failing in its predictions. And continuing to optimize the model. Uh and this applies across our modeling suite. It finds new features. It actually is able to like look at the underlying data, create new features as well. And uh find opportunities to continue to improve our models.

These features are handed over to the data science team to actually develop. These are like leads for new features.

Exactly. So that you it's basically and it goes even beyond features. Because it'll actually retrain the new model as well. Uh and show you you know that this is sort of uh you know the new features. This is the retrained model. It'll actually do this in a manner where you know one of the things we were very worried about is it going to just overfit to the data. And uh generate a lot of biases in the predictions. So we actually you know created the agent in a manner where it does uh what we call out of time testing. So you build the model on a different time frame, and you test and validate it on a completely different time frame. So you know we did we used that technique. We did an out of sample testing as well. Just to make sure that the new model that's been generated is not overfit. We understand the benefits of the new features. And you know we found this to be very uh impactful. In fact for uh you know one of our core portfolio management models, I think about a fifth of the features are now found from uh this technique.

Super super interesting. Um yeah and I and again I think that these are exactly the kind of like super advanced use cases. It's really fascinating to see a team that already uh have that running. So, you've built all of these agents, right? And you said so yourself beforehand. Uh you know, we got rid of a lot of work and our uh analysts, or our our our team members now have a lot of time to do the high leverage work and so on. Did you guys actually downsize the the the size of the team? Because I think that you know in today's world when we're speaking about adopting AI in most cases, if not all cases, we're speaking about it in the context of uh creating efficiencies in the budget. And not only in um in the team uh the team's work. So like have you've seen this uh uh team size reduction?

No actually uh uh that's not sort of been our uh you know philosophy around this. What we have seen is you know that uh we have looked at this from the lens of what else could we be doing for our members and partners with you know the same team in place. And the results here have truly been remarkable. Uh like just to give you a a sense like you know over the last year year and a half, uh we have grown our loan book by about 150%. We have, you know, increased revenues by 2.5x. And we've done this while actually reducing our loss rates, on a uh for all of our newer cohorts. Like our newest cohorts are about, you know, 40% lower in terms of early delinquency rates and loss rates than they were a year ago. So we are actually seeing all the the great work here in terms of being able to iterate more rapidly. Uh you know detect uh opportunities for optimization sooner, but also implement this changes so much faster. Translate into such tangible results. That for us it's just been you know what all are we now able to accomplish that we were not able to do uh with the same size of team. Uh you know I'll give you an uh another example here to make this just really real. You know I think our fraud team is you know again uh you know just like uh we have a lot of thought leaders on our fraud team with a lot of experience from their years of like you know uh fighting fraud at many different institutions. And what they've done is they have actually found a way to really put all that context, uh that you know and all that uh experience as well, and leverage uh that in a manner that really scales. They've we call this uh you know project shield. And uh this is a suite of 10 different agents that run in tandem. And they're looking at our you know most recent fraud disputes. And finding newer rules, uh to actually uh be able to better mitigate that fraud. Uh so this is something that we are we run for about 6 hours at a time. And uh it uses about I think $300 of tokens. And the rules that are generated from this uh are actually saving us hundreds of thousands of dollars in fraud losses. And inbuilt in that agent suite is you know there's an agent that's actually proposing new features, that's proposing new rules. There's an agent that is specifically designed to be a skeptic. Uh that is actually you know evaluating those rules and seeing you know what is the precision of these rules. Is the false positive rate here actually meeting the hurdle. You know is this something that is, you know was working a few months ago but is not capturing the more recent trends. So you know it is uh and it it's so amazing to see all that rich deep context and experience. Um also be reflected uh in our agentic work Now.

That is so cool. I I it's interesting. And I I I I wanted to ask you, you know, if if you're not measuring the success of the project by the amount of uh headcount that you managed to cut, which I'm glad that that was never on the table. Um I'm guessing that you know in some cases measuring the the value that you create is quite straightforward. Taking you just described it, right? Taking um a list of rules that were created for $300 and a bunch of uh you know days that we had to develop this. Uh against the hundreds of thousands of dollars that you save yearly in fraud, that is pretty straightforward. But I'm guessing that there are some effects that might even be more strategic. That because you talked about it earlier when you talked about Aria as a as as a as um as a capability that lets you push the portfolio and may maybe accept clients that you wouldn't have otherwise. Uh how do you how do you measure these very tricky kind of like value ads?

No, I I think that's like a uh you know question that we think about deeply as well. You know, I think like for us, we've been able to definitely quantify it because, you know, like I guess in the form of fraud losses, in the form of our delinquency curves for newer cohorts, uh, you know, looking better and demonstrating better credit quality. But beyond that, what we've been able to do is, uh, you know, we're on this journey where, uh, like I mentioned, we have access to very rich data and context about, uh, our, uh, customers. And we're on this journey to really deepen uh personalization in every interaction with the customer through the life cycle. And uh you know that what that does is of course it deepens engagement, uh it uh you know also helps us optimize lifetime value. And expand monetization uh both for us but also our partners, and I'll give you a great example of this. Like you know and this is you know something we is only possible with really leaning in on AI. So, you know, you think about it. We have a, you know, Booking.com customer. They're in, you know, the the checkout flow and they're looking to book uh a flight. And uh you know, the flight would bring them close to their credit limit. You know for us what we are building you know through all these tools is, how do we in that moment in that high intent moment actually give the our customer a credit line increase. So that they are able to not only book their uh you know flight uh with Booking.com but they can also book their hotel they can also book their rental car and uh and you know with some room to spare. So I think that's the kind of work that uh you know we are uh driving here, is uh just and we you know see that already translate into such tangible value. But we expect that to just continue to compound over time.

[Hey folks, I want to take a quick break to speak about today's sponsor, me. If you're finding this useful, do me a small favor, like and subscribe. It really helps with the algorithm. And if you're not already on my newsletter, the Saturday fraud strategist, you should definitely check it out. Every Saturday, I break down fraud trends, real cases, and strategies from my own experience of building and operating fraud prevention systems. Whether you're new to the space or a seasoned practitioner, I'm sure you'll find it interesting. Check the link in the description. Takes 2 seconds. You'll thank yourself next Saturday. Now, back to the video.]

How do you uh measure and monitor your your agents? Like in what kind of ways? I'm guessing, you know, are these agents uh up or not? Uh are they costing me $300 per run or $500 per run? But what are the kind of monitoring frameworks that someone should have in mind when they are uh uh rolling out a ticket at scale?

Yeah, definitely. And I'll, you know, this actually goes back uh in some ways to what I talked about is that centralized platform that should be in place as you kind of roll out and you know build more specific domain specific use cases out. So for us uh you know what uh we have uh like the central uh AI and tech teams have enabled is they you know all of the agents and the skills we've built live in a you know in a centralized GitHub repository. They have you know very clear owners. Uh there's a you know rigorous process, in terms of uh when you want to change any make any changes to code. In the form of you know peer review, uh review of the code as well. Before any changes are actually productionized. So I think that is like you know just part of our machinery and exists as well. What that also means is that you know all the data sources we're using have you know like I mentioned uh you know canonical definitions. They have clear owners uh who are accountable for uh you know is the data consistent? Is it uh timely? The freshness of the data as well, and uh and we have audit trails around that as well.

And and these owners are like fraud odd specialists on the team? Or or are these engineers that build the agents according to the specs that the risk team gives them?

So we have actually uh different use cases. So if you're looking at data uh that is in, so we have the data on the raw production the raw data in our data mods is owned by our centralized data team. So they are ensuring that. But then there's you know tables that the fraud team has built, on top of that uh the data mart. And there you'll have somebody with that fraud context that is the owner. They have actually you know you know what is uh how do you define a dispute rate? You know what exactly is the numerator? What exactly is the denominator? Making sure that all that semantic context is also well documented, and uh that would be owned by somebody who has that domain expertise on the risk side.

On the data scheme I understand and I agree. I think that's the right uh the right approach. I But who actually builds out the agents themselves? Is it the risk team or is it like are they only given the specs?

Yeah. No, I think the risk team actually builds the agents themselves. We have uh you know actually across the team, we are building agents. And uh like I said we've you know the the central team has enabled us in terms of you know uh the the best practices, and the tools. Before we can actually you know push something to production. We have a peer review uh of the code as well. Uh by an independent data scientist. Uh it you know it could be a data scientist on our central data team. It could be another data scientist on the risk team. But the agents are developed by the risk team.

So let let me ask you about this. Because I think you make it sound very straightforward and trivial. But I like I don't know from my experience, this can be like a very very hard shift right. Because in order to build, and deploy, and maintain code. A agents are code. You need to have like a very very different mindset. Than what risk specialists usually have. Uh so how do you how do you deal with that? Because like it sounds like what you guys have been, uh like you know how you've rolled it out, sounds like very professional it sounds like engineering grade. How did you manage to get your team, who are not engineers, to actually comply with these best practices?

Definitely. So I think one thing I will call out is that you know the the team, uh the risk team at Imprint, is a a team that is uh you know we've built a team with very strong data chops as well. So we have uh a lot of data scientists on the team as well. So they are also we have our very own machine learning function within the risk team as well. So I think what we have is a set of folks that have worked in risk, but are highly specialized in terms of their ability to work with data to understand uh you know data structures to be to be build to even build features over time. To train models as well over time. And I think that actually has been a big part of why I think the risk team was one of the first functions, after engineering, to really adopt the the AI wave. So uh you the risk team is uh I think the number two user of AI today at the company after engineering. And I think uh a lot of that is driven by the fact that uh you know the team is uh, has a more uh you know uh has more technical experience as well. And that also uh enabled uh adoption more,

Yeah, still kudos I've seen very strong uh risk teams. I don't think it is that uh easy uh even when they have uh technical chops. Um what do you think are the prerequisite for a non-engineering team? So they can actually build and maintain an AI program that doesn't you know like degrade, and rot very quickly, and end up just being a waste of time and budget for everyone.

Yeah, I think uh that's a that's again a question that we think about deeply. And I'll actually uh add to one of the other uh you know things that's been very valuable in terms of uh adoption for within the risk team. And really scaling this program. Is what again I go back to the what the central AI team has enabled for us here right. Is they have created what we call a shared skills repository across the company. So that's a you know set of skills that has gone through the entire uh you know rigorous process. Of uh code review, peer review, uh making sure that everything is version controlled in GitHub as well. And that's available for different team members to build upon. You know, we actually have skills that also call out like these are the gotchas, right? As you're building your skill, watch out for this. And uh make sure you're kind of adding that into your context as you build more agents as well. Uh so uh you know that that's where I think the the foundational work, that uh the central tech teams have done have really enabled this. On the data side as well. You know you can imagine that if you are trying to build an agent that is needs to grab data from your data mod, it it can be uh it it's not a straightforward task right. Like which table should you use? Which field is the correct field? And uh you know you can get results that are just not correct. But are stated in a very confident manner. Which is actually uh you know which is uh not an ideal outcome at all. So I think the fact that the data team is also putting in so much time and investment into really creating uh that uh that semantic layer. Where uh you know we have well- defined data tables, with canonical definitions. So that the the agent knows exactly which data mod to query which field to look at. Uh I think all that really was a big part of why you know the risk team, which was you know uh which has had a lot of technical chops, was enabled to be able to kind of uh to build on this.

You know it it all comes down to the infrastructure right. I mean where we started, and it's uh and it's very fascinating to see the difference the difference that it can make on how well you and execute on your program. Since we started touching on it. Let's talk about uh like the build decision here. Because you know there the there is no lack whatsoever of companies out there that offer you know agents including for risk. You know like yes, or they also have, 35:33 you decide to build it on your own. And I think uh that I would say I I see a lot of teams that go down this route. Actually probably all of them if most of them, right? Um they they go down this route because um I think it's very it's very sexy and very enticing. Everybody wants to play. And it also sounds um relatively uh uh easy. When did you guys start actually? When when was when did you guys start with this uh journey?

So we started this uh I think in Q1 of this year is when it really took off.

Wow. Wow.

And started getting a lot of momentum. But actually even in Q4, uh you know is sort of when the uh we started to kind of explore the early use cases. So I'll give you an example of an early use case right. The first thing we did is we have a weekly business review report. That needed contribution from the credit team, it needed contribution from the fraud team, from the collections team. And then everybody would put this like giant you know PDF together and ship it. And that's the first piece of work that and use case that we used. And and it demonstrated you know now it's a click for us right. And you know we've also built it in a manner where that the output is deterministic. There's no uh you know we've uh there's uh we can replicate it 100% of the time. And it has just saved so much time for so many teams. That it just made that use case like so obvious and it got that momentum going. So I think we did that actually in Q4 of last year itself. And then you know in Q1 like I said is when we really started to kind of think about what does this mean for how we operate. And it's just been you know a game changer since then.

Yeah

Like quality. Yeah, I was going to say the quality of the life actually uh on the team is one thing that you know has improved so much. Uh because they're doing the work that you know they really want to do. They are spending time engaged on uh you know problems uh that are uh you know thought provoking. Uh that are strategic. Uh you know we have uh an ops uh agent who is uh saying that you know they don't miss the fact that you know they spend the first few hours of their day reviewing obvious like false positives uh that now they spend way less time on.

This is fascinating because I think and and this is such an important learning. We really are uh indoctrinized to see AI as a threat, as an enemy, and as the thing that would take our jobs. But what I'm hearing from you is that not only that it didn't take anyone's job, despite you being like very advanced. Uh but not not only that but it also makes your lives better.

That's actually been really uh you know fun to see. Like you know so much of the work that was uh you know our fraud uh reviewers, now can spend time actually finding newer patterns of fraud. Uh focusing on emerging threats. And uh that's just like you know uh of course helps us in terms of uh our uh improvement in our performance and our frameworks. But the the team uh also enjoys add so much more.

Uh, incredible. Incredible. Um, I I think this is, you know, if there's anything for you guys to take out of this conversation, I think this is it. Um, you know, we can speak about how to roll out AI, or how to build it or whether to build it or buy it for hours upon hours. But I think this goes to you know like a very key part, which is motivation versus. Also because we're talking about the transformation journey about a change management process. Uh about resistance. And I think that like seeing this example of of uh Lalitha's team being on one hand so advanced and on the other hand having like pure gains, not only on the business side, but really on the individual quality of life uh of the employees, that that I think is the most important takeaway. And I'm I'm already in the summary, but really this is like I'm I'm stopping in the middle just to highlight this takeaway because I think it is so so important.

No, I I think that's super resonates. I was going to say, you know, it's it's not come without hiccups. We've definitely had our hiccups along the way.

Tell me more. Tell me more.

Uh you know so I I'll tell you like for instance as we uh you know were rolling initially had a lot of adoption. Uh on the team what we saw is that proposals were getting much longer. Like three times the length of the proposals. But uh you know the the number of charts in the proposals had come down by half. Or actually you know in many cases we didn't have charts because you know claude works better with tables. So uh you know one thing that you know we were worried about is is like you know claude doing too much of our thinking for us. And how do we make sure that for all new proposals we have standardized best practices. You know it is uh you know good to be able to look at visually at charts and see whether results are intuitive or not. And uh longer proposals are not necessarily better quality as well. So I think what we did is once we saw that you know we as a team recognized that. And we were like okay how do we make sure we create best practices around uh proposals in this new era.

A kind of a human in the loop right. It it is not as um formalized as in in an investigation process. But it just shows how in the end it's very very hard to separate humans from AI. And how like and even if something is automated, yes it is automated but it is still monitored. It is still uh assessed and audited by humans. And that's something that you cannot really take away or take out of the process ever.

Totally

You can minimize it. Yeah.

Yeah. Definitely. Like another example here was you know as we were looking at our operational reviews that were you know again you know had uh were quite manual sometimes. It would entail people you know our expert reviewers going in opening up five different tabs. Gaining context from each tab. And trying to kind of make that decision. Uh and then document that decision. And uh you know obviously this was a use case that was just ripe for uh reinvention. And we definitely took that opportunity. But what we saw is our first pass at it, you know we had agents write up uh the review in a manner where uh you know it looked very you know comprehensive, very polished. Almost a sense of false certainty in the outcome of the review. So as you know we we saw that, we were like we went back and reflected on you know what's the the right way to actually make this something that is a a really good uh tool for our uh agents. But also with has the right guard rails. So we actually started creating uh uh within the output, uh we started measuring certainty associated with the decision making. You know where did you what data sources were behind the decision? Where was there uh you know missing data? Or potentially conflicting uh sort of uh set of ideas as well? And exposing that in the output. So that when the reviewer looks at it they can look at all those areas and really bring in that human judgment. And make a better more informed decision.

I started asking you before about build or buy. And you've you guys have been on this insane journey for less than a year. So rewinding to Q4, help me kind of like understand your um frame of thought when it comes to whether you now invest in building it yourself or whether you try at least to some extent um to to outsource and maybe uh buy some uh agents from from from vendors. How did you guys approach this decision?

Yeah. No, I think for us the way we approach we are all you know for uh buying as well. Uh where it makes sense. And we do that as well. So for our you know mental model here is, you know where we have access to proprietary information, uh that is part of our mode. Uh that allows us to uh really improvise the workflows. That's when we hone in on building. So uh but otherwise we're all about you know buying as well where uh uh you know other uh providers have the advantage of scale, have uh the advantage of uh potentially more subject matter experience in that domain. So a great example of this is we use Sardine for all of our transaction fraud uh decisioning. We use their rule engine. We use their feature store as well. And uh uh we also you know purchase external data from a variety of data sources. You know we uh we use uh uh you we use a host of vendors. We use Lexus Nexus, Prove, uh you know Sardine of course. And uh on the uh credit bureau side as well. To complement the data that we have internally. And like I mentioned the very unique uh data we get access to by virtue of our partnerships to build proprietary models. Uh because that truly is part of our competitive mode. So that's sort of how we think about the build versus buy decision. You know some of these uh agents that we've built are really specialized workflows that have so much of our unique context. In the form of you know what is our credit policy, what is our comprehensive set of fraud rules, uh you know what are our requirements for our partners and our risk appetite? So that is so unique to us. That we are able to really bring in that additional layer of intelligence there. And drive uh you know more value and incrementality.

I I wonder if you guys use also agents that you've, you know from external vendors, without going into names uh more from a perspective of if you do that like how do you how do you manage these two programs simultaneously side by side.

We actually do use external vendors. A great example of this is you know when we uh look at things like even uh external information. Any uh as we look at entity resolution within our KYC workflows. Uh and uh look at adverse media, screening and things like that. There's you know a lot of vendors that have really figured this out, in a very systematic manner. And we definitely lean in on those agents as well. So I think it really comes down to that, you know, where do we have that uh proprietary data and unique context that gives us that advantage? Versus where someone else has already figured this problem out, and has uh you know done that at scale.

Yeah. And it it sounds like between the lines and maybe maybe I'm wrong here, but the the use cases where you saw a lot of success is really where you tackled very heavy, very complex internal processes. That in the end vendors cannot cannot really touch, and definitely not throughout the entire chain.

Yeah, I I think that's actually well summarized.

Yeah. No, that's uh that's very interesting. Um okay, so Lalitha, maybe last question. I'm sure that there are a lot of uh like fraud leaders that are listening right now, you know, thinking, okay, yeah, yeah, yeah, agentic. Okay, I need to do something with it. Um what would be the one piece of advice, uh you know, with this three and a half Qs, uh on your belt. Um for for leaders who are just starting this journey,

I think the the you know biggest takeaways for me are, like I said going into this, you know I was really thinking about uh you know how do you uh optimize and make your uh move to a more optimal point on the current efficient frontier uh by bringing in more operating leverage and efficiencies. But really I think the opportunity is just so much bigger to just completely shift that frontier. You know like we are doing with being able to just bring in so much more personalization. In every interaction with our customers. And that is you know so valuable in terms of deepening engagement. Uh and uh expanding monetization as well down the line. So I think that is ultimately, I would say you know it's important to have that perspective. Uh aside from that, I would say you know the way things are changing, the pace of change is is you know really really rapid as well. So I think not being too attached to anything you build, is and expecting that to continue to change. And really you know uh again I'll say this it's like you know uh it's it's more about driving those outcomes. And continuing to iterate on uh how you kind of adopt uh these tools to drive better outcomes.

Lalitha , we covered so many things. I want to maybe rehash some of the takeaways, uh because you said some very eye opening uh uh uh things, or I collected some very eye opening insights here. Where do I even start? Um, I think maybe when it comes to like the prerequisites. Or the the enabling enablement uh part. And I think this is super interesting because this was exactly the the red line throughout my conversation with Brian. I think you mentioned two things that I think all teams should think about and understand. Whether these are boxes they already checked before they go off and and and start building their own agents. You mentioned one is uh the data infrastructure itself. Um which is uh almost almost a given almost, trivial when we speak about. But I still see many many teams who don't really don't really think about it at all. Uh and really to make sure that all of the different data, and when we think data sources we tend to think databases. But all of the like all of the different systems that hold data uh uh on your business. Whether it is sales force or Google cloud or slack or whatever, all of that is is connected and connected well. But the other thing that you said is even more interesting. Um and that is the fact that you have a centralized, you call it the AI team. It's a centralized engineering team that has developed the architecture. Uh and also kind of like the the the the building blocks. The skills. The the sub agents. Around really the the scaff scaffolding for creating engineering uh AI agents. Uh by non-engineers. Which is I think a very interesting notion. The first time that I'm hearing about it. Um and I'm sure that this has a lot to do with the insane velocity that you managed to reach in just three and a half basically, within 9 months or so.

Definitely. I would say you uh it it might look uh you know like this was an easy switch. It certainly was not an easy you know switch here. And I think uh the reason we were able to do this, was like to your point, because we had this dedicated you know AI team. That uh you know was actually explicitly one of their remits was. How do you enable non- tech teams to do this. And what's the right sort of scaffolding and guard rails to really enable that. Uh you know we call this uh our harness platform internally, uh to be able to do this in a manner that's safe and uh encourages adoption as well. How do you you know make it easy for teams? So I think uh without that you know this would certainly uh not have been something that we would have been able to really you know run as a system. And it would have become so much more uh you know ad hoc, project based and not as impactful.

Yeah absolutely. That's how it sounds like. Um absolutely. Um we also talked a bit about, um well actually. Let me start with the build or buy. We talked a bit about build or buy just now. I think uh you made the point that you know where you want to focus. I mean you can build a lot of things. And when you where you want to focus your efforts is really around where you have proprietary data that you don't want to or can't share with uh external parties. And these are probably the uh the areas around which you want to build. But you also made another point, uh when we talked about what are the use cases that you want to start with. Um and you mentioned like that specifically agent AI as means for automation. Uh what it solves, or where it brings unique capabilities is is around use cases that have multiple varied data sources in all sorts of formats. And the uh um an AI's ability to basically just kind of like digest all of that as one piece. Uh these are probably kind of like the the the use cases that that should pop uh first in priority. Um, you I think I think. Okay. And here here's I think the one of the things that I loved most about our conversation. Vision. Um, I think that a lot of the teams that enter the AI transformation journey, enter it with a very specific frame. And that is how can we be more efficient? How can we cut cost? How can we downsize the team? And your vision, your lens was completely 180 degrees different. And that was how can we make our product better? And I love it. Because it like it it is I mean first of all of course you would see uh better gains. Uh and of course you would be able to measure these gains in a very straightforward manner right. Um there are no like second order effect uh uh value here. It's like this is what you're setting up to do. I'm setting up to create more value to the company.

And I think for us like what was top of mind was like how do we create more value for our members. Uh as well as our partners. And uh like I said you know it's just you know so much part of our mission is to be able to like uh deepen our ability to understand each and every member very holistically. Across all the interactions we've had with them. What was that last interaction with our customer service team? How do we bring that context into every interaction? So, we really deciding what is the you know next best action for us? Uh an interaction for our member uh that will help deepen engagement as well.

And on top of that, I think that the the almost unexpected uh thing that you found out is that while we're making the company more profitable, or growing faster, and while we're making the lives of our partners and members better, also our own team uh is is doing better. Their quality of life has been uh has been improved by that. And I think this is such, you know it's almost it's funny to say, it but it is such a positive story around AI integration. Which has no, I mean it's s it's like win win win. And I really really think that this is a great example of not only how to do things, but also how to set up the vision. And how to align the this like AI uh transformation vision between you as a leader, the company leadership, but also your team. And how everybody can win uh out of that. That I think is very very inspiring and um super important. Um, the other thing that I really liked uh that you said, and also came up uh in my 58:02 conversation with Brian is that you know it's very easy to look at all of the cool things that you've built. And this you know this agent that trains the model on its own. And builds the features on the fly. And so on. And it and it sounds very scary. Like how can I do that. But I think the fact that you mentioned that you started out with automating a silly report. And just doing like the smallest thing. Just to show that it can work. And just to kind of like get your bearings and understand like how to even approach this technology and how to approach such an endeavor. Uh this is also like super important and I really love that you shared it. Um, I also love that you shared um, you know, the hiccups or kind of like the guard rails that you've learned um, on the fly that you need to introduce. And you mentioned both human governance and the fact that you know while we are saying that we're automating um, and that is true that doesn't mean that the human is suddenly uh, absent from the process. Uh, the human is still there. It's like a major part of of of monitoring and measuring. Uh and the second thing that you mentioned in that uh in that context was uh, and I personally also use it and I found it to be a gamechanger working with AI, is to um measure certainty and expose the certainty and the I would that's my experience. The reasoning for that certainty, whether it's high or low, to the human. And that is a gamechanger in terms of how you do governance

100%. I think everything has to be explainable. Like it that's like table stakes for us. Especially you know when we're looking at making risk decisions. Um so they're very aligned

So many great insights. Uh Lialitha uh first of all thank you very much for having this conversation and sharing from your experience. It it it is really really interesting to hear and also like super interesting to see how much you can learn in 9 months. And I'm just I'm I'm wondering if we'll talk in 9 months from today. Uh how much more things you will have to share. Uh and we should definitely do that. Um yeah, I I really enjoyed the conversation. Uh thank you very much for joining us today.

Thank you so much Chen. And I really enjoy it. I'm, you know, a fan of the podcast. So, it's such a pleasure to be here today.

No, no, the pleasure is all mine. I assure you. Uh, and folks, I I I really hope that you enjoyed it. Uh, I really hope that you uh took some takeaways with you. And I'll see you next Saturday. YouTube Description Scaling Agentic AI: Lessons From the Field October 10. 2026 Episode Guest: Lalitha Rao, Chief Risk Officer at Imprint A few weeks ago I walked through the framework for the AI transformation journey. This conversation with Lalitha Rao, Chief Risk Officer at Imprint, is the natural next step. It is a real, in-the-field look at what it actually takes when a team commits to scaling agentic AI past the theory stage. In the nine months since her risk team started building out their agentic AI infrastructure in earnest, they've grown their loan book by 150%, increased revenue by 2.5x, and actually lowered loss rates on newer cohorts. All without cutting a single headcount. Lalitha's team didn't treat this as a cost-cutting exercise, they treated it as a way to create more value for Imprint's members and partners. And the results speak for themselves. If you're further along in your own AI transformation journey and want a real example of what scaling agentic AI actually looks like once the framework meets the field, this is the conversation. What you'll hear in this episode: ● Why Imprint's centralized AI platform, built by their CTO's team, turned out to matter more than any individual agent they've built ● How ARIA, an autonomous credit policy agent, scours bureau and internal data to find early signs of portfolio deterioration and pushes the efficient frontier outward instead of just optimizing it ● Why agentic AI unlocked something a SQL query or a standalone model couldn't, digesting multiple data sources in different formats into one coherent, repeatable workflow ● How a residual mining agent continuously finds model errors, builds new features, and retrains models using out-of-time testing to guard against overfitting ● Why Imprint never measured success by headcount reduction, and what they built instead with the team's freed-up time ● Inside Project Shield, a suite of ten fraud agents that runs for six hours, costs about $300 in tokens, and saves hundreds of thousands of dollars in fraud losses ● How a shared skills repository and clear data ownership let a non-engineering risk team build and maintain production-grade agents ● Lalitha's honest build versus buy framework, and the specific vendors, including Sardine, LexisNexis, and Proof, Imprint still relies on ● The real hiccups Imprint hit along the way, bloated proposals, false certainty in AI-generated reviews, and how they built guardrails to fix both You should listen to this episode if you: ● Are further along in the AI transformation journey and want a concrete look at what scaling actually requires beyond the initial framework ● Lead a risk, fraud, or credit team and are trying to decide which agentic AI use cases to prioritize first ● Are weighing build versus buy decisions for AI agents and want a clear mental model for where proprietary data changes the calculus ● Worry that scaling AI means shrinking your team, and want to hear a real counterexample ● Want practical detail on how to actually enable non-engineers to build and maintain production-grade AI agents Episode notes & key takeaways: Building the agentic AI infrastructure first About 80% of the work behind scaling AI agents in risk is agentic AI data infrastructure, a centralized AI platform fintech teams can actually build on, with standardized data access and tooling. In Imprint's case that meant an MCP connection into Snowflake, Notion, Linear, and Slack, plus clear documentation on data lineage. An AI agent shared skills repository, peer-reviewed and version-controlled, is what let a non-engineer AI agent development effort succeed at all, turning the risk team into the company's second-heaviest internal AI agent adoption fintech story after engineering itself. That combination is really the full list of AI agent build prerequisites worth taking seriously before anything else gets built. What ARIA unlocks that other tools can't ARIA, Imprint's acquisition risk intelligence agent, functions as an autonomous credit policy analyst, scanning portfolio performance and bureau data in parallel across every partner program for agentic AI underwriting optimization, catching early signs of deterioration before they show up as delinquencies. A second, residual risk AI agent works in the background hunting for where models are failing, generating new features and handling agentic AI model retraining through out-of-time testing AI agents rely on to guard against overfitting. Together, these two didn't just move Imprint to a more optimal point on the agentic AI efficient frontier. They pushed the frontier itself outward. No headcount cut, better quality of life This was the point I think matters most. Imprint never framed AI agent deployment risk team work as an efficiency or cost-cutting project. Fraud reviewers who used to spend the first hours of every day clearing obvious false positives now spend that time hunting for genuinely new fraud patterns, a real gain in AI agent false positive reduction. The team's own reported AI agent quality of life improved, not despite the AI rollout, but because of it. Project Shield scales institutional fraud knowledge Imprint's fraud team built a suite of ten agents, called Project Shield, that runs for about six hours at a time, costs roughly $300 in tokens, and handles AI fraud rule generation that saves hundreds of thousands of dollars a year. The suite includes an agent that proposes new rules and features, and a separate agent built specifically to play skeptic, checking precision and false positive rates before anything reaches production. Early AI agent hiccups production issues, like AI-generated reviews that read as more confident than they should have, pushed the team toward real AI agent certainty measurement and AI agent explainability risk practices, exposing a certainty score and its underlying reasoning in every output so a human reviewer could see exactly where data was missing before making the final call. Build versus buy, and the bigger mindset shift Lalitha's framework for build vs buy AI agents is straightforward, build where proprietary data creates a genuine moat, and buy where a vendor already has the scale and depth to do it better, which is why Imprint still uses Sardine, LexisNexis, and Proof alongside its own in-house agents. Her closing advice ties it together, and it's really how to scale AI agents in risk without losing the plot. Most teams start this journey asking how to become more efficient at their current point on the efficient frontier, including early AI agent personalization fintech use cases. The bigger opportunity is shifting that frontier entirely, and staying loose enough about what you've already built, since the pace of change means almost nothing stays the best solution for long. Final takeaway: If there's one thing I want every fraud and risk leader to take from this conversation, it's that scaling agentic AI was never really about replacing people. Lalitha's team didn't get smaller. It got the chance to do work that actually matters, spotting new fraud patterns instead of clearing false positives, thinking about credit strategy instead of assembling weekly reports. That's not a side effect of doing this well. It's the whole point. Accordion dropdown close Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Lalitha Rao | LinkedIn Chief Risk Officer, Imprint Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: Is It 1st Party Fraud or 3rd Party Fraud? Meta Title: First Party Fraud vs Third Party Fraud: Detection Signals Meta Description: First party fraud vs third party fraud isn't just semantics, it's different detection signals and tools. Mixing them up causes friction and costs real money... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/first-party-fraud-vs-third-party-fraud Keywords Primary Keyword: first party fraud vs third party fraud Secondary Keywords: chargeback fraud returns fraud, promo abuse, synthetic identity fraud, account takeover detection, device fingerprinting fraud, IP network analysis fraud, behavioral biometrics fraud detection, KYC identity verification, velocity counters fraud detection, money mule detection, collusion fraud, transaction history analysis, credit risk scoring fraud, chargeback dispute management, fraud prevention ROI returns fraud promo abuse synthetic identity fraud account takeover detection device fingerprinting fraud IP network analysis fraud behavioral biometrics fraud detection KYC identity verification velocity counters fraud detection money mule detection collusion fraud transaction history analysis credit risk scoring fraud chargeback dispute management fraud prevention ROI

When you get hit by a fraud attack, your first question should be, how bad is it? But the second question should be, what kind of fraud am I dealing with? The difference between first party fraud and third party fraud isn't just technical. It fundamentally changes how you respond, what tools you deploy, and ultimately how successful you'll be at stopping it. Yet, I'm constantly amazed by how many teams can't tell the difference or worse, don't think it matters. So, today I'm going to break down how to identify which type of fraud you're facing and why getting it right is so critical to your business.

Let's start with the basics. Third party fraud happens when someone steals payment methods or identities or creates completely new but fake synthetic identities with the intention to defraud your business. The fraudster pretends to be someone else entirely. First party fraud happens when real customers use their real identities but have no intention of honoring their commitments. They are who they say they are but their intentions are fraudulent. Whether they commit chargeback fraud, returns fraud, promo abuse, or any other form of policy abuse. Sounds simple on paper, right? The problem is that in the real world, the lines get blurry fast.

After years of working with fraud teams across dozens of fintechs, I've noticed distinct patterns that separate these fraud types. So, let's talk about what you should be looking for. Starting with firstparty fraud, fraud cases that are notably not connected by online assets, device IDs or IP networks. Early chargeback maturation, especially in the first week after transactions. Absence of traditional fraud signals like geo mismatches or bad links. Higher transaction velocity is also often the only suspicious signal and established account history with no ATO indicators. Now, to be clear, I don't mean that you need to see all of these signals in the same account to say it's first party fraud. These are just examples for what to look for when tagging the loss events. Now, let's compare it to third party fraud. Here, you want to look for strong connections between fraud cases like shared devices or IP networks, abnormal shared behavioral patterns, for example, identical email conventions across supposedly different people. So John Smith777@gmail.com, jane smith777@gmail.com and so on. Suspiciously new yet clean identity assets like emails and phone numbers. Geographic mismatches like a new foreign country IP addressing your US service and unusually high issuer decline rates. So it's pretty clear that the difference is stark once you know what to look for. Yet, I regularly encounter teams using the wrong detection methods for the fraud type they're actually facing. And by the way, just to get it out of the way, and as I mentioned a minute ago, in some fraud technologies, the lines get blurry. This is often the case with money mules, money laundering, and collusion fraud. So, unfortunately, it's not so easy sometimes to tell the difference.

Here's what I keep seeing in the industry. Companies implementing the wrong solutions because they haven't properly identified what they're up against. And it happens so frequently that I'm starting to think it's the rule, not the exception. And it cost these companies millions. Why? Because fraud tools, like all tools, are built to solve specific problems. Use them on the wrong problem and you're essentially throwing money away. Let me give you some real examples I've encountered. A fintech dealing with obvious first party fraud, early chargebacks in established accounts decided to invest heavily in advanced KYC verification, multifactor authentication, and device fingerprinting upgrades. Unsurprisingly, their fraud rates kept climbing because these tools verify identity, something first party fraudsters already have legitimately. Then there's the opposite scenario, a SAS platform hit by classic third party fraud. Connected devices, new email accounts focused on transaction history analysis, credit risk scoring, and account tenure rules. Meanwhile, they completely missed that 80% of their fraud was coming from the same three IP networks. There's also a critical insight here that is worth highlighting. Third party fraud is actually easier to fight effectively. Why? Because fraudulent behavior patterns are distinctly different from legitimate user behavior. When someone is using a stolen identity, they behave differently than the real account owner would. These differences create detectable patterns that separate good users from fraudsters with high accuracy. First party fraud, however, is trickier because the user's behavior often appears perfectly normal until the moment they decide not to pay. And sometimes that decision is made after the payment was made, which makes it nearly impossible to detect at the time of payment. So, what do you do?

As you can learn from the examples I just shared, how you prepare and react to these two different threats is unsurprisingly different as well. Here are the hallmarks of good fraud prevention for each fraud type. And you want to make sure that you can tick most boxes. Let's start with uh third party fraud. You want to look at KYC, identity and document verification, device fingerprinting and IP intelligence, velocity counters and network analysis, behavioral biometrics, identity intelligence like email, phone, etc. Two factor authentication or multifactor authentication. For firstparty fraud, you want to look at consortium data, dynamic returns and refunds policy, chargeback dispute management, and device fingerprinting, which is mainly relevant for account sharing and promo use. See, what works for one fraud type likely won't work for the other. And that's why it's so critical to know what you're actually facing.

When you look at its roots, you realize that the fraud prevention industry was built primarily around third-party fraud detection. That's why so many teams struggle when facing first party fraud. They're using tools designed for a completely different problem. So before investing in another solution, make sure you've correctly identified what you're up against. And the telltale signs are there if you know what to look for. But here's the thing, most businesses face both types simultaneously. And the most successful fraud teams deploy targeted approaches for each rather than trying to find a one-size fits-all solution because it's simply doesn't work. Anyway, that's all for today and I'll see you next Saturday. YouTube Description Is It 1st Party Fraud or 3rd Party Fraud? October 3 2026 Episode 24 When you get hit by a fraud attack, the second question should be what kind of fraud am I actually dealing with. First party fraud vs third party fraud isn't just a technical distinction. It fundamentally changes how you respond and what tools you deploy. I'm constantly amazed by how many teams either can't tell the difference or don't think it matters. This episode is about fixing that. What you'll hear in this episode: ● The core definitions of first party fraud and third party fraud ● The specific signals that separate the two in practice ● Two real, costly examples of teams deploying the wrong tools ● Why third party fraud may be easier to detect than most people assume and why first party fraud may be harder ● The specific tool stack each fraud type requires ● Why most fraud prevention tooling was built around third party fraud by default You should listen to this episode if you: ● Are trying to diagnose a rising fraud rate and aren't sure whether you're facing first party or third party fraud ● Have invested in KYC identity verification or device fingerprinting fraud tools without seeing the results you expected ● Are responsible for account takeover detection and want a clearer sense of which signals actually matter ● Need to make the case internally for fraud prevention ROI by matching the right tool to the right fraud type ● Are dealing with money mule detection or collusion fraud cases that don't cleanly fit either category Episode notes & key takeaways: The core distinction between first party fraud vs third party fraud Third party fraud means someone stole a payment method or an identity, built a synthetic identity fraud case from scratch, and are pretending to be someone they are not. First party fraud means a real customer, using their real identity, simply has no intention of honoring what they agreed to. Whether that shows up as chargeback fraud, returns fraud, or promo abuse. ON paper that sounds simple, In practice the lines blur fast. Getting the distinction wrong is one of the most expensive mistakes a fraud team can make. What to look for First party fraud tends to show up as cases with no connection across devices or IP networks. There are early chargebacks within the first week of a transaction, an absence of typical fraud signals like geo mismatches, and established account history with no account takeover detection flags at all. Third party fraud looks almost the opposite. There are strong shared connections across devices and IP networks, suspiciously clean but brand new identity assets, abnormal patterns like near-identical email conventions across supposedly different people, and geographic mismatches paired with unusually high issuer decline rates. Two expensive examples A fintech was dealing with clear first party fraud. They responded by investing heavily in stronger KYC identity verification, multifactor authentication, and device fingerprinting. Their fraud rate kept climbing. Those tools verify identity, something first party fraudsters already have. On the flip side, a SaaS platform was facing classic third party fraud. They focused energy on transaction history analysis, credit risk scoring, and account tenure rules. All while missing that 80% of its fraud was coming from just three IP networks the whole time. The easier one to catch Fraudulent behavior built on a stolen or synthetic identity looks distinctly different from how a real account owner behaves, which creates detectable patterns with high accuracy. First party fraud is harder because the person's behavior looks completely normal right up until the moment they decide not to pay, and sometimes that decision only happens after the payment has already gone through. This makes it nearly impossible to catch at the point of transaction. Not every case fits cleanly into one category. Money mule detection and collusion fraud are the clearest examples where first party and third party fraud overlap and are genuinely hard to separate. The right toolkit The fraud prevention industry was built primarily around third party fraud detection. That's a bigger part of why so many teams struggle the first time they face a serious first party fraud problem. They may be reaching for tools built to solve a different problem entirely. Most businesses deal with both types at once. The teams that succeed build targeted approaches for each rather than hoping one tool stack covers everything. Final takeaway: The single costliest mistake I see fraud teams make isn't choosing a bad tool. It's choosing the right tool for the wrong problem. First party fraud vs third party fraud isn't an academic distinction, it's the first diagnostic question that should shape everything that follows, which signals you look for, which tools you deploy, and ultimately whether your fraud prevention spend actually pays off. Most businesses are facing both at once, which means the real skill isn't picking one approach. It's knowing which one to apply, and when. Accordion dropdown close Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: Who is the Saturday Fraud Strategist, with Hailey Windham Meta Title: Fraud Strategy Isn't Prediction, It's Reaction Meta Description: Fraud strategy means reacting well, not predicting the future. Flipping the mic to talk proactive fraud analytics, skepticism, and going solo... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/fraud-strategy Keywords Primary Keyword: fraud strategy Secondary Keywords: proactive vs predictive fraud analytics, reactive fraud response, fraud tactics vs strategy, foundation models in fraud, fraud team decision making, fraud leadership mindset, fraud consulting, fractional fraud executive, building a personal brand in fraud, fraud practitioner career growth, fraud fighter burnout, deep work for fraud teams, fraud technology build vs buy, fraud team attention management, fraud industry networking, fraud consultant positioning, fraud strategist mindset proactive vs predictive fraud analytic reactive fraud response fraud tactics vs strategy foundation models in fraud fraud team decision making fraud leadership mindset fraud consulting fractional fraud executive building a personal brand in fraud fraud practitioner career growth fraud fighter burnout deep work for fraud teams fraud technology build vs buy fraud team attention management fraud industry networking fraud consultant positioning fraud strategist mindset

They imagine that they would be able to be one step ahead of fraudsters. Never. It's not ever going to happen ever. You're lucky if you're two steps behind fraudsters and not 10. Like there's so much shit out there that you can buy and plug in. Like why? Like why build it for the hundredth time yourself? All right. Hailey, you're going to get an exclusive. This is a story I never...

What's up everybody? Welcome to the Saturday Fraud Strategist. I'm not your usual host, but today we're going to do something a little fun. So, I'm Hailey Windham and Chen thank you so much for uh joining me, having me on your podcast.

No, thank you. I mean, uh it's it's a I'm I'm super nervous. You know, I've been on podcast on the interview seat uh before, but uh you know, I've I've uh grown accustomed to being the interviewer, you know, in my very long and prosperous uh podcast career for the last 4 months. Uh so I'm now very nervous of uh how how it is uh to be back on the hotseat. Uh thank you Hailey for joining us. That's awesome.

I I think this is great. Um so we're going to give a little bit of backstory. Obviously, people want to know why the heck is Hailey here and why is she the one asking the questions. So, uh we'll answer I'll answer that in short and then Chen I'll kick it back over to you. So, I'm here. We're going to be doing some reverse uh podcast interviews both Chen and I. So, I'm joining him today on the Saturday Fraud Strategist and then he will be joining me on Fraud Forward where he will be interviewing me. So, we thought what a perfect way to kind of tee up something else that we've been working on and that we're really excited to talk about. Um, so Chen, I'll let you take it from here. Where will we be soon?

Yeah, I mean, it's not super crazy, but both Hailey and I are heading to uh Vegas. Uh, we'll be in Vegas uh next month. So, a couple of weeks after this episode is published, there's Money 2020. Uh, Vegas. Both Hailey and I would be there uh together with the Sardine crew, the Sardine media crew uh and uh a special surprise, we're going uh both to record there a few mini podcast episodes uh interviewing uh folks on the floor. Uh so if you're there and if you would like to participate, I think Hailey uh I mean I have two three slots open still. I think I I think you're the same. So, shoot us a DM or an email and uh yeah, I mean uh would love to see you there. Uh and by the way, would love to see you there also if you're not on the podcast. That's also a possibility.

Yes, absolutely. We're probably going to have a happy hour or two. We'll be around uh walking the floors. We'll be hosting podcast again live on the floor uh the show floor, which will be really awesome. But you'll also find us um you'll definitely find me somewhere probably lost in the Venetian. I always uh seem to get lost. It's very large there. So, if you see a blonde chick just kind of looking around like walking aimlessly, it's me. Help a girl out and tell me where I'm supposed to be. I I'll take care of you, Hailey. I'll take care of

Oh, thank you. So, so much appreciated. So, yes, please join us. Send us a DM if you're interested in being on one of one of our podcasts uh while we're live at uh Money2020 or again, just stop by the booth to say hi. We'd love that. All right. So, Chen normally you're the one asking the questions around here, but like we said, today we are flipping the microphones. Uh, so for anyone who knows Chen Zamir, you probably know him as a builder, uh, a fraud strategist, and someone who has spent a lot of time thinking about where fraud technology is going next. What I think is interesting about where you are today, though, is that that builder word, right? It doesn't really cover it anymore. You're advising, you're educating, you're connecting people, you're writing, you're challenging the industry to think differently. A and so I really want to start there. Uh Chen you've spent a huge part of your career building things to fight fraud. Today you're combating fraud in a lot more ways than just building technology. So how would you describe your mission now?

Well, I think it it's it's always an evolution, right? So I've been doing it for 17 years now. Um, yeah, actually when this episode is out, it's it's officially 17 years and uh I I wore a lot of different hats and, you know, fought fraud uh in in kind of like in all sorts of different capacities. And I think that over the last couple of years, especially when I when I finished my my journey with Frogster, uh it it really became clear to me that there there are two things that kind of like I want to lean in uh or to lean more into. And one of these things is uh helping more teams uh and making sure that you know when I fight fraud I do that with you know the the the most amount of leverage that I can get. And on the other hand uh also sharing uh my experience, sharing my knowledge, sharing my very hard won lessons uh and mistakes and make sure that you know that you know I I I give back to the to the community because you know I I also feel privileged to you know had both the mentors and the resources to you know make all of these mistakes and gain all of this experience. And yeah so I mean naturally I kind of like these two things also go hand in I don't know maybe not naturally but to me these two things go hand in hand. And and currently the current evolution phase of of Chen is is really, well, I I spoke about myself in third person. Now, I won't do that again. I'm sorry. I apologize.

I like it. I like it.

My my current uh uh uh evolution phrase is really to to try and do these two things together. Is it going to last for good? Probably not. Is it going for to last for the next 5 years? Maybe. Probably. I don't know. Let's see.

I I love that. Uh, do you think that that evolution though of of Chen in the third person? Do you think that that evolution was intentional or did you just look around one day and realize that that your role had changed?

I mean, I I always thought it was intentional and I always thought that, you know, I'm in control of my life and in my and my career. But now that you ask it in this way, I'm I'm trying to to reflect on whether this is a conscious decision or just, you know, how like we're alive. I don't know. I'm not sure. But back then when I made these decisions, it these were like very intentional decisions. I I knew that, you know, uh I wanted to keep uh uh chasing like an entrepreneurial life or an entrepreneurial career. And I think that, you know, entrepreneurship can manifest in a lot of different ways and forms. And to me, doing both of these things like doing consulting, doing education, uh and I never know like is it 50/50, is it 60/40, like and to what to which side. Um that seemed to me like a good way to continue this entrepreneurship journey, but maybe without the kind of like VC pressure and you know like the general pressure that you have when you're raising VC funds and you're uh going out and hiring a team and starting and trying to to win a market. So that was the idea back then and it was definitely intentional and after quite a few long months of soulsearching.

I I love that. And I think there's something pretty interesting about going from being the person responsible for building the answer to becoming someone who helps other people figure out what the answer should be. You know, that requires a different kind of thinking. Which brings me to something you mentioned when we were prepping for this conversation, and that's skepticism. Uh you told me one of the biggest lessons from your career has been learning to balance that skepticism with being open to new ideas. I want you to tell the two stories that that taught you that lesson. So let's let's start with the Frogster and and the machine learning algorithm.

Yeah, I mean I'll say that skepticism I I I think it's uh ingrained in me in general. Maybe that's, you know, like me always thinking like an analyst and always kind of like thinking critically about problems, solutions and so on. But I think that as a fraud fighter specifically, um I I have this, you know, uh as I said, a very ingrained notion that if something looks too good to be true, it's too good to be true. And because this is how fraud works and this is how you get scammed. Uh but but be because you know how many times did I write a rule and you know was super proud about myself just to find out you know it's like yeah I blocked like 100,000 accounts but none of them ever kind of like you know transacted or they they did transact but some other rules already blocked them or whatever. You know how many models did I you know uh uh launched and you know saw completely different results in production as opposed to what we've seen in the in the simulation. So, it happened to me so many times in the past that, you know, I I'm I'm kind of like it feels to me that I'm trained to immediately start looking at the bugs and the overfitting and the issues when I see very good results and and and the case that you referred, I actually wrote about it long long long time ago, like almost two years ago. Um like my most co my most costly mistake was when we launched a new algorithm at Frogster and you know there was some like research uh project and the data scientists came back with um yeah very good results much better than our like super sophisticated algorithm proprietary technology blah blah blah that we had back there back then. And the algorithm that he trained was let's say very simplistic in terms of like the architecture and technology and I I couldn't I couldn't believe that's the case. So I sent the team to do more experiments and and more experiments and more experiments like one after another um and always okay let's test it on this vertical and let's test it on this fraud trend and let it it never ceased and in the. And uh I found out that yeah like it's it's just a better model and we went out and you know like uh uh shipped it to production but it took a year since the research was done and until it was in production mainly because I didn't accept that these results that look to me too good to be true um that they are correct. Um, you know, we talked about it before. I mean, the like we just had it we literally just published the foundation model that we train at Sardine for issuing fraud and and I remember that I don't remember who put me on that. I think it was uh maybe it was Soups. Uh yeah we've done some research let's let's write about it. And I remember coming in very skeptic uh about this entire thing foundation models AI okay hype hype hype let's see what actually went down there. And I started you know poking around and asking questions and why did we do it like that and how did we kind of like uh um how did we set up the the the sim simulations and what data did we put in the training. And I found some holes and I asked a couple of questions but still the results uh looked good like very good. And at this point I you know like I I I really kind of like try to consciously tell myself Chen remember that story remember like how pissed were you with yourself um so maybe maybe don't go into this uh rabbit hole this time. And of course, you know, it's it's different like at Frogster, I was the CTO, it was my team. I could tell them to go and waste a year of their time. Um, and in Sardine, obviously, I can ask questions, but I cannot dictate to the data scientist team exactly how they run things. Um, and that's probably for for the best given my prior offenses. Um, but yeah, um, I would say that this is this is definitely one of kind of like the the mistakes that I'm still trying to learn from.

Yeah. So, you positioned uh, you know, the next question perfectly with with how you just ended that. Um, but before I ask it, I I definitely want to say I I feel you, I hear you, I see you. Uh literally I I remember being that practitioner and thinking there's no way this exists. This technology is not we're never going to get where I need it to be. You know, I I need to be able to see everything holistically. And unfortunately, there's too many gaps between connecting data and integrations and whatnot. And then I come here, even before I get here, I'm at money 2020 last year. So, we're about to bring this thing full circle of Hailey and Sardine where I go to a demo and I'm seeing Osent within the case management platform and I'm like, wait a minute, we can do that? That's a thing. And so having that skepticism also like just blindly before even giving it a chance and then getting to see it in action. I I totally understand exactly where you're coming from, man.

Yeah. Yeah. I mean, uh, every fraud fighter probably can relate to this story, uh, to a degree, I'm sure.

Yes. Yes. So, um, you know, again, teeing up that perfect question, uh, or the question perfectly, where is the line between that healthy skepticism and becoming so convinced that you're right that you stop seeing what's possible?

Wow. I don't I don't even know if I if I know right. It's it's a very blurry line. It's a very blurry line. I think I would say it like that and maybe this is kind of like my learning from you know from the last few years. Um it's completely okay to be uh skeptic and it's completely I think it's it's completely fine to have this um this mentality of you know too good to be true. Um I I think it is very important. It's an important guard rail for fraud fighters. However, um if you have enough proof points, you don't necessarily need to prove it to 100% or 110% which is my uh uh unjustified uh bar before you go live. You can still go live if you're convinced 70 80% and if you see okay yes this solution or this whatever that I shipped it works most of the time but not in some use cases or not in all not on all populations or not over time and I need to retrain it or tweak it that's fine. Uh do that as you go. It's better to ship something fast that is slightly crude as as long as it brings some you know like incremental value and um you can always work and you can always Perfect it and you can always validate it uh when it's live. Now of course you can take what I just said and you know really abuse this this statement and and go live with exactly what you don't want to go live. So it's a very blurry line and it's it's very hard for me to draw, you know, this kind of like uh general rule that would always be right for every situation.

Yeah. So true. So is there something that you believe about fraud technology today that Chen from 5 years ago would have argued with you about?

Well, honestly, I think the foundation models uh are are exactly the point. Uh I mean, Chen five years ago would probably not understand what a foundation model is. Um but I think that especially you know like looking at the recent kind of like publications uh and and of course looking very closely at what we've done at Sardine being able to like really understand the experiment not the data the actual experiment and how we how we set it up. But most of all really understand what is the role of foundation models. And in my mind uh a model is a synonym to a scoring mechanism, right? It produces scores and we use these scores to enact whatever our policy, our strategy. Um, but foundation models at least where we see them bring value is actually not with the scoring part. It's actually with enriching the data and creating new features. Um and it's a whole different topic but you know this kind of like mind shift is something that I I you know I I I I can say outright it happened for me only in the last two months. Before that I would say foundation models yeah maybe great maybe not I don't know but no chance it can work in in production because of all sorts of technical limitations considering the scoring use case, but for features that that's a different case and that's something that definitely I definitely only recently learned.

I like that. So, is that how you like pressure test a new idea without immediately dismissing it?

Well, let's say it was again a very important um lesson, a humbling lesson uh for me. Yes. Yes. I have I have a lot of those. Yeah.

Yeah. Yeah. Same. Uh truly I I don't think um I haven't had a week or a month recently where it's like okay you you didn't know what you thought you knew but it's okay it's a learning experience and it's how we grow and it's how we push ourselves to do more and so yeah I I'm I'm right there with you.

Yeah. Yeah. Especially, I mean, with AI and our world in the last two years, it's just it's been a whirlwind and it's very hard to keep up and it's very hard to separate what's hype and what's real. Um, and you know, that's that's also like literally our job. Um, so yeah. Um, yeah, step by step.

Yeah. Yeah. I I think that that willingness to to challenge your own think think thinking, you know, actually leads perfectly into what you've built outside of your day job because you've created an entire platform around thinking more strategically about fraud. Um so obviously we are recording uh this for the fraud uh the Saturday Fraud Strategist. So we talk um we have to talk about the name. Uh but first of all, why Saturday?

All right, Hailey, you're going to get an exclusive. This is a story I never

An exclusive for your podcast on your podcast. Here we go.

For you. Yeah. Uh, and it's something that I I think I I never shared, by the way, not because uh like I hiding it or anything, but just because I never got to talk about uh the name. But actually when I when I going back to the to what I talked about before um my like me ending the journey with Frogster and thinking what's next and thinking about entrepreneurship. Um I knew that whatever I'll do I want my next kind of like phase uh to be like solo entrepreneur not to have a team not to have co-founders not to have investors and so on and I started uh following a guy that produces a lot of content for uh solopreneurs. Uh his name is Justin Welsh. I highly recommend him especially if you're interested in uh making content and Justin Welsh has um I I think he still does has a newsletter and the newsletter is called the Saturday solopreneur. He talks uh in this uh in this newsletter about a ton of different topics, but one of these topics was how important it is to start your own newsletter when you're trying to kind of like position yourself in the market as a as a solopreneur, as an advisor, as a whatever like completely like it directly gave me the idea of uh of you know starting my own email list, a The Saturday Fraud Strategist. The original newsletter uh uh uh on which this podcast was named. But also the name itself, the reason why I liked it is because the the Saturday solopreneur was also like distributed on Saturday and I would read it every Saturday. It was like 5 minute read every Saturday and at that moment I understood that this is my passion. I really want to learn about this. This is not part of my job. This is not part of my workday. I don't do that on my laptop. I do that, you know, on my phone when I'm drinking my coffee on Saturday morning. These are exactly the kind of folks that I want to uh talk to uh that I want to engage with in my newsletter. And so the Saturday Fraud Strategist was a a shameless shameless theft from Justin Welsh, but but also um kind of like you know um um a declaration of who I see my my audience as. And you know many times people kind of like reach out to me and you know like refer to me as the Saturday Fraud Strategist but that's the thing. I'm not the Saturday Fraud Strategist and that's the thing because a lot of folks uh reach out to me and say oh yeah you're the uh Saturday Fraud Strategist and that's exactly the point. It's not that I am the Saturday Fraud Strategist. You are the Saturday Fraud Strategist. I'm not writing this newsletter on Saturday. I'm not waking up at 3:00 a.m. and writing it. You are reading it on Saturday. Uh you are the strategist it you're reading it on Saturday because it is important to you. It is because you are passionate about it and that is basically what the name uh signifies but also shameless theft for sure

Well I will have you know that imitation is the sincerest form of flattery, okay? That was uh from uh the Charles Coloulton, um, in his 1820 book, just so you know. I had to look up the the quote for you. Um um I love

The quote I know and if I if I don't know the the person who who said it first, it's always Mark Twain. That would be my

There you go. There you go. I I would I would go with that as well. So, um I I love that. I love that you, you know, have have obviously shared who you imagine uh the the fraud strategists are. Um I I love that that that you know, hopefully they're enjoying you for five minutes while they're drinking their coffee. Um, so do you think the audience has changed at all? And we can say, you know, from the beginning of the newsletter to now the podcast,

I think I had a couple of uh different uh personas or different types of folks uh in my mind when I first uh started writing. And a big part of that was because, you know, uh I had to answer it to myself like who am I? Am I sharing? Am I a practitioner sharing my knowledge with practitioners? Am I a builder sharing my knowledge with builders? Am I a founder sharing my knowledge with founders or with execs? So I would say it started uh a bit of a mixed bag. Uh but as time progressed, I think I kind of like uh started talking less and less to execs. Uh and to builders and really talk to practitioners in all levels. Uh so from you know like the analysts, specialist, investigator level to you know like the head of VP whatever uh CRO um but mainly to practitioners. Yes. Um but it I would say it was more kind of like a niching down slash pruning um journey more so than a shift.

Yeah. I can I can see that and I I think I've I've gone through the same thing as well. But I definitely started talking to the practitioners because I felt like they needed to have somebody that was in their corner and somebody that understood where they are, where they could be, and what could be next for them. Which, you know, brings me to my next question to you is, [Ad Break 26:15 hey folks, I want to take a quick break to speak about today's sponsor, me. If you're finding this useful, do me a small favor, like and subscribe. It really helps with the algorithm. And if you're not already on my newsletter, the Saturday Fraud Strategist, you should definitely check it out. Every Saturday, I break down fraud trends, real cases, and strategies from my own experience of building and operating fraud prevention systems. Whether you're new to the space or a season practitioner, I'm sure you'll find it interesting. Check the link in the description. Takes 2 seconds. You'll thank yourself next Saturday. Now, back to the video. ]

What is the fraud strategist? Like what does being a fraud strategist mean to you in in regards to either how you're positioning your content and and how they are receiving it or what maybe they're doing with with what you're providing afterwards.

You're asking tough questions today, Hailey.

That I told you I was putting you in the hot seat.

Um I think you know as always in our industry everybody has or you know put two people in the same room and ask them you know about a term then they will have like three definitions. Um so people have all sorts of different concepts in their minds uh when it comes to fraud strategists and what they are. And I must say that I see it less of a role and less of a function and more as a state of mind uh or a mindset. And to me, strategy means that you're thinking about long-term goals and effects that you want to achieve rather than how to go about it tactically. Um, I actually I wrote it I think not so long ago. Uh ah but I didn't release this newsletter yet so maybe I won't spoil it too much. Uh but I would say that you know I think that I always say at least I don't know if you would agree with me. I always say that predictive fraud analytics is is I don't want to say a joke but it's kind of like you know it's it's a fab. It's a fab. And I say that as someone who believed that this is real and claimed that this is what they do for a very very long time. They came to realize no that we're not doing really uh we're not doing predictive analytics here and in the best cases maybe we are proactive but I think a lot of folks misinterpret what proactive means in fraud fighting. They think they imagine that they would be able to be one step ahead of fraudsters. Never. It's not ever going to happen ever. You're lucky if you're two steps behind fraudsters and not 10. That's kind of like the best of your ambitions or at least that's how I sit realistically and skeptically. Um, so if we cannot be proactive and if we cannot be predictive, what's the what's the point of fraud strategy? And to me the point of fraud strategy is to make sure that when you are reactive when something happens and this can be a fraud attack but it can also be you know like something breaking in the system and false positives spiking or customers complaining or partners complain whatever that you can react to that uh effectively efficiently and with very little uh you know wood chips uh flying over all over the place. Uh now you you can go uh much deeper into what that means and if you are if you're subscribed to my newsletter just wait for a week or two. Um but yeah basically basically to me this is fraud strategy. How do I build and create the capacity to react to fraud at scale?

I'm so glad that that's what you said. Um, when you started talking about predictive fraud analytics and you seemed hype for a second, I was like, "Oh no, Chen and I are about to have our first disagreement where I'm going to tell him I also think that broad anal fraud predictive analytics are not real. You how how are we going to predict that? We're not going to be that's not a thing. That's not a thing at all." Um, I'm so glad that that's what you said because honestly I was so afraid that we were about to have our first disagreement live on your podcast and that was not going to be good. Crisis averted. Thank goodness. Um, but yeah, totally agree and I love the way that you frame that also and I wrote that down as hopefully uh one of the shorts that we'll use for this episode, but where you said that it's not about, you know, even whenever we misuse or misinterpret the word proactive. Proactive just means that we're going to be able to react better. Proactive fraud strategy is being able to effectively and efficiently react when fraud happens and plug the holes, use the duct tape or whatever we have on hand to get it right and then we can strategize how to how to really make it work smoothly with the overall um infrastructure of our organization.

Absolutely. And I think this is exactly it. Like when you're when you're busy with tactics, you are reactive by definition. Uh when you're busy with, you know, like managing this fire right now, you're being tactical by definition. And my job as a fraud strategist and definitely a a a person that tries to educate and elevate fraud strategist is to help you elevate your discourse from the tactical level to the strategic level. Because only on the strategic level you can truly be proactive. That's that's how I see it.

Yeah. Yeah. Do you think that fraud teams spend enough time actually thinking strategically or are we just so consumed by today's queue that strategy constantly gets pushed to tomorrow?

Really depends on the team. Uh I' I've seen all sorts of teams and yes there are definitely like teams out there that they are that are thinking strategically. Sometimes it's just one person, sometimes it's just the the leader of that team. And and sometimes, you know, it's like it's it's most of it. Um but really the the tell the very clear tell is how um how much of your day-to-day goes into putting out fires and how stressed you are by putting out fires. And if that's the case, I guarantee that you're being sucked to the tactics. You're not putting resources and resources include also attention uh just attention into the strategic layer and and and that's your Achilles heel basically. Uh and it's a vicious cycle. It's a vicious cycle that is very hard to get out of. And and usually, you know, when I when I go into like a new client and a new kind of like engagement, that's kind of like the cycle that you want to break up.

Yeah. So, of course, uh you know, giving other people strategic advice is one thing. Um but now I'm going to call us both out. Following our own advice is entirely different situation. Um what is one piece of advice you regularly give clients that you're terrible at following yourself?

Um well I I'll start by saying that like we we as as as people on this earth uh and in 2026 we all consume so many tips and guides and best practices and playbooks and whatnot. Um, and there's this pressure that you know you you're you're always not up to par with all of the things that you should be doing or how you should be doing that. And so I will say that I say a lot of things. I say a lot of things. It doesn't mean that I, you know, that I keep them 100% of the time each and every time. Uh, I always I also make mistakes. I I make a lot of mistakes. Um and I also get inspired by others. Uh it happens almost on a daily basis, right? Um and so I would start by saying that I don't think that like you should stress like you the audience should stress about it too much. Uh I try also not to stress because you know I always consume a lot of content around how to create content or how to build your brand and there are like gazillion things that I can be doing right now. Uh, and I'm

Right same.

Having said that, um, one thing that I'm that I can share that I'm like historically bad at is and I think that has to do with me being a builder is that, you know, it's very hard for me to outsource building. Um, so lucky for me there is AI today. Uh so I can be much more efficient and effective. But it is very very hard for me to you know like get um outside of like step outside of the mindset of someone can do it as at least as good as I can not to mention better. Um, and that's that's a that's a cycle that I find myself in uh very often. And I saw that when I was building a fraud product and I see that today when I'm building, you know, my own u my own systems uh within Sardine within Native Risk. Um there, you know, builders are pretty bad at uh at requesting help. Uh and that's something I for sure I can get better at.

Yeah, I can I can relate to that one. Um, I don't know even know if it's necessarily uh if if it's like asking for help even in the sense of well maybe I can just do it faster. I'll just do it myself. You know, it's it's really hard uh delegating or asking. Um I think a lot of high performing people fall into this trap of you know I can I can do it myself. Let me do it. Um, I'm even guilty of this as a mom where, you know, it's like, "Go pick this up." Well, they're not doing it the way I would have picked it up. So then I'm like, "Well, let me just do it." But then what does that teach them, right? Um, and what does it do to me? It adds to my stress level. So why not just let them do it at their own pace? Um, and you know, who knows? They might learn something or see something better. Um I think the question ultimately will come down to especially in our uh in our uh fraud labs is you know maybe maybe you you can do it faster or you can do it yourself but should you uh should you do it yourself? Should you just let someone else do it so that you can focus on other things?

Yeah. Yeah. Yeah. Absolutely. I mean it's it's exactly that. It's it's very easy to fall into exactly the trap say I can do it better. Maybe I can do it even faster but I can do it better. But there are a lot of things that I can do right now. Especially with AI, it's like in this world, there are so many options that you know open up to you as a person and your skill set that uh you know, you get pulled to into so many directions. Uh and it's very hard to decide with a clear head where your time and attention should go uh to and and where you may maybe you can let other people help with that. Um, yeah, for sure. I I can look at it from a different perspective uh to make it kind of like grounded into the fraud fighting world. I see a lot of teams that build stuff unnecessarily like there's so much shit out there that you can buy and plug in like why like why build it for the 100th time yourself? Why spend the time like why go onto this two-year adventure that you think is a one month adventure? Uh so many times like you know I I I don't get frustrated with such teams but you know I I see that very often I see it kind of like especially especially in fintech where the ethos of teams is that we are going to build something we are not bankers we are technologists we are the tech in the fintech um and it's very easy to fall into the trap of let's build our own rule engine and device ID and graph uh network analysis tool and KYC process and whatever. Uh but it's it's been done before and by probably more than uh not just Sardine let's say like that.

Yeah. Yeah. You know, you you mentioned something else that I just want to kind of double click on for a second. You mentioned like you know your attention and then um deep work. When did you realize that those were resources that you needed to protect?

I think honestly I think it was when I joined Sardine. When I joined Sardine, I suddenly realized that I have now two content engines that I need to work. One for Native Risk uh and one for Sardine. And uh and I also realized that I became basically like a full-time author. It was also the time where I was still writing my book. Um so I I had a book and and two full content engines and all I did was write. And I realized that, you know, I had several laptops on my desk and I just, you know, like skip between the different laptops and uh and operate whatever system to write a blog, a newsletter, a LinkedIn uh uh uh post, the book. Uh it was all over the place. Um and AI can help like to a point, right? Um, and I realized first of all that when I like starting when I'm starting to jump between all of these different threads, um, I realized two things. One, I suddenly found myself like back in my founder days because that was like a like this very high narrative shift, high interrupt uh, kind of like day-to-day routine that I was used to. Like I would go into my day, it would be like 10 hours back to back and it would go from you know HR policy meeting to like deep dive into this algorithm test results and then go into like an engineering planning session and then go back and you know prep for a board meeting and this is something that you can get addicted to. You can get addicted to, you know, like I'm doing so many things and I'm so productive and um and you know the adrenaline and so on. But I also realized that you tend to think less and you also tend to especially with AI, you tend to kind of like um give it too much leash. Um and that's not something that that that you want to do with AI today. Um, and so I I understood and I actually had to, you know, like work on these like different elements separately and make sure that, you know, my full attention is is devoted to them. And even if I have like some downtime because, you know, Claude is running something in the background or whatever, I'm waiting for, you know, someone to to return with an answer, um, I try not to, you know, just jump to the the the next project. That that can be quite um you know you can do that if you're super stressed but in the long term that can be quite um uh detrimental.

Yeah. So true. Um so then I'll I'll again ask you the hard question. What is something that you're still doing yourself right now that you absolutely know you should hand off?

I don't know. You know, and not knowing. I wonder if that's like me in a very good place or me in a very bad place that I'm not even aware. Uh so, yes, it's a tough question. I I uh more more things to reflect on uh when I go to sleep tonight.

Sorry. That's one of those things that'll keep you up until 3:00 a.m. Yeah,

For sure.

Yeah. Sorry about that. Um, I know that like one of the things I struggle with is at least even writing down what things I'm still doing manually that I know I could use AI to help me with that. It's just the the task of sitting down to write that list just is one of those ADHD things that I'm like, I would rather watch paint dry than write that list, but I know it will help me. So, I need to do something like that.

Yeah, I get it. I get it.

Yeah. So, there's another group of people who struggle, I think, with with this enormously, and that's people who decide to work for themselves. You made that leap, right? So, let's finish with some practical advice for anyone listening who has been quietly wondering whether they could do the same thing. So, someone listening right now has spent years working inside a bank, a fintech, a fraud team, or even a technology company. They know they're good at what they do and they're thinking maybe I could actually advise people on this. What is the first thing that you would tell them?

The first thing I would ask ask actually ask them a question and I would ask like how do they envision their uh their work or their day-to-day in whatever that means um when they say I want to work for myself because I think different people and also with different experience levels have different notions of what it means some would say I just want to be a freelancer I still want to be like a fraud professional but I'll just freelance. Um, and some would say that, you know, uh, I'm a CRO. Uh, and I still I want to work for myself, so I'll be a um, like a a fractional CRO uh, or a fractional, you know, chief compliance officer or whatever. That's completely different. Um, some would say, yeah, I want just to do like uh, more strategy pro like more advisory. Uh so is advisory like you sitting down with you know like a leader a VP a C level and talking to them like having a one-on-one one every week or is it that you coming in and you're doing like a full-time two month project. So there are a lot of different um different ways that in which you can like work for yourself and obviously you know all of these things really dictate how your sales pipeline should look like who's your ICP how you build content around it and all of that. But I think this is kind of like how you start. Um I think there are two things that I would say. The first one is that you need to understand that when you are saying that I want to work for myself. You are no longer a fraud fighter. You are no longer a fraud analyst or a fraud investigator or you know a head of fraud. You are no longer that. You are now a business owner.

Yeah.

And being a business owner means that first and foremost you're in charge of sales and you're in charge of marketing and you're in charge of admin which include finance and legal. Um and you need to have all of that in place to get your first customer. And only after you get your first customer then you can do what you love to do, what you used to do, what is easy for you to do. Um these are honestly in my mind the easy and boring parts of my work doing the job. Um so one understand that this is like a very big shift and is it really what you want? Uh I I don't think that everybody that talks to me about this kind of like possibility really understand what that entails and that's fine but I think it's it's it's important to to realize that. The second thing that I will say is that uh in order to get booked, you need to have attention. Uh people need to know who you are, know or know who you are, trust you and know that you are uh available for this kind of work that you imagine yourself doing. And to do that, you can build attention in one of two ways. And you can do one of them, and you can do both of them. But I never heard of anyone successful long-term that did neither of these. So there are two things that you can do. One is that you can hit everybody in your network and have a coffee with them. You don't ask them for anything. You don't ask them for a work for a job. You don't ask them if they know someone who is looking for uh I don't know like a consultant or friends. You don't ask any of that. You have a coffee. You you catch up, you say what you're up to and you check if there's any way that you can help. Not for any kind of um uh gratitude, right? Like is there a way that you can help them? Um and you need to do that constantly and consistently. That's way number one. Way number two or second method is uh to go berserk on content and to just make sure that people know who you are and trust you and uh uh and know that you are available for these kind of uh projects not because you talk to them but because you talked at them. And these two things, what they what they do, what basically the the engine that you want to put in place is the fact that when someone suddenly has a need or someone calls them a friend and that friend has a need, the first name that pops into their head is, "Oh, it's Hailey." Yeah, I know. I have the perfect person for you, Hailey. And this is because they just read a post of you yesterday or read a uh a newsletter or uh or had a call with you last week. And do that enough uh and over a long period of time that doesn't happen overnight like at all at all. Do that enough um and and and people will start uh reaching out or your network will start reaching out.

I love that and it it resonates with me just because I recently went through it myself. Um but then I I joined the the Sardine team. But it's exactly right. Making sure that you know when you're reaching out it's not for any disingenuous reason. It is honestly just to say hey how can I help? We're not looking I'm not looking for compensation. I just want you to know that I'm here and I have the this experience and this knowledge and this know how and I'd love to, you know, advise or talk or whatever. And it just starts out as friendly and then eventually it turns into a a paid for gig down the road through word of mouth, through referrals, through things like that. Um, yeah, I literally experience just that.

And and I would say just to be like super super clear when I say you know check how you can help and when you say this is not like pay for uh sorry work for free this is not what uh this is about right. It's not about offering your services for free. It is to see if you can give them a piece of advice is to see if you can give them if you can refer uh them to someone that you know that can help or something that you read that can help or something that you've written that can help. That's the best combo. Uh or um you can maybe connect them to someone in your network that you know is looking for for a job and they are ready to join full-time or you know a good vendor that can fix their problem. You can like uh get them a nice introduction. So there are a lot of different ways to help not to mention like personal stuff as well uh that doesn't require you to work for free. Never do that.

Right. Yep. Um, okay. Last uh last question. Um, well, I say last question, I'll probably have another one for you, but if someone uh listening decides today that, you know, a year from now, I want to have the option to work for myself, what should they start doing this week?

Exactly these two things. Like, start with these two things. Now, you don't need to understand who you want to work with and how you want to do it and whether you want to do that. But like my biggest regret is that I didn't start with these two things with these like attention engines a year before I started. Um this would uh put you in a very very different position when you decide to start and when you decide to start um and and a much better position at that. Now the nice thing is if you end up deciding not to do that you don't want to be you know like to to kind of like go on your own what you would gain by doing these things by getting attention to you can cannot hurt you in any way. Even if you decide to continue with your career kind of like you know like be the the the salary man salary woman and kind of like step up the ladder and so on more people would know about you. Uh it would be much easier for you to DM your way into uh a role. It would be you would get so like the like I remember me being frustrated in the first 6 months of me like uh starting with my content is that I would get job offers like you know full-time job offers which is not what I wanted. Uh and I would get frustrated because I would like waste my time when people who are not serious and don't want me to consult them they want me to join the team. Um so doing that whether you've decided now or not uh I think anyway serves a purpose and honestly I would recommend doing that to anyone literally it can only improve your uh your life like literally.

So Chen we are going to flip this whole thing again because next you'll be interviewing me over on fraud forward.

Yes.

But before I give you your microphone back, um I want to end with this last final question. You've built tech fraud technology, you've advised companies, you're educating people, you're connecting people, you're writing and podcasting about this industry. What are you trying to build now that can't be measured in code, revenue, or a product map?

Wow. Uh I think you know it's like uh a perfect tie to our previous topic, relationships, right? Um I know that the relationships that I've built from my laptop with complete strangers uh but Hailey and I never met in person never

Yeah Money 2020 we're going to put it in the books.

Yeah. Yeah. Um Hailey you know you are just one example. The amount of relationships that I've made and that you know I I would get regardless of what kind of you know like opportunities or or revenue would come out of it doesn't matter. It's like it's I I know that I'm richer for it. Um and I know that it fulfills me um on a different level. So that's you know 100% relationship.

So true. So true and so important in this industry man.

Yeah.

What a How LinkedIn has changed my career.

Yeah.

The power of connections and relationships.

Yeah. Yeah.

All right. Chen Zamir, the Saturday Fraud Strategist, I can't thank you enough for letting me steal your microphone for for this episode.

You know, it's been a pleasure, Hailey. I now because we haven't recorded yours yet. We we are two days away from it. Now I will uh I will prep myself very well so I know to crank up the heat. Uh so so I I give you some I dish out some uh hard questions of my own. Uh I'll think about it.

Bring it on. I'm so excited.

Awesome. Hailey, it was it was a pleasure having you on. Uh we should do it again sometime.

I would love that. Thank you so much. Chen

Thank you. Uh and for all of you folks, I see you next Saturday. YouTube Description Who is the Saturday Fraud Strategist, with Hailey Windham September 26, 2026 Episode 23 I handed my microphone over to Hailey Windham, host of Fraud Forward. For once, I'm the one answering the questions instead of asking them. Hailey opens by asking how I'd describe my mission today, since "builder" doesn't quite cover what I do anymore between advising, educating, and writing. That question sets up the real spine of this conversation. What fraud strategy actually means, why I think predictive fraud analytics is mostly a fabrication, and how a fraud strategist's real job is making sure a team can react well when things break, not pretending anyone can stay ahead of fraudsters. Hailey and I talk about where healthy skepticism turns into a blind spot, why foundation models are proving more valuable for enriching data than for scoring transactions, and I close out with real advice for anyone quietly wondering whether they could go independent themselves. What you'll hear in this episode: ● Why "builder" no longer fully describes my role, and how I think about mission and mindset shifting over a 17 year career ● Why I believe predictive fraud analytics is mostly a fabrication, and what proactive fraud strategy actually means instead ● Why I walked into Sardine's foundation model research already skeptical, and what I learned about where foundation models actually create value ● Where the line sits, or doesn't sit cleanly, between healthy skepticism and closing yourself off to new ideas ● Why so many fraud teams default to building technology in-house when buying an existing solution would serve them better ● How I learned to protect deep work and attention once AI made constant context switching dangerously easy ● Real, practical advice for anyone considering independent fraud consulting, fractional leadership, or advisory work ● The origin story behind the name Saturday Fraud Strategist, and why I don't think of myself as the strategist, you are You should listen to this episode if you: ● Want to understand fraud strategy as a mindset rather than a job title, and how that shift changes your day-to-day priorities ● Have ever felt torn between healthy skepticism and staying open to genuinely new ideas in fraud technology ● Are evaluating whether to build fraud technology in-house or buy an existing solution ● Are curious how foundation models are actually creating value in fraud detection right now ● Are a fraud practitioner quietly considering independent consulting, fractional work, or advisory services ● Want practical advice on building attention and a personal brand in the fraud industry, without it feeling disingenuous Episode notes & key takeaways: Fraud strategy is a mindset, not a job title Hailey asked me directly what being a fraud strategist actually means, and I think the honest answer is that it's less a function and more a mindset. Strategy means thinking about long-term goals and effects rather than tactics. If your day-to-day is consumed by putting out fires, you're operating tactically by definition, and only stepping back from that lets you actually be strategic. Predictive fraud analytics is mostly a fabrication I said something on this episode I've believed for a while but rarely say this bluntly. Predictive fraud analytics is a fab. Teams that think they'll get ahead of fraudsters are chasing something that never happens. You're lucky to be two steps behind instead of ten. If we can't be predictive, the real value of fraud strategy is building the capacity to react to fraud effectively and efficiently when it happens, not pretending we saw it coming. Two stories about skepticism, and where it becomes a blind spot I told Hailey about the most expensive mistake of my career, refusing to believe a simplified model had actually outperformed our proprietary one, which delayed shipping it by a full year. I also told her about walking into Sardine's foundation model research already convinced it was hype, and having to consciously remind myself not to repeat the same mistake twice. Healthy skepticism is a real guardrail for fraud fighters. The line between that and stubbornness is genuinely blurry, and I don't think there's a clean rule for where it sits. Foundation models earn their value in a different place than most people expect Something I only fully understood in the last couple of months is that foundation models aren't proving their worth in scoring transactions, they're proving it in enriching data and creating new features. That's a real shift from how I thought about them even recently, and it's exactly the kind of assumption I want fraud teams to keep testing rather than treating as settled. Why so many fraud teams build things they should just buy I told Hailey I see this constantly, especially in fintech, teams rebuilding a rule engine, a device ID system, or a KYC process from scratch because their internal ethos is "we're technologists, we build things." Most of the time, that decision costs a two-year detour disguised as a one-month project. Fraud technology build vs buy is a real decision, and defaulting to build almost never serves the team well. Protecting attention and deep work is harder than it sounds I'm historically bad at delegating, especially anything related to building, and I told Hailey that directly. Once I joined Sardine and found myself running two full content engines plus writing a book, I realized I'd slipped back into founder-mode context switching, and that constant narrative shifting quietly erodes how well you think, especially when you're also leaning on AI and giving it too much leash without noticing. Real advice for anyone thinking about going independent Hailey asked me what I'd tell someone quietly considering consulting or advisory work, and I gave her the two things I actually believe matter. First, understand that going independent means you're now a business owner first, handling sales, marketing, and admin, before you ever get to do the work you're actually good at. Second, build attention deliberately, either by reaching out to your network with zero ask attached, just checking how you can help, or by putting out content consistently, so that when someone needs what you do, your name is the one that comes to mind. Final takeaway: If there's one idea from this conversation I hope sticks, it's that fraud strategy was never about outsmarting the future. It's about building a team, and a mindset, that can react to whatever actually happens with speed and clarity instead of panic. That applies whether you're running a fraud program, evaluating a new piece of technology, or deciding whether you're ready to build a career on your own name instead of someone else's team. None of it happens by prediction. It happens by being ready. Accordion dropdown close Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Hailey Windham | LinkedIn Host of Fraud Forward Community Banking Lead at Sardine Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: The Rise of Agentic Ops, part 4: How to Monitor AI Agents Meta Title: AI Agent Monitoring for Fraud Teams That Actually Works Meta Description: AI agent monitoring means knowing when an agent degrades before it costs you. Breaking down agreement rate, distribution shift, and real fraud agent alerts... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/ai-agent-monitoring Keywords Primary Keyword: AI agent monitoring Secondary Keywords: fraud agent drift detection, AI agent performance metrics fraud, human in the loop agent monitoring, autonomous agent monitoring fraud, fraud reaction cycle measurement, agreement rate AI agent, cross-source agreement rate fraud, AI agent governance fraud, fraud alert clustering monitoring, AI agent ROI fraud, fraud agent degradation signals, agentic fraud ops monitoring, fraud AI agent rejection rate, distribution shift fraud agent, fraud agent value measurement, AI labeling agent monitoring, real time fraud agent alerts, fraud agent weekly review, fraud reaction cycle speed, how to monitor AI agents in fraud, fraud agent Slack alerts, AI agent cascade failure fraud, fraud AI agent cost ROI, autonomous fraud agent silent failure, fraud agent output quality tracking, AI agent monitoring dashboard fraud, agentic AI fraud team accountability, fraud agent backtest monitoring, AI fraud agent leading indicators fraud agent drift detection AI agent performance metrics fraud human in the loop agent monitoring autonomous agent monitoring fraud fraud reaction cycle measurement agreement rate AI agent cross-source agreement rate fraud AI agent governance fraud fraud alert clustering monitoring AI agent ROI fraud fraud agent degradation signals agentic fraud ops monitoring fraud AI agent rejection rate distribution shift fraud agent fraud agent value measurement AI labeling agent monitoring real time fraud agent alerts fraud agent weekly review fraud reaction cycle speed how to monitor AI agents in fraud fraud agent Slack alerts AI agent cascade failure fraud fraud AI agent cost ROI autonomous fraud agent silent failure fraud agent output quality tracking AI agent monitoring dashboard fraud agentic AI fraud team accountability fraud agent backtest monitoring AI fraud agent leading indicators

Here's a question I've been asking fraud leaders lately. If I told you one of your AI agents had been degrading for 6 weeks, would you know? Most would say no. And it's a problem because drifting agents are actually worse than no agents. They generate outputs that look fine from the outside while they degrade. And it's especially problematic when these outputs are being used as inputs for downstream processes like other agents. And so you can imagine how easy it is to encounter cascading events that very quickly get out of control. But the truth is that if you take a look at your average dashboard, it's designed to answer something else entirely than these questions. Is this agent even running? That is the least useful question to ask. I mean, of course, system health is important, but focus only on that and you'll miss the really important things to monitor. Whether your agents are improving your outcomes and not just whether they are live. So, how do you keep track of your agents? There are two things you want to watch for. First, is the agent actually contributing value? And the second, is it behaving as expected? Let's talk about it. In the first video of this series, I mentioned that in my view, the master KPI of fraud effectiveness is the reaction cycle. The time it takes your system to detect a gap and deploy a fix for it. And in the second video of the series, I outline how you can streamline your reaction cycle with Agentic AI to make it significantly faster. But how do you actually measure it? How do you connect AI agent performance metrics to how fast you're stopping fraud? Fraud AI agents can definitely help with that, but not necessarily in an even manner across the board. Here's what I would suggest to track and how. First, detecting coordinated fraud patterns faster. The first step in the fraud reaction cycle is detecting that there's a system gap. Without agents, this is slow by default. Alerts arrive and analysts eventually notices a pattern and then someone needs to pull related cases by hand to validate there's an issue. A new attack can run for days or even weeks before anyone connects a dots easily. But if you follow the steps in the second video of this series and you implemented alert clustering, a new ring can now surface in hours. Every alert triggers an agent that searches against known cases and matches events to existing rigs. So how do you track this? You track it by measuring the time from the first event of a new attack entering your system to when your team actually recognizing it. Now, I know what you're thinking. This is super tricky. So, I let you in on a little secret. Not every metric you measure needs to happen automatically. Just make sure that every week someone in your team goes through the alerts they've actually picked up and note down how long it took since the issue first started. Do it enough times and you'll be able to see a trend, especially if you start doing that before you implement agentic AI, as you should. The second thing you want to do is to scope fraud rings with AI agents. Once a pattern is flagged, the next question to ask is how big is it? How many accounts? What time period? What's the actual loss exposure? This is where Agentic AI moves from spotting a signal to mapping the full population at risk. Without AI, the scoping is manual and analysts will have to pull related cases, cross reference them with device data and build a picture account by account. But with AI agents, the same work can be done in a much shorter time window. How do you track this? Measure the time between when an issue got noticed to when you had it scoped in dollar or account exposure terms. Again, this doesn't have to be super sophisticated. The point isn't to say it takes 5 minutes and 12 seconds to scope an attack instead of 5 minutes and 36 seconds. It's being able to show it takes less than 30 minutes versus the day or two it took before. And that is pretty easy to do in the same manner I mentioned before by recording it manually once a week. By the way, I know that what I just said may sound like sacrilege to some of you. What? Do something manually when we talk about AI monitoring? How can I say that while I preach for more automation? But this is exactly where I see teams get distracted by investing their attention and tokens into productivity hacks instead of focusing on the right things. Don't get me wrong, if you can automate these metrics with or without AI, you should absolutely do so. But don't let it stop you from measuring them if you can't. Finally, the last thing you want to track is how fast you design and test fixes.

Once you understand the issue and what causes it, you need a fix. And AI agent can help with the two steps that used to consume the most time. Proposing a solution and proving it works. When fed a specific data set, agents can identify patterns, learn from labels, and propose a fix like a new rule. Then they can run the back test all before human even reviews it. So the analyst's job shifts from building the rule to stress testing and approving it. Now the reason why I would measure these two phases together is because in some cases like rule writing, it's a bit difficult to know when design ends and testing begins. And in other cases like SOP or policy changes, it might be that there would be no testing phase. So when you think about how to measure it, this one is a bit tricky. Supposedly you need to start measuring it when you have the root cause figured out, but this can prove to be quite a fuzzy definition. So again, you don't have to overengineer it and get a super accurate measure. It's enough to have a ballpark range for how much time it took since you started working on a solution and until it was approved for deployment. You want to see if when using AI agents, you can consistently shave a meaningful chunk of time. And if you don't see through rough measurements, it's likely not doing enough work anyway. Here's the problem with cycle level metrics. They lag. Meaning, you're likely going to measure some sort of a roll in average. And as we just discussed, a rough one at that. So if for example your rules agents start to degrade, it'll be quite hard to catch. Analyst would spend more time on prompting or tasks that it would oneshot before would now take several prompts to achieve. Or a labeling agent that started misclassifying 3 weeks ago will eventually cause your rules to drift and elevate fraud rates, but until you notice it and understand where it's coming from, it might be going on for weeks. And that's why you want to make sure your agents are not only live are not only noticeably making you react faster, but also that they do not degrade with time. And you want to know something is wrong before it shows up in your reaction cycle metrics as by then it's too late. But these signals look different depending on whether the agent has a human-in-the-loop or not. Human-in-the-loop agents are the first type of AI agents you'd monitor. These include investigation co-pilots or rule recommendation agents that have humans reviewing every output before anything gets actioned. And because you have a human in the loop, these agents are easier to monitor because the review itself creates a record. Did the human agree or reject the agents decision?

This creates a natural trail you can record and monitor. Agreement rate is the best leading indicator and one of the most useful agent metrics for human reviewed workflows. If investigators are ruling differently than the agent more often than they used to, then the agent is degrading. For rule recommendation agents, rejection rate is the equivalent. It's another AI agent performance metric that shows whether the agent is still producing useful outputs. It's a bit trickier because rejected rules don't reach production, so they don't necessarily impact your general KPIs. But a rise in rejection rate means the agent is generating faulty proposals that consume human time and resources without moving the needle. And by the way, there can be many reasons for agent degradation. Maybe there are underlying data issues. Maybe fraud has shifted. Maybe moving to a newer model didn't work well. Whatever the reason is, you could identify issues earlier by tracking these metrics. Meaning you could also start solving it faster. Exactly the concept behind the reaction cycle. Autonomous agents such as auto labeling and alert clustering don't have humans reviewing every decision they make. So when they fail, the failure is silent until something downstream breaks. Let me give an example. When an auto labeling agent starts misclassifying legitimate accounts could get labeled as fraud without anyone noticing. Obviously, no one wants that. This is why AI governance has to account for autonomous agents differently than human-in-the-loop flows. But the problem here is scale. The reason humans are not in the loop is because these flows simply make too many decisions or they make decisions in near real time. So not only you cannot track human agreement rates, the danger here is substantial. Any slight degradation can have a severe impact, but you can track other agreement rates coming from other sources or cross- source agreement rates. Let me give an example. Say you want to monitor your labeling agent.

It's not your only source of labels, right? You probably have at the very least the chargebacks you're getting as well. Now, if two sources for the same decision that were aligned 90% of the time drop to 70% of the time, you know something changed. You don't know which source is wrong yet, but you know where and when to look before the damage compounds. Another signal worth tracking is distribution shift. If your labeling agent starts flagging 40% more events as fraud in a segment that hasn't seen elevated fraud rates, that's worth investigating. Again, there can be many reasons for why an autonomous agent degrades. Just like with human in the flow agents, the point is that you now have the metrics and the alerts so you could catch it early. In the previous video of this series, I outlined how managing AI agents, including monitoring them, falls under the responsibility of your fraud analytics function. And from a tooling perspective, it's nothing new. What do we actually have here? Real-time alerts for threshold breaches. For example, when the agreement rate of a human-in-the-loop agent falls down below a certain value or when an autonomous agent's distribution skews more than a set rate. These alerts form the foundation layer of AI governance and should fire automatically on Slack or email or wherever your team catches alerts and get looked at the same day. Then you have weekly reviews of trend lines. Here you're not only making sure everything looks good regardless of whether an alert was fired or not. You also take a look at impact. Measure your reaction cycle speed and make sure that your agents provide actual value. And lastly, you have monthly reports on cost and ROI that you can share with your leadership team. And especially for the reaction cycle, it's pretty straightforward. If you can demonstrate an attack was blocked 4 days faster, you just save 4 days of losses. That's part of the ROI calculation, not just how many work hours you saved with automation. I started this video with a simple scenario. One of your fraud agents has been degrading for six weeks.

What do you know? If you're tracking agreement rates for your human-in-the-loop agents, you'd know in days. And if you're tracking cross agreements and distribution shifts for your autonomous agents, you'd know in a week. At the same time, you can now also track the value agents create. Not only how many hours you saved, but also how many dollars you saved by detecting and reacting to fraud quicker. That's how you connect your reaction cycle to your top KPIs. And this is the difference between you running agents and the agents running you. YouTube Description The Rise of Agentic Ops, part 4: How to Monitor AI Agents September 19, 2026 Episode 22 If one of your AI agents had been degrading for six weeks would you know? Most people tell me no, and that's the problem I'm digging into. Drifting agents can generate outputs that look fine on the outside while quietly getting worse underneath. Earlier in this series I talked about the reaction cycle as the master KPI of fraud effectiveness, and how agentic AI can make that cycle dramatically faster. This time I want to answer the question that matters once you've deployed those agents. How do you know they're still working. Most dashboards answer the wrong questions and only answer whether an agent is running. AI agent monitoring means tracking an agent deliberately, and I will walk you through exactly how to do it. What you'll hear in this episode: ● Why a degrading agent is genuinely more dangerous than no agent at all. ● How to measure fraud reaction cycle speed at each individual stage rather than just watching one lagging number. ● Why AI agent performance metrics fraud teams should track don't need to be perfectly automated to be useful. ● The difference between human-in-the-loop agent monitoring and autonomous agent monitoring for agents making decisions at scale. ● What a rising rejection rate actually tells you. ● How to catch a silently failing autonomous agent before real damage compounds. ● A practical three-layer AI agent monitoring dashboard fraud teams can build. You should listen to this episode if you: ● Are running any agentic fraud ops monitoring program and want a real framework for catching a degrading agent before it shows up in your losses. ● Are responsible for AI agent governance fraud policies and need language that connects technical monitoring to leadership reporting. ● Have deployed human-in-the-loop tools like investigation copilots or rule recommendation agents and want to know what to actually track. ● Are running autonomous agents, like auto-labeling or alert clustering, with no human reviewing every decision, and worry about silent failure. ● Want to build a genuine business case for AI agent ROI fraud investment using the reaction cycle instead of just automation hours saved. Episode notes & key takeaways: Uptime is the wrong thing to monitor Most dashboards only tell you whether an agent is live, and that's the least useful question you can ask. A drifting agent can look perfectly healthy while it quietly gets worse at its actual job. Real AI agent monitoring means tracking whether an agent is still contributing value and behaving as expected, not just whether it's still running. Break the reaction cycle into stages you can actually measure Detection speed, scoping speed, and fix design speed are each worth tracking on their own instead of watching one lagging average. None of this needs to be perfectly automated to be useful. A rough manual log kept once a week is enough to show whether AI agents are meaningfully shaving time off each of these stages, and that's a better use of your effort than waiting to build a perfect measurement system first. Cycle-level metrics lag, and that's exactly the danger Reaction cycle numbers are rolling averages, which means fraud agent degradation signals can hide inside them for weeks before anyone notices. By the time a slowdown finally shows up in your overall cycle time, the damage has usually already been done. You want signals that catch the drift earlier than that. Human-in-the-loop agents leave a paper trail, but autonomous agents don't For agents that a human reviews before anything gets actioned, like investigation copilots or rule recommendation tools, agreement rate and rejection rate are your best leading indicators. For autonomous agents making decisions with no review step at all, like auto-labeling or alert clustering, failure is silent by default. Cross-source agreement rate and distribution shift are the signals that let you catch that kind of drift before it compounds into a real problem. Build your monitoring in three layers Real time alerts should catch threshold breaches the same day they happen. A weekly review should look at trend lines and overall impact, not just whether an alert fired. And a monthly report should connect agent performance to cost and ROI for leadership. None of this requires new tooling, and the responsibility for it sits with your fraud analytics function. Final takeaway: I opened this episode with a simple scenario. One of your agents has been degrading for six weeks, and you don't know it yet. If you're tracking agreement rate for your human-in-the-loop agents, you'd catch that in days. If you're tracking cross-source agreement and distribution shift for your autonomous agents, you'd catch it within a week. At the same time, you'd be able to show exactly how much value those same agents are creating, not just in hours saved, but in dollars saved by reacting to fraud faster. That's the real difference between running your agents and letting your agents run you. Accordion dropdown close Transcript > ( Accordion dropdown) Resources & links: This is part of a series. If you landed here first, you may want to go back and listen to the previous episodes. We've already covered quite a lot that will make this one much easier to follow. Catch up on The Rise of Agentic Fraud Ops, part 1 Catch up on The Rise of Agentic Fraud Ops, part 2 Catch up on The Rise of Agentic Fraud Ops, part 3 Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: The AI Adoption Journey for Fraud & Risk Teams Meta Title: AI Adoption Journey For Fraud and Risk Teams Meta Description: AI adoption for fraud and risk teams isn't one leap, it's five stages. Breaking down AI activation vs AI enablement, and what actually works... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/ai-adoption-for-fraud-and-risk-teams Keywords Primary Keyword: ai adoption for fraud and risk teams Secondary Keywords: fraud team AI adoption stages, AI activation vs AI enablement, AI blocked to AI native journey, fraud team change management, AI literacy for fraud teams, AI enablement leadership buy-in, fraud analyst upskilling with AI, build versus buy AI fraud tools, AI incentivization for teams, fraud team AI pilot programs, workflow design for AI adoption, fraud team AI training, fraud practitioner AI use cases, reducing engineering dependency with AI, fraud team AI governance policies, AI first fraud organizations, AI native fraud teams, non-technical AI use cases for fraud, fraud team productivity with AI AI transformation journey for risk teams

Once I get hired, I'm lazy. Full stop. Here's a tool. Go use it. We're giving you no guidance. You might get in trouble. You might not get in trouble. One team could see that they're AI native. And then another team in the same organization will say we're AI aware, meaning we're not doing anything, but we should. The simplest one I do is,

Welcome everybody to another episode of The Saturday Fraud Strategist and with me today Brian Davis in the house the Brian Davis. Uh actually this is how I need to introduce you. My assumption is that if you're listening to this podcast uh you probably know who Brian Davis is. That's my assumption. But for the two three folks who are listening to us and somehow didn't come across you uh tell us a bit about yourself.

Absolutely. I've been in the fraud and risk space for a little over a decade. Um I wanted to get into fraud because I got a little bit of like education within my college and my master's program even though I had a masters in accounting placed out I did a 5-year program so I placed out a lot of my courses. So I ultimately in my master's program only took entrepreneurship and fraud. Although I got a piece of paper that said I'm really really good at budgeting. Good luck getting anything else. Fast forward a year, I was finally able to get into fraud. Didn't know what that meant, what it really would entail. I wanted to work in e-commerce. Didn't really know what that meant. A decade ago in Boston, there weren't really many jobs in e-commerce and fraud. So, I found one company, convinced them to hire me that I studied a little bit about fraud, so I should be your first fraud hire. Um, and haven't looked back since. My weird little quirk is I like to see how different business models are abused by fraud. So I've done physical goods, e-commerce, marketplaces, fintech, consulting, vendor side. Now today I sit in the center of the ecosystem putting on uh AI deep dive retreat. So now in events and community building. So it's I've seen it from a lot of different lenses. It's given me appreciation of how different tactics are used in different industries, but there are little nuances or the type of data or what you can or can't do for controls, kind of impacts how you can mitigate different uh types of risks. So like a fintech is going to handle that a lot differently than a digital subscription uh for onboarding and fake accounts and circumvention and like I like there isn't really necessarily enough information on the digital subscription to say do we have synthetic identities? I don't know. It's just a fake account at that point because we don't collect enough information. However, are they building something to ultimately use down the line? So being able to see how these different tactics evolve and are just tested from platform to platform has uh been a way that I like to challenge myself on top of always being the first fraud hire. So I like to build I'm the zero to one person who comes in and says none of you want to deal with fraud for some reason. I do. Let me come in. I'll build out the team, the tools, the rules, everything sort like that. I'll help hire the team, build out the function, and at that point, there's too much structure in place for me, and I go on to the next one.

That's super, you know, super interesting because I remember, I don't remember exactly when I bumped into you, obviously on LinkedIn. It was, I'm guessing, very early into my own uh LinkedIn journey. And I remember two things. I remember one that you know we we both had very different careers in fraud, but also very similar uh to an extent. Meaning we are kind of like hovering around this problem from all sorts of different perspectives and wearing all sorts of different hats. Uh and I think especially I don't know a lot of other folks who have, who who transition from being a practitioner to being, sorry I'm you know I'm using a very bad word towards you and towards myself as well, but being a marketer. Uh so that's that's one that's interesting. Secondly, I also remember that you know reading your post and you're you didn't mention that you're very active on LinkedIn for many many years right now. Oh yeah, many years. Um and I remember when I first started uh writing on LinkedIn and and producing content, I remember trying to understand like okay, who who can I copy from? Like who can I get inspiration from? And there was literally no one. There was you.

Yeah.

Mainly on LinkedIn and there was Karisse who hadn't, still has her podcast. Um so obviously you know I I bumped into you very early on. Uh and I remember reading your post still uh till today and agreeing with everything that you say. Which is incredibly annoying Brian. Incredibly annoying. Uh and many times you say before I before I have the chance to say that. Yeah. So uh no it's really a pleasure to to you know to have you on the podcast. We always have like great conversations. So this one we'll also share with the crowd. I want to start where you know we had we had a call I think it was like two three weeks ago. Uh I think you were on vacation since, I was on vacation since. Um and during that call we talked about AI. I mean what can you talk about in the fraud space uh in 2026 right? Um and and specifically we talked about the kind of like the adoption journey of AI, that we see in the market. And and I remember listening to you and you know a lot of the insights that you get. I think uh you know you've seen a lot, because of your role at at Safeguard event. Um and I remember that you you raised this very interesting concept and I think you kind of like distilled it very nicely. Uh, and I remember you saying there's a difference between AI activation and AI enablement. So maybe we can start there and and you can kind of like walk us through like what's the difference and and how you see it.

I think a lot of people are thinking about how can I use AI today. I'm very pro AI. I'm always looking for ways to make my life easier. I've this has always been my mentality. I've always told people once I get hired, I'm lazy. Full stop. So, I'm going to try to not, I'm going to try to find ways to automate my role. Try to find ways to say we keep investigating the same patterns over and over and over. What data points missing for me to increase rules? So, it's always been my mentality. Then it became like process improvement, setter level excellence. For me, I'm lazy. I'm just trying to find better ways to make my output, my mentality, kind of like be able to see the full complexity of whatever business or platform I'm working on today. So that's like the root foundation of all of this. So when it comes to like AI, it's a again it's a tool. And a lot of people don't necessarily know how to use it, don't necessarily have the time to sit down and use it. And it's changing so fast, you ultimately have to be unemployed to keep up with the different models. What do I use this for that? Do I just stick with one platform versus the other platform? Do I self-host? Do I use the cloud? Do I like, what do I do? It's very overwhelming to start. So I love the initiative for a lot of companies be like we want to find ways. We want to empower our our own employees to design their workflows of like, what do you hate? What, where can we use AI? However let's throw it over the fence and hope for the best. That is AI activation. That is buying licenses, getting tooling that's already like involved with your contracts, or tools that you already use. Just kind of like you could be handcuffed of like what you have. And you just kind of take the easiest path to say, we use AI at insert company. Then the employee doesn't really know what to do. They haven't been trained. They don't know the, I'm generalizing. I'm generalizing the person who doesn't really have the time that I, doesn't dedicate any any bit, and don't really have maybe the comfort or feeling of support to test, in house. So what does that mean? A lot of companies are unclear what can you connect it to? Can I connect it to my email? Can I connect it to Zendesk? Can I connect it to Stripe, Adyen, Checkout? Whatever platform you use. Uh because payments in our space is a lot of great indicators. Can I connect it to any other tooling? How do I connect it to the tooling? Do I need an engineer? Do I need a product manager? Can I do this? Do I have access to do this? What can I or can't I do? So, the first step is, be clear around the boundaries and constraints of your own policies internally of what can your employees do. I know some companies that can connect it to everything. I know some companies who can't connect it to anything. So then the power of what you're really going to be able to do, kind of changes what you can. It's just like automation or rule-based systems. You have data, you put those as inputs and then that will dictate the outputs you're able to get. If you don't have the inputs, you're can't get the outputs. It really kind of still comes down to that foundation of being able to have that data. So that is AI activation. Here's a tool. Go use it. We're giving you no guidance. You might get in trouble. You might not get in trouble.

And we take the box. We take the box for our board. Right.

Exactly. To say, "Yes, we're doing it. Yes, we rolled it out. Yes. Yes. Yes. Yes. Yes. Sounds good." Then you're going to get a couple things out of that, I guess, before we get to AI enablement. You're going to get people, some a couple power users. And they don't know if they're doing anything wrong. They might, and they might get in trouble. Then they're not going to be power users. You have some people who could be power users but are cautious and careful about what they do, what they connect, what they say. So they end up barely using it and going back to the other ways. And then the other people are like, well, the way I'm doing it works. Why do I have to change what I'm doing now? It's just change management right there. Uh again, if you throw over the fence, you're going to fall into those three pools. The first person, they might get in trouble, they might not. That second group which is your biggest opportunity to get power users, are cautious and careful and are not using it not using the full-time. And then that third pool of like they don't, there's no incentive. There's no reason for them to ultimately start using these tooling. The way that it works, it works. I've been doing this for 20 years, 10 years, 3 years. Doesn't really matter, it just works. I don't need this tool. And then what's AI enablement really look like. Comes from a lot of different forms. But really it starts with communicating your expectations, your policies, your boundaries and constraints. Being clear of what you can connect to, how you can use it, what you can put into it, what you can't do, what you can't connect to it. Just being very clear so you know what you're operating in as an individual and an enduser. Then that's one. Train them of the dos and don'ts of using whatever platform you are. This is platform agnostic. It doesn't really matter because they all offer a lot of this functionality and you need the data. Without the data, can't really do much. Then how do you actually get people to have time? There's two two ways to kind of do it. One, incentivize. How can you incentivize people to look for ways to play with this tool? To learn it, to test it, to build it, to push things into production, to build internal tools They have to be incentivized. It could be through OKRs. It could be through little hackathons where there is prize pools. There could be recognition. There could be uh just new channels spun up in either your teams, or Slack, or for you something else of like just show and tell. Of just getting the comfort of allowing people to test, build, and then show off. So you're incentivizing. Then the other aspect of that, where does the time come from? You have to be able to allow your employees and set dedicated time to ultimately talk about it. Show off things, or just have heads down time to just pick one process, or one report, or connect one data flow. To be able to actually do it. Because if you work 40 hours a week, and then in order to use AI they need to work 41, 42, 50, 51, you're going to lose a lot of people in that depending on the size of your company. So one, it has to be incentivized. Two, you have to have time. And three, you have to have boundaries and constraints and teach people how to actually use it. And then they have to all go together. And the final one, this is where usually any change management dies. Like this is an AI adoption. This is just change management in general. You have to let that run, see how it does, and then how do you reinforce and reassess. So, the feedback loop. How do you really make sure that this wasn't a 3-week blip of excitement? Maybe great for team bonding, maybe great for production for a little blip. But how do you carry that momentum forward? And ultimately make sure these new processes, these new ways to investigate, these new ways to build reports, these new ways to communicate actually end up being adopted. Again, it can be incentivized. Again, it can be built into OKRs. Again, it can be built into like monthly or quarterly syncs or reviews for if you want to call it AI task force. Someone to review across work or own your own team to say, are we making progress with AI? How and what are, what is the improvement impact? Be loud about that. That's how you ultimately how to get people bought in. Show off. Um, so those are kind of the ways that I think about the difference between AI activation and then how do you actually enable AI within your team?

That's great. And I I would ask, or I would say this. I'm guessing that the the organizations that are stuck in the act activation stage, uh I'm guessing most of them uh you can split into two categories. Uh one are the unaware. They simply don't know that it's not enough to buy tokens. Right? Uh and say to their employees that they have I don't know like a token badge. Uh they just they're not aware that you need all of these things. Uh the other group um are probably not motivated. Uh meaning they are aware that they are using AI as kind of like you know a veneer, uh you know. Um but um they are probably not motivated enough to invest all of these things. All of this effort that you talked about. And before we kind of like deep dive a bit more uh into this kind of like entire journey, um like how how do you, like when you speak to such organizations, how do you uh tickle their motivation nerves or whatever? Um and I'm guessing that like like there are two layers to it. There's like the personal motivation. Why should I do this change if everything works? Why should I actually like go invest in in AI? And also as an organization, what do we get as an organization from enabling AI, not just activating it?

I've coached basketball also for over a decade now that I can say that. I think about this with a lot of same mentalities of that. There's different teams. You have different players who have different personalities. One year you can motivate a team and get them bought in and kind of like giving their full effort with certain tactics. The next year those tactics might not work. So part of it is understanding who's within your company and who's within your team. And ultimately to get true adoption and to get true motivation, it has to start at the top. So when executives are using it and showing how they're using it and what they're connecting to, one, it'll give people ideas, but two, it'll give people this comfort level of, okay, they're doing it. They're not perfect. They're learning. It removes a lot of the impostor syndrome of like, well, I don't want to show this. People are going to make fun of me. I don't want to show this. People are going to think this is stupid. So it removes a lot of that and kind of creates a little bit of psychological safety within your company. So, one, set the the tone and goals at the top. If they're unaware, they don't know how to use it, then like uh what you're going to typically get is you're going to find people who are using it a ton and then execs are going to keep saying, "I want to be AI native. I want to be AI-pilled. I want to keep using more AI, more AI." But if they're not using it, they don't really understand it either. So, one, set the goal, set the tone at the top. Um that ultimately will give the direction and strategy of what tactics ultimately work us up into there. What incentives can we use? Can we use incentives as a company level? Can we use incentives on a team level? Does teams have budgets to incentivize? So, there's kind of like all these little bits. How do you want to kick it off? Make if you want it to be a big thing, make it a big moment. Company hackathon, uh company training session, team like yes, it takes time. Yes, it stops overall um in short-term production of the company, but if you're making a big bet on AI and you roll it out week, probably either going to take way longer to adopt or just fail. So, you got to make it a moment within the company to get people excited to get people bought in. That's still not going to cover 100% of people. Now, you ultimately need to get like who are your coaches and assistant coaches, how I think about it. That might be your managers, be your directors, depends on your team structure. But you're going to have certain like pods across all the different orgs. Even if you roll it out as a company, one team, here's what I'm seeing. One team could see that they're AI native, and then another team in the same organization will say, "We're AI aware", meaning we're not doing anything, but we should. So there's going to be a huge variance depending on the size of the company within, maybe not even depending on the size of the company, but huge variance between the perspectives of where they are on the AI journey. Just two teams apart. So proximity is close where you can learn from each other. So now whoever owns that pod, the group, a function, the team, you know your people on that team better. How can I motivate the majority of the people with one specific tactic? There's usually one motivation lever that you can get 60 to 70% of your team. Then there's going to be a couple of the people that like drag their feet, just don't do it. It's going to take my job. I don't want to use this. I don't want to learn this. I'm afraid. Um, so how do you motivate those people on an individual basis? Ultimately, the goal is, you will be typically have one-on- ones and meetings with people. Make it a moment to talk about, make it a moment to help people. How can you get the group bought in? And then who are the individuals that need a little extra love, who need a little extra support? And then how do you get that love and support for them? To ultimately make them feel comfortable to remove those fears, remove those hesitations, understand what is actually blocking them and work with them to remove those blockers. That's very hard to do at a large scale company. It's very hard to do when you're responsible for hundreds of people. So there are ways around that. But you got to kind of break up and segment your teams in the pods or whatever it may be to ultimately find what those individual incentifications and support systems look like to get people to just feel comfortable trying it out. Understand it, it isn't going to take your job. Uh the way that I think about it though of like, you're going to have the people who are top performers who are going to use this and become even top, even more top performers. You're going to have a bunch of people in the middle, who weren't top performers with this that will become top performers. Then you're going to have the people who probably are going to end up on pips performance plans anyway, and this could help them get out of that bucket to average performer. And then there's going to be people that drag their feet that they were underperforming to begin with, that they're going to be underperforming here. Pre or post AI, that's an entirely different conversation. But I see AI as a tool to either speed up those processes to learn to move people up on that performance chart for yourselves. Of like the quality of output. And then the transfer of knowledge is a huge component to this, of the transfer of knowledge. What are our 1% on our team doing? How can we leverage AI to feed that back into the people we are figuring out our own upskilling plans for them? How can we leverage AI into this to move the average group? Which that's where most people fall in in most companies. And average means something entirely different at every spot, every team. I'm not saying average to be average, just that's math. Of how it the the bell curve works. And then how can you move people up on that?

Yeah. I I love how very quickly you kind of like draw the line from like a technological adoption effort, right? Well, we're talking about core technology. Supposedly, we should speak about should we use Chat GPT or Claude? Should we use this model or that model? How many like what's the token budget per team member? But very quickly you kind of like go, and kind of like connect it to a transformation journey. And then very quickly into people management, uh into people in general. And it's fascinating to see how a project that seems like a technological project first and foremost, is actually first and foremost um a people management project. Maybe all projects are. I wonder though what happens, or I'm guessing that you also meet sometimes, I don't know maybe maybe you don't. Um leadership teams that themselves don't really understand why they should, or what's for them to gain other than say yeah we we do that too. Um, what like when you're thinking about teams that actually enabled AI, and went very far with it, how would you kind of like present their gains, their kind of like the fruits of their efforts to these skeptics that say, "Yeah, AI is another hype train and it it will pass and it will be something else. I'm not going to invest time and money in that."

It's a tinker tool to me honestly. Once people start to play with it, it allows a lot of non-technical people, not going to point any fingers. But nontechnical people get over those hurdles of like well I need product, I need engineering, I need a fight for roadmap, they don't understand this problem. Uh there's a bunch of different use cases, like the one that I talked about a lot of. Like if I can build a prototype, it's going to be a lot easier for me to communicate with product and engineering what I'm trying to communicate. And a visualization of like this is what I envision, this is what I'm thinking, this is the portal layout, this is the functionality. You can build all of that without even connecting it to any data. Let's say you're in one of the companies that have don't connect to anything. You can still design your workflows. You can still design the what you want to work out of. You could still design reports. And then hand it off to somebody. That's going to be a lot easier of an ask. It's going to take a lot less mental load, for whoever is sizing that from an engineering scope, to ultimately say, "Oh, I thought this was way harder looking at this. We could do this." And then that becomes a lot easier to get dedicated resources consistently through it. So again, that is just an example of using AI as a tool for communication to different teams of how they think and see differently that you're always negotiating with of, I need more support. There's never enough engineers. And even with AI there's still not enough engineers. There just maybe some engineers that their output is getting insane. Maybe their token usage as well. But that goes into the trade-offs and budgeting and some visibility into like the economics behind it. I'm not going to speak on the economics today. Uh but it is another lever into all of this, of like the the equilibrium across teams of like what is the right usage levels for everybody. But I mean that's one way that I kind of start to think about it. [Ad Break (25:53): Hey folks, I want to take a quick break to speak about today's sponsor, me. If you're finding this useful, do me a small favor, like and subscribe. It really helps with the algorithm. And if you're not already on my newsletter, The Saturday Fraud Strategist, you should definitely check it out. Every Saturday, I break down fraud trends, real cases, and strategies from my own experience of building and operating fraud prevention systems. Whether you're new to the space or a system practitioner, I'm sure you'll find it interesting. Check the link in the description. Takes 2 seconds. You'll thank yourself next Saturday. Now, back to the video.]

You know, I've seen so many risk teams. And you know, obviously I've led risk teams. And it's always like risk teams always have so many ideas of what can be done and so little budget and resources, because risk usually doesn't grow the business. You touched a very interesting motivation lever that was right there. And I never kind of like articulated it to myself. So I I yeah I love it. Um let's talk a bit about, so okay so we talked you know generally about activation versus enablement and kind of like how to get the the team there. When you look at teams today, and you look at the journey, and where they are on that journey, like how do you see that. What are the like common steps that you see teams go through when it comes to AI enablement?

I know we were just talking about this. And I gave you a similar answer a couple weeks ago. But even from the conversation, so as like I kick off Safeguard, I talked to up until the event, I'll talk to about 200, 300 practitioners, senior leaders who are thinking about this. And I talked to everybody from anti- AI to I don't want to imagine life without AI. So I see the full spectrum of all of this. And how I, when you asked me this as we're prepping for this call right before I hop on recording, I kind of changed my answer to you. Like it's still similar, and I still got five stages, and I'm going to lay them out, and then you kind of like poke at which one you want to explore. We can explore all of them one by one. But um as I'm kind of talking to all these people and getting a better understanding of where they sit, I kind of see it in five buckets. One is AI blocked. I just can't do it. It could and this is like a honestly a catch-all category. It could be we don't have the data, we don't have the tools, we don't have the supporting company. Uh just for whatever reason, we just can't start it. They could be aware, they could be unaware. So it doesn't really segment between that and just for a multitude of reasons, just can't use AI. AI aware. So this will be companies who are a little bit earlier in their journey. Of they're learning, maybe experimenting, maybe thinking about getting tooling or deciding between tooling, haven't really started using it yet. Um really haven't rolled it out across the company, but are learning and educating themselves of we should probably be using AI. How could we be using it? They're going to be starting to answer some of those questions, pre-tinkering. And then AI enabled. So I think about this on a company level. Honestly, it can roll out to like a team level too. But I think about it on a company level. Where AI is not really everywhere. Not everyone's really using it. They might be having a little bit of a challenge with adoption, or usage and enablement. But they are using AI somewhere in a specific use case across the company. So it is being used. It might be a small use case. It could be a large use case, but is being used somewhere specific, but not everywhere. AI first. I think this is where most people, especially companies who are probably realistically older than 5 years want to be. AI first. So it's really kind of becoming core to product, core to decision-making, core to team structures, core to processes and programs. So I think this is where most people want to be. You're going to probably find most people today in AI aware or AI enabled for the most part. I think we've in the last 12 months moved a lot of people from AI blocked to AI aware and AI enabled. But I think most companies' goals today, just due to the structure and the tooling the legacy when companies are built, AI first is where I think a lot of people ultimately are setting their goals today. We want it core to everything. And then there's AI native. This is the company is built entirely around AI. Really, it's not many companies have like redone the entire plumbing to be entirely AI native. These are going to be typically more of your companies who have launched in the last couple years. Uh so they don't have as much legacy tooling, processes, people, products, onboarding, etc. So it's a little bit easier for them to build the entire company around AI. So those are the five parts of the journey that as we speak today, like if we speak in a month, I might have either another journey. Bucket them. Uh but as I answer this today, we're recording on September 1st. Can't believe it. We're already in September, but as of we're recording September 1st, this is where I sit today with um where company AI journeys are in the five stages.

And tell me, you mentioned that within these five stages, you perceive the market mostly to be and when I when we say the market, we mean fraud teams, right? And fraud organizations. Most fraud organizations sit somewhere between step two and step three, AR, AI aware or AI enabled. And would you would you think that the kind of like the normal fraud practitioner out there would answer the same? Would they also say about themselves that they are probably somewhere there and that most of the market is also there?

It's weird. Um I just from conversations with people I'll see what they're saying, or how what they categorize themselves, and then what they explain is different. So and it goes in both ways. Some people are underestimating where they are and some people are overestimating where they are. I won't say everybody's overestimating. I won't say everyone's underestimating but people also aren't clear with this of like where they actually sit. So I think people this also goes into the learning of it. Of like what really is the adoption AI journey stages. What are the methodologies that should be transferred from company to company? Like there really isn't a true standard yet. So like I might categorize this as like this is where I see the five stages today. Someone else might say you're wrong. These are the stages simpler blah blah blah. Uh but the way that I think about it today is most companies, and this is I think different than from 12 months ago. I think a lot of people were AI blocked for a multitude of reasons, uh to use it. I'm seeing from conversations more people are falling in tier 2 and tier three. Probably a little bit heavier in tier 2 than tier three today. But I am feeling that there's more companies in the next like six, I won't even say 12 months, 6 to 9 months that will move up to that uh third tier. Just from like what their plans are. What they've been sharing with me and what they want to execute on. And if some of these companies can execute on their own road maps and get that delivered, that would bump them up from my perspective to that tier three. Uh but I think there's definitely a lot of learning, a lot of experimenting. The space and the tooling is moving so fast. Um if you follow any of the AI bubble, um there's a lot of people who stay focused as like are in the AI bubble. Like I am. I know I am. I know I'm pro, but I'll have conversations sometimes where someone will be like what is Claude? What is an agent? And I have to take a step back. Sometimes it's different. Sometimes it's just a conversation outside of the industry that aren't necessarily in tech. Uh but I think it's still cool that people are trying to understand. And just shows that the widespread variance of where people actually are today on the AI journey. And there are some companies that would surprise you that are higher. And there are some companies that would surprise you that are lower. And I think that also goes with like fraud protection as well. You'll learn about some companies being like, I would have sworn you have way more in place and way more mature on your fraud journey. Same thing goes with AI.

Yeah, it's interesting. Because I mean I I see it the same and differently. Meaning, and I don't know what you what you would think about it. And maybe I'm just you know a cynic. But the the way that I explain this uh this kind of like this this state where teams at the same time are um overconfident or overestimating and underestimating themselves. Is that on one hand when you would ask teams, um like are you adopting AI? Um most teams would would overestimate themselves. They would think they are AI enabled. Where in the best case maybe they are AI activated. Um, but I think that when you would ask them, not necessarily straightforward, but when they would need to kind of like rate themselves against the market, and when where they are in this journey compared to the rest of the market, I think many teams experience I don't know if FOMO or an imposer syndrome are the correct uh.

I think FOMO appropriate here. Cuz if you just log on to any social media to try to read anything, one, there are people that are well beyond everybody. And then two, there are people who just take those and then rewrite it even though they don't do anything in it. So then there's more of an echo chamber than people actually doing it. But if you're trying to research and learn that echo chamber becomes dangerous. Because it's like, am I behind? Am I so far behind? And if you're caring about this today, you are ahead. And you're doing the right things to educate and learn, how can I better use this for my own personal career to put myself in a better position in 12 months from now? So I am a top performer in my company and in the industry. So if you're thinking about this, if you're researching and trying learning, if it's hard. Honestly, this stuff is confusing to everybody unless you're actively building it. Like inhouse at one of the frontier labs. Everybody else is on a learning journey of understanding this and how to best use it. I will admit my simplicity sometimes. My best way to learn it is say, "Hey, Mr. LLM, Mrs. LLM of your choice. How can I do this? This is my goal. I don't know if this is possible." And I like to use voice dictation because sometimes I'll rant and ramble. I'll give it a lot more context. And I'll just sit back, and I'll just kind of like ramble. And say, "These are my goals. This is what I'm trying to do. This is like the data I have. Am I missing something here? Can we do this? What's the best way to do it?" And then I'll go back and forth a little bit with my LLM. And then ultimately from that I can come to a decision of, is this possible or not? And then if it is possible I can probably build it with my LLM together uh of using it. And then this is where the different types you have like Rockbot, uh more of like a personal agent. You could have co-work or ChatGPT work. Where it's like a little bit more non-technical for more everyday users to like implement and automate more in your work. And then you have Codex and Claude. These are kind of like your two main platforms from OpenAI and Anthropic, of like what you're more comfortable building, being a little bit more technical. But as a non-technical person, just being able to use either voice dictation or just typing out, and using your native language to ultimately achieve the outcomes that you're trying to accomplish.

So, okay. So, so far we've been kind of like high level bird's eye view of uh kind of like the industry, and the state, and kind of like the state of AI adoption, and so on. Let's get a bit more kind of like nitty-gritty and really give some uh um concrete examples of how all of this would look like specifically in a fraud team. Because a lot of what you said I'm guessing that you know would apply to to any any kind of team. Yeah. And you know starting at the bottom, like how would you know, like how like going through each of these stages. How do you recognize and self kind of like identify that this is where you are? What is probably blocking you from going over to the next stage? And what is likely, or what are the likely things that you probably need to do or think about to solve these blockers? And I'm guessing that you've seen you know like live examples or real examples of on of all of these stages. So I'm guessing that you can, without naming names, give some some concrete examples.

Yeah I think um the way, the blockers. Let's talk blockers. They're people, policies, data. If you don't have data, not going to really be able to do much. Policies, if you aren't clear what you can or can't do, people, especially in our industry, tend to be a little bit more cautious on taking risk. Naturally, I think that's a good mentality. But if you aren't clear, people individuals are typically going to be a little bit more cautious in pushing boundaries of what they can or can't do. That won't be everybody, but it'll probably be just due to the nature, and behaviors, and psychological factors in the industry we work in. Will be safer. And then people. People goes into the executive setting the tone. People goes into people not wanting to do it. People go into just being afraid to show off what they're doing, to get feedback. So those are going to be your your blockers, ultimately to all of that. Uh and then all that goes into tooling and budget. Uh but tooling and budget. Tooling can kind of go into data. Budget goes into people assigning the budget and winning them over. So I kind of lump those into the data. Uh because data could be tooling in itself. It could be buying data. It could be your own data. It could be some hybrid of the two. Uh so those are going to be your obstacles. And each one of those has its own way to ultimately kind of like overcome those. I think we've kind of hit a little bit on those already earlier. So, I'm not going to go deeper into that. Uh once you get past the obstacles, then that will give you a little bit more of an understanding what you can or can't do with building, what tools you have. It could be a third party vendor. It could be the LLM directly. It could be something self-hosted by your company. There could be that once we overcome those obstacles, you'll have a clear understanding what you can and can't do and what tools you're actually going to be using for it. Um, for this exercise and me talking through, I'm going to be talking about the LLM uh specifically again platform agnostic. This can be done across any of the platforms, but platform agnostic. The mentality. Now it's mentality. After all this now, it's mentality of the individual could kind of go into blocker but I put this in a different spot. A lot of people think when, especially in our space, when conversations, when I want to do something, building agents, I I should go big. Transaction monitoring, KYC, all wonderful use cases. All ripe in opportunity. So the mentality of this is my biggest core of my work, we should start here, that's going to be more expensive. It's going to be a lot more risky to roll it out in production with customer facing. It's going to take probably additional tooling, something you're probably not going to be able to build in house right away, with um the tools where you have. So I think it for me that's like where you work up into. Or if you already have a solution provider that you have an agreement with. They might have functionality that you can work with them. Either from like a customer advisory spot of like we're your customer. We want to do this. We don't want to build this in house. We're already buying and paying for you. We think you could do this. Could we build this together? A lot of companies are going to probably take on that conversation. It's mutually beneficial at that point. Or there are vendors who have already rolled out agents or sell agents. So then like it's a build versus buy at that point. Do we need to build it?

Yeah. No, I'm sorry to interrupt. It's just so interesting that you're raising this point. Because I think in reality what happens is exactly the opposite, right? Meaning the like this the teams that are early in the journey, and are just starting out with AI, are one very excited. And two, they are unaware of the complexities. And then what happens is that they think they can do that. They think they can build. They think they don't need to buy. And actually this like we see a lot of teams that are just, you know, uh you know just starting this kind of like build journey. And invest a lot of effort and budget into it. And then fail and then you know it kind of like reflect uh on the entire kind of like AI adoption journey. Oh we we kind of like we failed. While the journey itself is not the issue. It's just the strategy.

Yeah. Now, I think you know where I'm going with this one. And the it's unsexy and it's boring, but it's been a major component of my career of everywhere I go. Is I'm assessing the entire customer journey. I'm assessing the different workflows. I'm assessing the different processes, internal and external. And that workflow design, nothing to do with AI, is ultimately giving me indicators and bets for where can we actually use and adopt AI. And the internal external piece is where I start to kind of like help people communicate like where do I start? You could start anywhere. So this is just my recommendation because I want people to build up AI literacy and AI confidence. And then kind of like team camaraderie around it. So those are the three pillars I'm honestly working for. So if you start big, you might get people a little bit frustrated. You're gonna it's going to take longer to see like outputs. And you're going to probably create a little fear with your team of, oh if we do agents here am I going to get fired? And then that's people you're going to start to pull away. So where I usually recommend, again those three little pieces, of you you've done the the unsexy non AI, you could use AI honestly to do the workflow of help insert my URL. Help me walk through what onboarding look like. What fraud risk do I have? You don't have to put in anything at that point that is unique. And keep it all external. And just get a sense of what would use, that as consultant, what would a consultant kind of assess our business risk profile and our different workflows? Help me prioritize the impact, blah blah blah. And you're kind of working through this logic of, okay, help me map my company. Help me map the work to our risks. And then from there, now you have lots of different pockets of opportunities for AI. It could be a report, it could be an alert, it could be an escalation, it could be rule assessments, it could be surface area. The simplest one I do is, and again, I have a little bit more uh clarity in what I can and can't do in my own role. Uh, and this isn't fraud specific, but it helps me show up better, one for my family, show up better, two, for my company. I like a lot of people, you get out of bed, you open up your phone, you check what, who texted me, what emails, what's the weather. Before I know it, I'm on my phone for way too long. So, every morning it sends me a message at 7:30 a.m. uh local time, and it will send me what my calendar is, a little note, and then the weather, and I'm just in one. I don't have to open up my uh multiple tabs. I don't even have to go into my phone. I can look at it, phone away. I have kids. It allows me to be more present, get out of bed a little bit faster, help with breakfast, help with a lot of different things in my house. But, it allows me to be prepared of what does my day look like? Is there anything outstanding? Are there any emails or Slack messages that are higher priority? Or I owe someone a response. So it gives me little insights to that so I don't drop the ball on other aspects. I build a lot of little things like that. Um so that's one. And then I use it for like a lot of pattern analysis of I might have a hunter hypothesis. So I'll take the data again I have control of putting and clarity that there is some data that I can put into my LLM. Uh so I can use that specific data to look for patterns or trends for the work that I do. And then either create into visualizations for other people to use or for my own reports. I can upload specific um data, and put like one create a one-time internal local cue for myself to work through of the work that I need to do. And then it's just dead after that. It's not it's self-hosted on my computer. There is no uh URL. It's just only on my computer, but it helps me do whatever that mini project is. That I don't need to build custom software around, but helps me stay focused and productive. So, pattern analysis is always a good one to start with. Internal tooling or reports or executive reports or your power like presentations, building out use cases, aligning uh OKRs. Your OKRs to other companies OKRs. Fraud team usually combats a lot with sales and marketing and product. So, help me better align or create my OKRs that I can be an ally to marketing. So I can be an ally to sales. Not uh competitive or seen as a constant blocker to sales in marketing. So I can not just be here's Brian to tell us no, fraud's just a cost center. So how can I work on my own presence? All right, I'll pause there. I think you got the wheel spinning. I just rapid fired a couple different use cases. We could also talk about uh teams doing it for like chargeback representation, and building their own compelling evidence, or sorting through all of these reason codes, and getting clarity around what are the true buckets based on the data we have, and having better visibility faster visibility into some of your chargeback and fraud trends as well.

So I think there's an interesting distinction to make here. And I'm wondering like whether like did we, are we still at AI aware? Which is phase two. Or are we already at phase three? Which is AI enabled, in your mind.

If you're doing that, that's AI enabled. Those are specific use cases. If you're using that, and you have those running on a recurring basis, not a one-time little project, if you're doing that on a recurring basis. Some of these could be monthly, some of them could be weekly, something that could be daily. Again, I don't need to do, when I was working in my roles, I didn't need do chargeback reporting every day. I needed to be aware of chargeback trends daily, but I did not need to report out unless anything significant on a daily basis. So, I could build in my report creation based on my data collection, and help me find better segments, and take feedback from other times, so I can create a easier to comprehend uh report. So if you're doing this on a recurring basis, that would be for me, enable. You're using it for a specific use case, on an ongoing basis.

I think I'm starting to to grasp uh how you see this model. And I and I want to kind of like you know like uh uh play it back to you and see if I got it right. So AI blocked, first phase. Okay, you're not doing anything with AI, or at least not anything of importance. Uh maybe you're taking

They can't, for whatever reason. I think you just can't. You don't have the tools, you don't have the comfort to try them, you haven't tried them, you don't have the data to do anything. Uh you just can't. And there are people who are still in this uh bucket. They might want to. They might not know even about LLMs. Those aren't really um, don't really matter for this bucket. Just aware, or unaware, just can't do.

Yeah. Then we get to the two two later stages. Which is AI aware, phase two. And AI enabled, phase three. And if I if I got you correct, the difference between how you use AI would be as follows. In phase 2 AI aware, you're using AI, but you're not using AI for uh like fraud specific uh tasks. You're using it for productivity. You're using it to just be a more efficient employee. And that doesn't really matter if you're a fraud analyst, or a developer, or a marketing specialist or whatever. While in AI enabled, you're already starting to tackle the first actual kind of like use cases for automation. This is not no longer productivity. This is like really like taking processes and automating them. Uh like fraud processes and automating them. Whether these are reports or these are whatever labeling or classification work or whatever. Um is that is that how I should look at it or did I miss it?

I think pretty close. The depending on productivity, could make you higher leverage and focus on other work. I think it it comes for me the consistency. So productivity could still be user case. Especially if you're a program manager, a people manager, depending on your work. The admin work, and that type of work, can help you focus on higher leverage work. Um AI aware, for me, it would be like they can. They just haven't. Or they've did it once and haven't done it. They might do it ever so often, but it's not the same task.

Got it.

More of like that phase. Of maybe start, stop, start, stop. I'm not really doing anything. I want to learn how to be better. I'm trying to consume information. I'm trying to learn what other people are doing. But we really haven't consistently done anything with ourselves. So, productivity could be like the start stop. I did this one thing, I don't really use it. I tried it, but we're not doing anything. Uh if you build out the use case specifically, could be admin oriented, it could be escalation oriented, it could be research oriented, it could be assessment oriented, it could be transaction monitoring, it could be like detection prevention and monitoring uh oriented. Those are use cases. So if you're now tackling one of those, but not all of those, that would be enabled. Like we are using it for a use case. We're using it consistently and it's helping us. We are going to continue using it like this. That would be enabled. We're solving a specific use case. We're not solving maybe all of our use cases. Uh and then uh, that's now as you start to compound more and more onto that. And now the mentality goes into like our decision making, our escalations, our when it becomes more core to all of that, that's now when you would get to the next tier from there. And then AI native was like everybody's, like everything is built on the foundation of AI. Not necessarily pre AI.

So at which point, at which of these stages, like I'm thinking there's usually with uh with technology that you can do two things, at least when we're speaking about automation. One you can take an existing process and automate it. I'm doing, uh I don't know, like whatever. I I'm I'm reporting something weekly, now I can do it daily, and I also don't need to spend time. Okay, so that's straightforward automation. But there's also another layer. Which is really to enable completely new processes, completely new capabilities, completely new efforts. Uh the the the use case that I always like to harp on, is you know like uh fraud operations teams that don't necessarily have data analytics, uh skill set or experience. And you can use uh and and can use AI to do data analysis, write rules and so on. Would that be in AI first or is that AI native?

Not AI native if everything's not built on it. Uh if you're kind of like building into that functionality, for me that would be AI first. AI native would be you've redone your entire function. And everything operates within AI, uh foundationally and into all of your processes. Decision- making, everything sort like that. AI first, that would be a good example of AI first. Assuming, well if you get to that point you've probably also have done other tasks. Have built out other functionality, other automations, built out agents or research. So my assumption is to get to that point you've probably done other work. So that would for me bump you up to AI first. Now you're thinking about, okay, we've opened up more time, more productivity, more output, higher leverage work. Now we're looking at pushing our team forward. Now that would be for me classification and AI first. I'd imagine most teams aren't going to start with that mentality, of like that specific example. So I'm going also on the assumption that that's not the only thing they've done. That's the only thing they've done. Then that would be enabled. Like you're probably going to make good progress. It is a specific use case, one specific use case. You're not really adopting and evolving it through your entire function. But if you're starting there, you're probably going to speed up pretty quickly through other aspects. But in that snapshot in time, kind of just depends. So my assumptions if you're doing that, you've probably have done other things, and now you're kind of pushing your team forward. You have more comfort, confidence to create new systems, create new processes, use new data or data you have in different ways. You're probably moving along in that journey. Based on kind of like assumptions I would make just from conversations I have and how most of the markets kind of think about it. Most people start with what's what's familiar or what do I hate doing? Most people start there. So just due to normal human behavior, my assumptions that I'm making based on your example, I would put that into first.

So while we're on this point, maybe my last question. So we just went through the entire kind of like uh uh transformation journey and the five steps that you recognize uh different teams going through today. Would you see the same journey um as one who is relevant to individuals? And specifically not like human beings in general, but specifically to fraud fighters. Should I as a fraud fighter look at my journey with AI in the same terms and in with the same phases?

Probably similar phases. Uh probably label it differently. Uh realistically, you'd probably label it slightly different, but the overall sentiment would be um probably pretty similar within the usage, adoption, understanding. You might go a little bit closer instead of like AI blocked, AI unaware, aware, you might go through a little bit more of that traditional of like product discovery phase, would probably be closer to that journey. And kind of like progress through there. The only thing I would add to all of that is, control aspect of like what do you actually have control with as an individual? And what do you want within your own career? And is this a tool that can help get you there? If it is and you believe in that, I'm pro. Again, I'm my biases is yes, which is why I spend and dedicate time weekly if not daily on trying to do something. Um, my challenge for myself, I use a specific LLM. I know what I can and can't do on it. Uh, my goal is to never leave that LLM. So, why when I have to leave that LLM, why do I need to? Is there uh data I'm missing that's not connected? Is something broken? It doesn't quite work. That manual is still better. Uh, is there something that just I can't get the data into it? So for me, when I'm thinking about my own workflow design, my own system design, my personal challenge to myself as like a mental checkpoint is why am I leaving this right now? And do I have to? And sometimes the answer is yes. But when the answer is no, then that's change management, that's behavior management, or that's getting access to data. So how can I improve my process so I can work within one tab and I can create moments of my own decision. It's not necessarily uh finding ways to offload to agents and automation LLMs. It's where should I be spending my mental load, my human intelligence and why do I have to leave another tool to get access to making a decision. So that's my own personal challenge. So controlwise always go on the assumption the company is not going to give you enough time to educate. Carve out an hour a week, 3 hours a week, whatever you can do consistently and just get into the tool. If you can't, if you know you can't do things or can't upload data, fine. Work on a use case of designing a report of what it could look like. Use dummy data, use consulting of like assess my fraud risk risk and help me gauge what AI could be here. Like there are still ways you can work these tools to get comfortable like AI literacy, AI confident in the tooling. So as you play more, it's going to give you more ideas. Can ask it give me ideas for other ways I could be using you. I just used Grockbot as one that's kind of getting a little bit of internet momentum right now. And when I first got on it, I was like, these are my goals. This is what I've struggled with other tools. Is this something you can do? Help me understand what agents that is a way for me to actually be able to build this. And then through a little bit of back and forth, a little bit of like I just have it interview me. It then repeats back and says, "Here's what I think you want to do. Is this correct? Here's what I would recommend." And then we're just kind of going through that process. And then it spun up agents to ultimately solve the goal I was trying to accomplish. So if you don't know or if you just are stuck in this mentality of like I don't know how to push myself yet, just ask it. Ask it to challenge you. Ask it to push you, ask it to roast you. Like tell it to do that. It won't necessarily do it on its own, but if you prompt it to do it, then it'll allow you. And I also always recommend uh doing voice dictation. You're going to say more, you're going to say it faster. It doesn't necessarily need to be a perfect prompt. Now, as models advanced, it doesn't necessarily need to follow specific prompt structure guidelines. Yes, it can help in some scenarios. In some other scenarios, not necessarily. So, my personal preference in those is just hold down, I have a hotkey. Hold down the hotkey, talk for two, three minutes, and say, "Help me figure out this mess. What am I actually trying to do?"

100%. Um, and by the way, I'll mention since you didn't. That Brian over the, you know, better part of a year or two now has, you know, posted uh, more than a few posts with several examples of prompts to do exactly the kind of things that he just talked about. Like how to do, you know, like threat assessment of your business, how to um, like basically ask questions that you would ask a consultant. And and I think these can be like really great starting points to to get into not just using AI, but also to use AI in the context of uh fraud prevention or fraud fighting. Uh so yeah check check uh check out Brian's uh LinkedIn. Uh he has he has these posts peppered out throughout the the last two years I think. Um

Also we're going on five years now. AI specific, definitely not five years but uh I've been writing now. Yeah. Yeah, those um started for myself probably honestly about two years is probably accurate. Like really trying to challenge myself to learn it. I saw this as, I see this is where work is going, the future of work is going. I don't necessarily know will this, what we're experiencing today, be what it's like in 12 months, 24 months. I don't know. But I do challenge myself to find as much time weekly if not daily to understand where the market's kind of going. And then understand what does that mean for my current role. And then what are other aspects or skills that I can use this tool to teach me. Again for like uh one that came up in conversation, this can be my last kind of example today, is I will admittedly I know SQL is something that has always been a job wreck for my entire career. I stink at SQL. I can reverse engineer it. My goal was always, I know I need probably six well-written SQL queries for me to do my job on an ongoing basis. Now, I need to know how to reverse engineer those. So, I would have people write those for me. Build my allies, get help, have them write me better queries that I know I just can't do. And then learn how to reverse engineer them. Well, now with AI, I don't need to have that ask. I don't need to waste my own human capital internally for these low-level ask for other people that I can get up and running with these queries that 12 months ago probably AI couldn't have done this well. I still couldn't do it this well. I can select all with the best of them. But really writing a solid query, not that that was never my skill. And I was always honest in my interviews. And some companies opted out with me. But the companies that don't necessarily understand how you can make an impact, especially as a first fraud hire. Like SQL wasn't necessarily 80% of my role. I could get by by knowing how to reverse engineer. AI now allows me to write way better capable queries and then teaches me what they're doing. So now I have a better understanding of what am I actually doing, why am I doing this and what are different aspects of this query that I can alter or do I even need to alter? I can just go back to the LLM of choice. But for me, my way that I feel like I'm at least stimulating my brain and learning is I will ask it to reverse engineer or teach me what it's done versus just entirely offloading it. Sometimes I can't. Sometimes I'm just run it. I need it. Let's get going. I don't have time.

Yeah. Choose your battles. Yeah.

But um. I do this is one way of like people, one argument with AI anti AI work space is your intelligence is going to go down because you're going to offload all the thinking. For me it's about designing the workflows understanding where my brain is most used where it's most needed. And focus my mental loading capacity at those points. There's different skills that I'll learn and develop all along the way but this is a tool to speed up my own personal learning for different skills that I probably would have never taken the time to learn, honestly.

Yeah, I agree. By the way, I don't take the time to memorize phone numbers anymore since uh you know, like smartphones and so on. I'm not stupider because of it. But yeah, anyway, uh Brian, this has been fantastic. I want to kind of like go over some of the takeaways that we went through in this conversation kind of like quickly. Uh we started talking about uh AI activation versus AI enablement. I really like your take and I think I kind kind of like captured four points that you mention that the the four things that you want to have in place to not only say that you have AI. Uh uh you know like budgeted and and and part of the company, but it is actually enable it's actually like bringing uh value. You mentioned four points policies um in terms of what how what do you connect to how do you use it and so on. Training people you actually want to train your people to use it how to use it when to use it and so on. Thirdly, uh you talked about budget. Both in terms of like incentivization but also in terms of time. I mean you know the the the the the resources in the end the time that you have to invest in AI, either it comes on the expense of your personal time or the company needs to budget time for it. And fourthly, you talked about uh the process uh and making sure that this is not just, you know, like um an announcement or a hackathon or a one-time kind of like thing. You know, we have like the AI month, but there are actually processes in place where we are running this iteratively and with feedback loops and all of these other terms that we fraud fighters love very much. We talked a bit about how to instill motivations. Um, on the kind of like team side, we I I really like the fact that you mentioned how it's the leadership's uh job. And this means that a they need to kind of like be first. They need to be front line with a adoption. Uh they cannot just say, "Okay, you get the tokens. Now make something out of that." You need to like really show and put yourself out there. Um, and the second thing is that this is, you said it so many times throughout the conversation. I cannot agree uh enough. This is uh a a change management process. This is a transformation process and this must be a very carefully managed process. Um, which I highly highly agree. On the leadership side, we also talked about the fact that AI is, especially for fraud teams or for risk teams, AI reduces the, or can reduce can reduce, the historical dependency that risk teams always had when it comes to external resources. Uh starting with engineering but not only engineering and being able to do more uh with less. It's uh you know this uh uh it's like a gag reflex sentence uh at this point in time but it is true uh at least when you frame it like that we talked a bit about the, not a bit we talked a lot, about the journey and how you see the journey and the five steps in the journey. AI blocked where either you're unaware of AI or you're aware but you're like you're physically blocked and you're not doing anything with AI yet. Then you have AI aware where you are starting to experiment with AI with very simple use cases. But mainly I liked your take you doing it inconsistently. Maybe you've done it once and you didn't come back to that. Uh as we talked about before uh you're not enabled yet. Uh and maybe you're doing it half-hazardly. Sometimes you do. Sometimes you opt to just do it the old fashioned way because it's faster. Uh at least right now. We, the next phase would be AI enabled where you're starting to do things consistently. Uh more and more uh use cases maybe especially around automating processes that already existed. The fourth stage stage would be AI first. Where not only that you're doing it consistently and across multiple use cases, but you're also starting to build new capabilities and you're pushing your team to do more. This is not only do better. This is not only to do faster. This is not only to do uh with less. This is really about doing new things that AI enables. And lastly AI native. Where the transformation has been completed. At this stage, this is not only about what you're doing, but it is also how you are set up. How you are organized. It's how your team looks like, it's how your platform looks like, it's how you how your budget looks like and so on. Love it. And you did mention one thing and we kind of like we went through all the phases but I think that uh is a very very sharp point. And it's probably it's probably somewhere between phase two and phase three. And that is to start small. And a lot of teams I think maybe are excited, maybe they have this FOMO, maybe they have the uh imposer syndrome. But basically they feel this pressure that they need to build in house, and they need to go big, and they end up biting more than they can chew. Uh, and I've seen it plenty of times and if I've seen it plenty of times, you speak to more practitioners about this topic than I am, then I'm sure you've seen it dozens of times. Um, and I I don't know if it's like a build or buy uh question necessarily, but it is definitely like how do you start, and how to do pilots, and how to do proof of concept and so on. Brian, it's as always it's been a pleasure uh just hanging out for an hour and a bit. Uh and also talk about my favorite topic AI and fraud.

I think it's, obviously everybody's talking about it, everybody wants to learn, but things like this is what's actually going to move our industry forward. Actually having conversations, actually letting people understand that there is this whole journey to AI adoption and using and being more comfortable and literate within AI. Just because you read something on LinkedIn, just because someone gives an example, just because you go on other social media platforms and look at them, doesn't mean you're really far behind. If you care about this stuff and you're putting in an effort to learn how you can better set yourself up and your company up and you're taking control in your own self-education, it's a wonderful time uh for you to actually speed up where you are on that journey where your own comfort confidence is. When you look at the general market, everybody underestimates where they are. In the general world, if you're caring about this, you're probably in the top 10% of the world. Uh just think about that within numbers of what you actually where you actually put yourself within the tech bubble. I understand I understand we hear a lot of things but in the general world general population you are ahead of way more people than you think and realize just keep going keep learning and find new ways to make the future work easier.

Absolutely. I uh I think the echo chamber point is so sharp. I uh I think fraud fighters might uh might think that in order to be a fraud fighter you need to be bald, bearded, and wear black. Uh but you don't. You don't.

Uh, it's a good start. I will say I'm biased again here, but it's a good start.

Yeah. Uh Brian, thank you very much. And I want also to thank our audience. And I'll see you next Saturday. Episode Title: The Rise of Agentic Fraud Ops, part 3: Scaling Fraud Analytics Meta Title: Scaling Fraud Analytics in Operations: AI Governance Meta Description: Scaling fraud analytics is the AI governance move most teams miss. Breaking down fraud team restructuring for AI, and the teams that must shrink or grow... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/scaling-fraud-analytics Keywords Primary Keyword: scaling fraud analytics Secondary Keywords: AI governance in fraud operations, fraud analytics team structure, agentic AI oversight, silent pipeline failure, fraud team headcount planning, automated fraud pipeline monitoring, fraud ops vs fraud analytics, AI transformation in fraud teams, fraud team restructuring for AI, model drift detection, fraud analyst role evolution, root cause analysis in fraud systems, KPI monitoring for automated systems, fraud strategy and risk appetite, data science in fraud teams, rule writing automation, fraud team budget planning, fraud team org design AI governance in fraud operations fraud analytics team structure agentic AI oversight silent pipeline failure fraud team headcount planning automated fraud pipeline monitoring fraud ops vs fraud analytics AI transformation in fraud teams fraud team restructuring for AI model drift detection fraud analyst role evolution root cause analysis in fraud systems KPI monitoring for automated systems fraud strategy and risk appetite data science in fraud teams rule writing automation fraud team budget planning fraud team org design

We started this series by talking about the pressure all risk leaders are facing right now. Use AI to cut costs and by cutting cost I mean cutting headcount. But in fraud operations the push for automation also creates a risk. What happens when teams cut the very function needed to govern automated systems? In the previous episode in this series, we outlined what a transformation journey looks like when you move from mostly manual operations to AI powered ones. And as we've discussed, there's a lot of nuance to it, because it's not a switch you just flip. It's a journey with a specific sequence and dependencies. But one thing I didn't cover previously is how you establish AI governance as you implement more and more AI automation. Specifically, there's a second order effect almost nobody's planning. One function on your team doesn't shrink with AI adoption. On the contrary, it actually needs to grow. Because automation creates new oversight requirements that most teams do not have staff today. And if you don't account for it, you'll end up with a smaller team that isn't equipped to govern all the automated systems it runs. And let me tell you that would not end well. That function is fraud analytics. And without it, your system would break faster than you'd like to think. It is the control layer that catches drift, monitors automated decisions, and prevents silent pipeline failures. So in this video, I want to talk about how fraud teams usually look like, what AI governance really means, and what it means to your future organization.

In my experience, most fraud teams are built from four functions. Now, I will say in our industry, terms and definitions are very loose. Even the word fraud might mean different things to different people or even the same person when considering different contexts. So it might be that you've seen other names or other organizational structures to this. The point I'm trying to make is around co responsibilities and skill sets an organization holds, rather than how it's actually structured or named. So bear with me. Anyway, back to the four functions I usually see. The first is fraud ops which handles investigations. Reviewing alerts, making rulings, managing chargebacks, and so on. The second is fraud analytics which owns rules and monitoring. Writing detection logic, tracking performance, analyzing attack patterns and producing reports. The third is fraud strategy that sets the risk appetite, risk policies and are likely in charge of vendor selection and architecture. And the fourth and last is data science that builds and maintains machine learning and AI models. But as I mentioned, not every fraud organization has all four. And there are a few reasons for that. First, it might be that some of these responsibilities simply don't exist. Not every organization develops their own AI models, for example. In other cases, especially in smaller organizations, some responsibilities are simply held by the team leader. For example, fraud strategy. It doesn't have to be its own function. So in effect, the bare minimum for a fraud team and what you'll almost always say is just one function, fraud ops. Meaning you can run a fraud function even if probably inefficiently with nothing more than investigators reviewing alerts. At the same time, fraud analytics exists in a lot of midsize and larger teams, but it's often informal or undersized. And that creates a potential oversight gap when these teams begin using AI agents to label cases, suggest rules, cluster alerts, all the things we talked about in the previous episodes. Meaning, if you need analytical skills to monitor agentic powered fraud systems, most teams are behind where they should be. And if you're thinking of downsizing them even more, then you might be risking self-sabotage when adopting AI at the same time. Because here's the thing, the traditional fraud analyst role was already important, but in an AI powered fraud team, it becomes central. Why do I think we'll need a bigger fraud analytics function in the age of agentic AI? In one word, governance.

Not all agentic AI creates the same kind of work. And understand the difference is what explains why certain functions on your team grow while others shrink. When fraud teams first roll out Agentic AI, they typically start with investigation assistance. The agent assembles the case and proposes a resolution and the investigator reviews and approves. Which means that this workflow isn't so different from managing a junior analyst. The agent does the leg work and the investigator uses their judgment to validate. And that's the thing, your team already knows how to do that. Governing individual decisions over individual cases is something your team members already do today. But fully automated pipelines are different. Auto labeling uh case clustering, rule recommendations, model training, these decisions operate at scale. They run continuously and you cannot have humans running around after agents because that would slow you down. Exactly the opposite of what you want to achieve. But the problem is that these pipelines can still fail. And when they do, they fail silently. A labeling agent that starts misclassifying doesn't raise its hand. It keeps labeling until someone checks. A rule built on a coincidental correlation looks fine in the back test. This system fails silently. They fail at scale and they fail fast. Unlike a human investigator who notices when something uh feels off, an automated pipeline doesn't feel anything. It just continues to run and spew garbage. But the governance these pipelines need is the same kind of monitoring fraud teams already apply today. KPIs, monitoring, alerting, systematic performance review, and most importantly, being able to run a root cause analysis when something breaks so you are able to fix it. Not just say that it's wrong. Does any of that sound familiar? Observing system performance through data, noticing issues, understanding what causes them, and fixing them, that's the job of fraud analytics.

What is the takeaway from looking at those two governing approaches? The obvious assumption when you start rolling out Agentic AI is that you can downsize your fraud team. But that is only correct for one function, fraud ops. For another, fraud analytics, it's not that simple. In fact, for most teams, it'll have to be exactly the opposite. Why? Because investigators benefit most directly from what AI agents automate. The part of their day that consumed the most time pulling transaction details, running IP lookups, checking device history, cross reference accounts. That's exactly what agents handle, an investigation. Simply put, the efficiency gains are straightforward because what used to take 30 minutes now takes five. Naturally, the function needs fewer people. But fraud analytics on the other hand, and as we just discussed, is different. Right now the team observes data, compiles reports, write rules when new attacks emerge and tweak score cut offs when needed. But when agents take over those tasks, all the automated pipelines we just described become their responsibility as well. Now you might think, but wouldn't I be automating the analysts jobs too? And you'll be right thinking that. But you need to remember that while some of the work is now automated, all the new automation you deployed both in analytics and in operations will now fall under their responsibility as well. Let's take rule writing as an example. Even if we reach an ideal state where agents write rules from scratch completely autonomously, which is definitely a stretch goal, your human analysts still have to review them. They need to check that the rules proposed went through all the tests correctly, that they don't contradict your business logic, and that they are not based on shaky statistics. That's still work, and it's only the start because we're still making individual decisions over single rules. But we also need to monitor how well the agents are proposing rules as a whole. How many times have the analysts corrected something fundamental? Did this number grow since last month? And if so, why? And how can we fix it? And this is just one example of how tasks that don't exist today will land in fraud analytics. And take into account that by the time you'll get to monitor your rules recommendation agents, you probably implemented at least five to 10 other systems. So all of them would need monitoring as well.

Hopefully, the argument I've outlined for why you'd actually need more analysts when you start automating has been convincing thus far. But the sad reality is that many teams don't have a fully fledged analytics team and so are likely blind to their repercussions. If you don't manage existing automated pipelines today or if engineering does it for you, odds are you have a blind spot there. And so when such teams come under cost pressure, it's very easy to commit to cutting the function that you actually need to enable AI adoption. It's all too easy to get to a point where your agentic investigation systems goes live, cases are being assembled, labels are being generated, and rules are being proposed. But the analytics team is still sized for quarterly rule reviews. And then the labeling pipeline has no error rate thresholds. The rule recommendation audits happen once a month when it should be happening weekly. So your fraud system is more automated, but your reaction cycle isn't faster. Even worse, your system is less stable and less safe than it was before. Why? Because we scaled automation without scaling governance. And that's exactly the recipe for failed AI adoption projects. Because it's easy to forget that redesigning your system means also redesigning your team and not just cutting it.

In the first episode in this series, I stressed that redesigning your fraud prevention system will come at a cost and that you'd want to fund it through the cost cutting you achieve. Part of that cost and only one part is making sure your team is properly staffed in the fraud analytics department and that you can increase its size as you go through the transformation journey. To an extent, it means that the savings you make from the shrinking fraud ops team should fund the growth of fraud analytics. And this can also be a way to upscale and retain some of the headcount instead of losing all of the institutional knowledge entirely. Of course, it might be that the target budget figure you show your board is higher than what they or you anticipated at first, but it's mandatory to build the function the new system depends on most because we should expect fraud teams to have a new bare minimum makeup. Investigative capabilities will continue to be part of it, but not all of it. In addition to that, teams will have to include strong analytical functions to operate its agentic workers, monitor automated pipelines, evaluate recommendations, and maintain your system stability as a whole. Or to frame it simply, think of it like that. Fraud analysts will be your new AI team leaders. That's the framing your board would understand. YouTube Description The Rise of Agentic Fraud Ops, part 3: Scaling Fraud Analytics September 5, 2026 Episode 20 Risk leaders are under pressure right now to use AI to cut costs. Cutting costs usually means cutting headcount. In fraud operations specifically, that instinct creates a blind spot in teams. The previous episodes in this series walked through what a transformation actually looks like moving from manual fraud operations to AI powered ones. What those episodes didn't get into is why scaling fraud analytics has to happen alongside that shift. There's a second order effect almost nobody plans for. One function doesn't shrink when a team adopts AI, it has to grow. If a fraud team headcount planning doesn't account for that, the result is a smaller team that isn't actually equipped to govern the automated systems it just deployed. That is fraud analytics. Skipping its growth is how AI rollout quietly turns into a bigger risk than the manual process it replaced. What you'll hear in this episode: - Why scaling fraud analytics matters more than any other staffing decision in an AI transformation, and most teams get this backwards. - Why most fraud teams break down into the four functions of fraud ops, fraud analytics, fraud strategy, and data science in fraud teams. And why almost none of them have all four fully staffed. - Why fraud ops vs fraud analytics respond in opposite directions to AI adoption. - The is a real difference between reviewing an individual agent decision and governing a fully automated pipeline at scale. - What silent pipeline failure actually looks like in practice, and why automated systems don't announce when they've gone wrong. - Why rule writing automation still requires human review, and what that review has to catch. - How KPI monitoring for automated systems and root cause analysis in fraud systems are skills fraud teams already have, just aimed at a new target. - Where fraud team restructuring for AI usually breaks down in quarterly reviews. Why it happens when missing error thresholds, and because audits happen monthly instead of weekly. - Why fraud analysts, not investigators or engineers, are becoming the new AI team leaders. - How to think about fraud team budget planning during this shift, including funding analytics growth from fraud ops savings. You should listen to this episode if you: - Lead a fraud team currently planning or mid-way through an AI transformation and haven't yet mapped what happens to your analytics function - Are under pressure to cut fraud team headcount and need a clear argument for where that logic breaks down - Have deployed or are about to deploy agentic AI for investigations, labeling, or rule writing and want to understand the governance gap most teams miss - Are building a fraud team budget case for your board and need language that connects cost savings to where they should actually be reinvested - Want a practical framework for fraud team org design that accounts for pipeline-level monitoring, not just individual case review - Are wondering whether your fraud analytics function is sized for the automation you're already running, or the automation you're about to add Episode notes & key takeaways: Why scaling fraud analytics is the decision most teams get backwards The default assumption going into an AI transformation is that every function shrinks. Investigation work, the thirty-minute tasks that become five-minute tasks, is exactly what agents are good at automating. That team naturally gets leaner. Applying the same logic to fraud analytics is the mistake worth flagging. Fraud ops vs fraud analytics isn't a story about automation shrinking everything equally, it's a story about work moving from one team to another. The governance problem hiding behind "it's working fine" Reviewing an individual agent's case recommendation is a skill fraud teams already have. It's close to reviewing a junior analyst's work. The real gap is pipeline-level governance, monitoring labeling systems, rule-recommendation engines, and model training processes that run continuously and were never built to be checked case by case. Silent pipeline failure is the risk worth taking seriously here, because these systems don't raise a flag when something's wrong. A mislabeling agent doesn't pause. It keeps going until someone happens to notice the damage. Root cause analysis doesn't disappear, it just moves up a level Fraud teams already know how to do this work. KPI monitoring for automated systems and root cause analysis in fraud systems are new skills. They're the same instincts fraud analysts have always applied to a rule or model, just aimed at a new layer. Even in a best-case scenario where an agent writes a rule entirely on its own, a human still has to confirm it passed every test, doesn't contradict existing business logic, and isn't built on statistical coincidence. Multiply that across every automated system a team runs, and the question becomes whether anyone is tracking how often analysts have had to step in to correct something fundamental. If that number is climbing and nobody's watching it, that's exactly how a fraud program gets less safe while looking more automated on paper. What failure actually looks like in practice An agentic investigation system goes fully live, but the analytics team is still staffed for a world of quarterly rule reviews. This is the most common failure pattern. There's no error rate threshold on the labeling pipeline. Rule audits happen monthly instead of weekly. The team ends up with more automation but no faster reaction cycle, and in some cases a system that's genuinely less stable than before AI arrived. It's a failure to scale governance at the same pace as automation, not a technology failure. Funding this the right way The savings from a leaner fraud ops team shouldn't disappear into a general cost-cutting line item, they should directly fund scaling fraud analytics instead. Beyond the financial logic, this is also one of the more realistic paths to retaining institutional knowledge instead of losing experienced people entirely during a transformation. The number that eventually goes to a board may be higher than anyone expected going in, and it's still worth bringing. Compressed into one line for that conversation is that fraud analysts are becoming the new AI team leaders. Not investigators, not engineers. And that is the role evolution this moment demands. Final takeaway: Scaling fraud analytics isn't a nice-to-have that gets added if the budget allows it, it's the one staffing decision that determines whether an AI transformation actually works or quietly makes a fraud program less safe. Fraud ops shrinking is the easy, obvious part of this story. Fraud analytics growing is the part almost nobody plans for, and it's the part that decides whether the whole system holds together. The teams that get this right won't be the ones that automated the fastest. They'll be the ones that scaled their oversight at the same pace as their automation, and funded that growth on purpose instead of discovering the gap after something already broke. (Accordion dropdown close) Transcript > ( Accordion dropdown) Resources & links: This is part of a series. If you landed here first, you may want to go back and listen to the previous episodes. We've already covered quite a lot that will make this one much easier to follow. Catch up on The Rise of Agentic Fraud Ops, part 1 Catch up on The Rise of Agentic Fraud Ops, part 2 Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: What's New in Merchant Fraud, with Dajana G. Meta Title: What's New in Merchant Fraud: Fraud, Refund Abuse, and AI Meta Description: Merchant fraud is evolving fast. The realities of first party fraud, refund abuse, fraud as a service, and the vulnerabilities most merchants are missing... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/merchant-fraud Keywords Primary Keyword: merchant fraud Secondary Keywords: first party fraud, refund abuse in ecommerce, friendly fraud chargebacks, fraud prevention strategies, ecommerce fraud detection, merchant fraud ring detection, chargeback fraud, fraud as a service, cross-merchant fraud intelligence sharing, end to end fraud monitoring, ecommerce fraud trends, fraud team structure, fraud operations, upstream fraud detection, merchant fraud visibility, merchant fraud collaboration, fraud collaboration for merchants, chargeback dispute rules regulation, fraud team cross-functional collaboration, merchant fraud vs bank fraud, gift card fraud prevention, merchant cyber security collaboration, ecommerce, fraud rate spike, how to detect merchant fraud attacks, merchant fraud benchmarking, fraud fighter career development first party fraud refund abuse in ecommerce friendly fraud chargebacks fraud prevention strategies ecommerce fraud detection merchant fraud ring detection chargeback fraud fraud as a service cross-merchant fraud intelligence sharing end to end fraud monitoring ecommerce fraud trends fraud team structure fraud operations upstream fraud detection merchant fraud visibility merchant fraud collaboration fraud collaboration for merchants chargeback dispute rules regulation fraud team cross-functional collaboration merchant fraud vs bank fraud gift card fraud prevention merchant cyber security collaboration ecommerce fraud rate spike how to detect merchant fraud attacks merchant fraud benchmarking fraud fighter career development

First party abuse is first party misuse, misuse, whatever we want to call it. Um, is definitely growing. I think that is in the last 12 months probably my biggest pain point. I remember it clearly because this was the only fraud attack that took me more than two days to detect. Because they were doing it so well, and we accidentally discovered it. If I start in a new role, if I start um some new project, there are two things I will always do. One is,

Hi everybody and welcome to another episode of The Saturday Fraud Strategist. With me today I have a special guest Dajana Gajic-Fisic. Wait, I need to do it properly. Dajana Gajic-Fisic. We just tried it a couple of times. Uh so second try. Not that bad. Not that bad. Uh Dajana, welcome to the show.

Oh thank you so much for having me.

Uh my pleasure. Dajana, you've been, you know, for, a very active voice in the industry. And you've been to the space, especially on the merchant side, uh, for what, 20 years or so, uh, now. And yeah,

A little bit longer than that. 20, 25 years.

25 years. Okay. Wow. Uh, insane. Uh and still I'm guessing that some of my listeners uh may have not come across you. Because you've spent most of your career on the merchant side. So maybe you can tell us uh a bit more about yourself, your career.

Absolutely. Well, um I'm trying to be very active on LinkedIn uh in the industry overall, but I don't think I'm as active as most of the people out there. So that could be a reason why some of the people did not run across uh my profile or some of my posts, some of my articles. But yes, I've been in an industry for probably 26 years almost. I think I started in 2000. Million years ago. And I always say I'm still 25. So let's just keep that in mind. But yes, I started in 2000 actually back at Macy's. I was working at Macy's credit granting department. Which wasn't really fraud team, but we were we were still reviewing credit applications of the people that were right there in the store at the POS. So there was some aspect of fraud in in my job. Um so there I started first working in a fraud team reviewing fraudulent applications, fraudulent online orders. I always like to joke and say that I was an ABS back 26 years ago. So we did not have these fancy systems that we have right now that addresses verified with the bank within a split second. But I was the person who was actually picking up the phone, calling Visa or Mastercard or American Express anywhere in the world and speaking several different languages trying to verify is your address one two three main street. So um now you can see how far along we came. But I always say that I kind of grew up with e-commerce industry and grew up with e-commerce uh fraud industry. And I just remember when we got first rulebased system I thought, this is the end of the world. This is so sophisticated and amazing. We have rules we can actually verify something quickly

And you're out of a job, right? Rule engines took your job.

Yes. You know, that really is really funny anecdote. And I think we can use that one later in our conversation. But yes, um started very long time ago. And as the e-commerce industry and e-commerce fraud industry progressed, I think I grew progressed with it. And of course that gave me a very uh, I would want to say unique perspective in this industry. What the challenges are. What uh, what the things are we need to change. And how we need to change it. Um, during these 26 years I was also um, as working for financial institutions. I was working for uh Barclay's Bank, but Macy's was actually bank too. So, they had their own bank. So, I was um a little bit on the financial institution side, a little bit of on e-commerce side of Macy's as well. So, I think that gives you an unique perspective of how things look like, at the other side. Um you know, what challenges they have, how they work, how the process looks like. So, it helps you be a little bit more active on the on the merchant side. Um, I was also in transportation industry for a little bit. You would think there are no uh there is no fraud in transportation industry. But think about instead of stealing a package, stealing a whole truck. So it was really fun time, and I am back to e-commerce now. Um, I am working for a digital gift card company. Uh it's resell gift cards. It's pretty fun and interesting. No fraud at all. Uh there is no risk at all. Um, nothing to to do. But not not at all. But in the last 26 years, like I said, um worked at various different organizations, various different teams. My position was probably going from being in customer service and finance team, to being standalone teams. And that's another progression I have seen in these 26 years. How fraud as a career grew. I think 10 years ago, we didn't have fraud directors, e-commerce fraud directors. Banking, yes. We highly had managers. We usually had analysts that were sitting in customer service teams. Now we have directors, now we have vice presidents, now we have this whole standalone department. So I think I've seen a progression of the fraud industry and fraud fighting fighting world as well. Um, another aspect of my career is uh trainings, webinars, podcasts, um, speaking at the at the conferences. I really enjoyed this education. Advocating for the for the industry. Trying to be voice for the industry. You mentioned some people may not know me because I'm not as active as some other people on LinkedIn. The reason for that is I actually fight fraud. So I don't really have that much time to be active. Not as much as I would like to. Um, but then I know that that's how the entire industry is. So I'm trying to be that voice for our industry and trying to be an advocate for how things need to change on our end with all this technology changing with industry changing with e-commerce space changing overall I'm trying to make sure that fraud industry is part of that change in a positive way. So long long time in this space I have seen a lot. Um, there are some good things that came out of it. There are still some challenges that I think we need to work on, but overall it's been quite fun. 25, 6 years.

It's been a ride. Um, awesome, Dajana. Well, um, I'm sure you've seen your share of, um, of fraud. Uh, and also seen your share of fraud teams. Uh, super interesting. We'll dive into that in a minute. I I'll say, you know, uh, it's Friday morning right now. Uh, we're both in Europe. I'm I'm in Spain.You're now uh in in Greece, right?

I'm Greece. I'm on Kythira.

Yeah. Yeah. Uh, enjoying enjoying the sound. It's quite toasty out there. I've got my uh my my iced coffee. I saw that you are drinking coffee as well. Um so, while the the US guys uh folks are are still sleeping. Um, let's uh let's figure out what's new on the merchant side. Uh, you know, I I want to start by asking, a couple of weeks ago I had David Liu on the podcast. And me and David spoke about, a lot about AI. And AI powered fraud. And one of the things that David noted, and of course David is uh I I think you've seen a lot of the merchant side as well, but mainly more on the financial uh services side. One of the things that David mentioned was one that we're already seeing quite a lot of AI powered fraud. Especially on the deep fake side KYC, KYB and so on. Um and also, one very kind of like uh you know like main thread of our conversation was, how do we think fraudsters would be able to leverage AI, Agentic AI to commit fraud in the near future? And we kind of like shared a couple of, I don't want to say horror stories, but more kind of like uh concerning scenarios. And you know someone who's been in the space for so long. And someone who is an actual fraud fighter. Uh in a business that is known for some elevated fraud levels. How do you see the impact of AI on the fraud ecosystem? Uh, specifically like in in from your eyes, as a practitioner.

Yeah. Um, I don't think that, I think you and I can both agree AI is not new in fraud industry. Not new for fraudsters, not new for fraud fighters. Um tools and abilities are not as sophisticated, uh, they were not as sophisticated 10 years ago. But AI at some, in some way existed 10 years ago, 5 years ago, even more. We all have and know of these machine learning, uh, tools. That are kind of AI tools. I mean that's what it is. And so I don't think it's anything new. And I think that the change in trend in fraud is nothing new. I want to bring up something that happened about 10 years ago, or maybe 11 years ago. Um, chip cards, chip and pin cards. That was, that wrecked havoc of e-commerce industry. The funny thing is because we didn't have fraud teams. Most of us didn't even know that this is happening. I did know because I had a pretty good team and pretty good partners. They're informed me time that this is happening. The shift is happening. And for those, uh, maybe younger listeners that don't know what that is. Is when the chip was inserted in a credit card. So now you did not have to have the actual, uh, authorization or actually you had to have authorization but you had to have an actual credit card present. You couldn't just type in the number you had to tap and get the PIN number. Um so that shifted fraud from the actual POS and brick and mortar, to internet. To to online space. and that's when we've seen so much creativity in a in a fraudster. And we seen the lack of creativity in a fraud team. So I just want to use that as an example of how the fraud world actually shifted drastically. And how the risk we were facing shifted drastically, because fraud rates went through the roof. Uh, it wasn't talked about as much. Internet was not used as as it is today. I don't know how many people were using LinkedIn and posting on LinkedIn as actively as they're posting right now. So, we didn't have much information of what's going on. So, when this went in place in October of 2015, I think. It was a nightmare.

Around that time, yeah.

And imagine this is go, it was a nightmare. It's October of 2015 and we're looking into holiday season. We're looking into Black Friday, Cyber Monday season. And and fraud is going through the roof. Because fraudsters now moved from using um counterfeit cards into going online and using those gift card. Uh using those credit cards. And now we're seeing same thing in AI. I think it's a new thing. It's a new fad that everyone is using. And some people are overusing, to be honest. And I, do I think it's going to create some, um, challenges? Absolutely. But also, I think we have to be smart. And use this AI to our advantage. Um, in one of my posts, um, I wrote, maybe a year ago. I say the fraudsters are typically leading this innovation curve. And I think this is where we have to get smarter. There are some AI driven attacks. There are some tools that fraudsters can use that can hide their identities very well, and appear as a legitimate customers. Um, but also we have to be smarter. And use these tools to help us. And use these tools to get ahead of them. I I think traditionally we are always lagging. We're always catching up to what they're doing. So they commit fraud, we catch up with. Oh, we're going to do this to stop them. Then they commit fraud, we're going to do this to stop them. Why don't we anticipate what's going to happen? We have enough knowledge, enough experience. And start building our tools, our strategies and tactics around what could have possibly happened. So I think that's where AI will assist many very creative and very smart fraud fighters. To get ahead, maybe for first time, in a fraud history had these fraudsters.

That's super interesting. I mean first of all I must say that in my mind, this period of elevated kind of like e-commerce losses and the explosion of e-commerce. Uh, in my mind was always kind of like somehow, uh, related to the rise of, uh, mobile. And mobile shopping. Um, which happened more or less, um, in in the same period of time. And and when you told it, sorry I I was kind of like going back in my mind 10 years ago. And I said, yes, chip and pin. And and kind of like my, like fraudsters migrating from point of sale to ecom, was a thing. Which I completely blanked. And it shows how institutional knowledge, which you would assume is one of the greatest, uh um, advantages. That us fraud fighters, and fraud teams have, in the fight of fraud. Uh, or or fight against fraud. Um, it's not that simple. Because even for someone like me, who's been a fraud fighter for, fraud fighter for several years, when that happened. Even I completely blanked that. So that is interesting. The other point that you made is also very interesting. You were saying, look there will always be these kind of like new things that happen. Sometime it might be new technology. Sometime it might be new regulation. That will radically change how the space looks, how the threats, uh, look like. And that's not the end of the world. We we will probably manage it.

It's not the end of the world. We just have to see this as a new challenge, and try to find a way around it. Try to find a way to stop it. I think this kind of open up, whole another topic. That you and I could probably spend another three hours, uh, on talking about. But it it's talking about position of your fraud team in your organization. And in this space, fraud teams traditionally were in silos. So I am sure that finance teams back in 2015, knew about chip and pin changes. I'm sure the brick-and-mortar teams knew. Because they had to change POS systems. But nobody communicated to fraud teams. Because fraud teams were typically in silo. And this is what I'm talking about, the progression of the fraud fighting industry. And our jobs as a careers. Is now we are more visible. Now we have, most of us, more equal seat at the table. Now we know of these things. We have resources, where we can read about what's happening. Visa changes, MasterCard changes, and all this new fun things. Um, so we can get ready. Back then we really didn't. Back then, I think I found out at mid September that this is coming in October. I'm like, oh well. What does that mean for me? But there is another aspect of at least the way I work. And this is back to AI. Yes it's going, we're going to probably see a lot more, um, AI powered attacks. We're going to see a lot more creativity of the fraudsters. But going back down to basics, is like, what are you trying to do? So let's stop thinking about how am I going to stop AI fraud attack? No, let's think about what am I trying to do? I'm trying to stop fraud. How fraud looks like. Well, this is how fraud looks like. It's not that I don't really care where it's coming from. Because if it's AI powered, it's probably going to be a lot faster, a lot more difficult to to detect and stop. But there has to be a a vector. There has to be some data that will indicate this anomaly. And help me detect it and stop it. So maybe first time it's going to take a day. Next time it's going to take two hours. Third time it's going to take 15 minutes. And fourth time I'm going to be ahead of it. But you know, let's let's talk about what is it that we are trying to do. We're trying to stop fraud. How fraud looks like. This is how it looks like. Well, let's stop it. It doesn't matter if it's third party fraud, first party fraud, or million other classifications and um names you have for it out there. It's like what are you trying to do? I'm not sure if you had a chance to read the book. It's called Working Backwards. Um,

Uh, I don't believe so.

Um, it's it's pretty interesting book. I think he was one of the original, a uh Amazon AI executives, another Amazon executives. Um, and it talks about, my at least my understanding is like, where do you want to be? What is your end goal? And then you layer down to how can I reach my goal. Instead of I'm here and this is where I want to be. You start at the top. You start with the definition of your goal, and your mission, and then you layer. Create layers on how to reach this goal. So going back to fraud fighting, okay, AI is going to be very difficult. It is already very difficult to detect. It's really helping a lot of bad people out there. And some good people that are acting bad. But what do I want to do? I want to stop fraud. I don't want to stop AI fraud. I want to stop fraud period. And how can I do that? I'm getting a little bit more philosophical about it, but I hope it makes sense.

I love it. Because I find myself very often saying exactly the same. Um, it's about the basics. And and well, I okay. Let's not get too much, uh, deep into it. Other than saying that I I completely agree with you. Uh, and I completely agree with the approach. But tell me, um, does that mean that you're actually seeing AI powered fraud? Uh, right now, hitting your your, or at least attempting to hit your business. And does that, uh, that that, like do these attacks, look differently? Are they harder to detect? Um, or like, what is it that you're seeing? What's new?

To be honest with you, I don't think I've seen, I'm going to knock on wood. I don't think I've seen a major AI powered fraud attack yet. I do see fraudsters using AI, um, in some shape and form to commit fraud. So, I think it's not scale yet. At least I haven't seen it yet. But here is my fear. And maybe I'm going to help fraudsters right now. My fear is that if someone

Wait, wait, wait. Fraudsters, close your ears, please, and don't listen in the next minute. Okay. Yeah, you're good.

I really think if someone figures out how to use AI. To execute, I'm not going to say this word, to execute a an attack. I don't think we'll be able to figure, to find it. To to truly fully use AI. The way that AI is being used right now, the the way AI capabilities right now for fraud attacks, it can still be detected with our regular traditional tools. Um, if you know how to look at it. And if you know what to look for. And I am not going to say what that is, because that's not what we want to deal with.

So it's interesting, because I think this is also kind of like, uh, was David's, uh, notion. That, I can see how AI can be used to create undetectable fraud. By the way, I think that us as fraud fighters, because we know the limits and the gaps in our systems, we can always think about what kind of fraud attacks would bypass it. With AI or without AI, as you said. AI might help fraudsters scale it. Um, but I think it is easier said than done. And you need to have, like a lot of intimate, uh, privileged knowledge, uh, to actually execute it. But what I found interesting, and I want to hear more about. Is that you said, I don't see AI powered fraud but I see fraudsters using AI. What does that mean? How do you see them using AI today?

I think there is a lot of, um, uh, creating accounts, creating different emails. Uh, using some type of automation to place the orders. So you can see the speed of the orders, and so on. And I don't think that's anything new. Because using some of these AI tools to create profiles, identities, um, and so on. Has been happening for quite some time. I just think right now is, as the as the AI is more accessible to fraud fighters and everyone else in the organization. That's how it's more accessible to the good people, that think they can use AI to commit fraud, gain financial benefits, without being detected. So I think it's just being much more adopted by the fraudsters, and fraudsters wannabe. So I'm going to call them that way. Um, and and and that's what I see. I see some of the AI tools being used to, uh, create identities. And and and maybe use the automation, place orders. I don't see really, so much more AI powered attack. And and maybe it's going to happen with some of these, um, agents. Like um commerce agents. Maybe it's going to happen more soon, but not yet.

You're not seeing it yet. Yeah, that's interesting. Because I always uh hold the belief, and and it may sound funny at first. That it is actually easier to detect fraud, when the attack is automated. Uh, regardless if you automate it with a script or with AI. Now, what makes it scary, is that because it's automated, and because the speed and the scale are quite high, every minute that passes and you didn't identify it, and you didn't stop it, you bleeding a lot of money. But the fraud attack itself, once you look at it, it it's obvious fraud. It's obvious fraud.

Absolutely. Um, I don't know if you know that I was in a uh athletic footwear industry. That is extremely risky. And there is a lot of bot used for, by the resellers. And that's one thing I always said. Bot users probably are not committing fraud. They are buying limited, um, quantity shoes. So they can resell them for the high price. But it's a it's a possibility out there, that if someone intends to use that. It's going to buy a lot of shoes really fast. And by the time I react, it's going to be gone. So that is 100% sure, correct. And that's my biggest concern. It's not about what they are using it for, but what could they possibly use it for. But you are 100% right on the other side too. Is that's a lot easier to detect. Because that's how you should be detecting fraud. You, maybe we're giving a little bit too much information right now, but you should look for anomalies. Um, you should look for anomalies. And I always say. When you want to know what's going on in your traffic, in in your space, you look for anomalies. If you want to know exactly how it's being done, you look for patterns. So that's like, kind of working backwards. If you want to know if something is happening, is there an attack? Is there attack of any type? Is there something going on? Is there any issue with the system, with the process? You look for the anomalies in your traffic. But when you want to know what actually is going on, then you dig deeper and look for the patterns between those data points, between orders, between identities. So yes, these type of attacks would be a lot easier to discover. Because they are an anomaly. They are not something that is showing up, as a clear normal traffic. But the speed at which it's happening is what's causing and creating a risk.

Um, yeah. Uh, that is that is such a,

I can see the wheels turning there.

Yeah, because it is such an interesting phenomena. Because you, like you have this perception of what AI fraud actually means, and why it is dangerous. But actually when you, kind of like peel the surface a bit, you understand that it is not dangerous because it is undetectable. It is dangerous because it hits you fast. And so, the way to kind of like, uh uh um, defend against it, is not with black magic and technology. And somehow piercing the identity behind the agent. And so, no. It's just reacting faster. And to react faster you just need to like, have good basics, as you, as you said

And I would say increasing visibility. So your visibility is not anymore at the checkout. Your visibility needs to be everywhere. You need to monitor end to end traffic, end to end uh customer journey. And you need to identify all those risk gaps on that journey. You need to attempt to close those risk gaps. And you need to identify your key risk indicators. What are your indicators? And I think this is not giving away too much information. Because each business has different risk indicator. So what's risk to me, may not be risk to the travel industry, may not be risk to the athletic footwear industry, or or um sneakers. Um, so identify your key risk indicators. And then create alerts around those key indicators.

Yeah.

And you can do that using AI right now 24/7.

Yeah. Basics. Like if we would have, uh uh, had this conversation 20 years ago, we would probably still have the same conversation. Uh, which is, which is kind of like, what what's mind boggling about this space. Yeah.

Well, it did take me about 20 years to learn this. So I probably wouldn't know this 20 years. So. But but, that's I think, that comes with an experiences. I think a lot of fraud professionals, fraud fighters right now, are very focused on, um, operations. And they're very focused on the, each order and analysis analytics of the data. And, um, an analytics of the fraud attack trying to understand what happened. Instead of stepping out. And if you want to grow in your career, you have to step out of that. And see what did you learn from the past attack? What did you learn from the data, and how can you implement that on much larger scale? And as a big picture to prevent fraud from happening. Not, I think way too many fraud fighters are very operationally focused. Instead of being strategically focused. So, I think especially with AI. AI gives us such great opportunity to be more of a strategist, rather than operational. Um, so I think that's really great opportunity for fraud fighters.

100%. I Wow, that that is such a such a great advice. [Ad Break (27:53): Hey folks, I want to take a quick break to speak about today's sponsor, me. If you're finding this useful, do me a small favor, like and subscribe. It really helps with the algorithm. And if you're not already on my newsletter, The Saturday Fraud Strategist, you should definitely check it out. Every Saturday, I break down fraud trends, real cases, and strategies from my own experience of building and operating fraud prevention systems. Whether you're new to the space or a seasoned practitioner, I'm sure you'll find it interesting. Check the link in the description. Takes two seconds. You'll thank yourself next Saturday. Now, back to the video.]

For merchants today, I would say ever since COVID, uh more or less uh, first party fraud. Uh friendly fraud, policy abusers, whatever you want to call it. Is one of the main threats. Uh, or I would say one of the main loss drivers that endanger merchants. And I'm wondering if you've seen any changes around, uh around this kind of like, uh specific threat. Uh, regardless, or or maybe, you know, uh, with regard to to AI.

Um, yes absolutely. I I think you're 100% correct that recent fraud, first party abuse is first party misuse, misuse, whatever we want to call it. Um, is definitely growing. I think that is in the last 12 months probably my biggest pain point. Um, and I think it's biggest pain point in the industry. Because if you look at some of these conversations on the social media, it there is a lot of conversation about, um, people filing dispute when it isn't fraud. People just calling bank. You can actually go to Instagram or Tik Tok, whatever you use, I don't use Tik Tok, I don't even have it. Uh, but, and I've seen a lot of these posts saying, if you don't like the service, just call the bank file a chargeback. And and these influencers are advising you to take the step of filing a chargeback if you don't like someone, the way someone did your nails. And I think normalizing this process as a way to get your money back, instead of communicating with the merchant, and and, um, going through the process, is actually really dangerous trend right now. And there are no rules and regulations. There is no accountability on anyone else's side other than merchants. And that is currently probably my biggest pain point. That this is becoming widespread use of the process intended to protect the consumer. And it's now turning against, not just merchants, but against banks as well. Uh, because people were just using this process of of the fraud, um, of the dispute, to get their money back. When it's really not fraud. So that's one aspect of it. The other one, um, we did see during COVID. Refund and DNRs, did not receive, INRs whatever. Someone calls them, uh, people claiming they didn't get the package in various different ways. Claiming they bought two, but they received only one. They send it, they didn't get the refund. So we have seen the increase in this trend of, again, filing a false claim. In this case, the claim was not filed with the bank. It was filed within the customer service teams, or within the organization company, um, but it's causing the same type of loss. And I think between those two, I'm not sure which one is more dangerous right now. Um, I would say maybe these internal claims, the refund abuse, uh, is a little bit more dangerous. Only because it's not really tracked appropriately by all of the organizations. So the full scale of impact and losses is not really measurable at this point. If you look at the last MRC global report, I think they have, I'm actually looking at it right now. They have refund policy and abuse number two, fraud trends in the United States. Realtime payments is number one. And first party misuse number three. Asia Pacific has refund and policy abuse number one. Latin America number one. Um, small business is also number one. And enterprise is also number one. Refund and, um, policy abuse. That tells you a lot. And I am still claiming that probably less than 50% of the merchants are truly measuring the impact of these, of this type and fraud. So can you imagine how high and how prevalent this would be if everyone measured it appropriately? It's still written off as a cost of doing business. So, um, I think we do have, we still have a big pandemic of a fraud issue here. I just don't think it's handled appropriately. And I personally think that industry is little little bit lost around how to define these fraud trends truly, what it is. I don't think we can still agree on what first party misuse is. I think one merchant thinks is this, other merchant thinks is that. People are obviously throwing several different definitions out there. So I think as an industry, we need to get a little bit more organized around these fraud trends. Maybe advocate more with issuing banks, with regulators on on getting this type of fraud under control.

Yeah, I I would say that on the financial services side, I think it's even worse. In the sense that I think nobody's really monitoring or or reporting it. Right? Um, and I think on the merchant side, it's really, I don't want to say confusion. Because maybe that's, like a very clear business decision, but it's it's a question of how do you balance customer experience with risk. Uh, and that that is always the case with fraud management. But specifically with first party fraud, it's um I am not 100% certain that when leadership teams make policy decisions they understand what is the cost of business that are, that that is basically tied to these decisions. Uh, for sure. Um, I wonder though, if the change that you that you've seen in the last 12 months is that, do you think it has somehow something to do with AI? Or is that, you know, is that a a shift, or I don't know, like a a search that, you know, is driven by other factors?

I think it's driven by this social accept acceptance of this behavior. Like like I mentioned, social media sharing these, uh, options as your first and only option. Instead of communicating with the merchants, resolving whatever problem you have. So I think that's maybe normalizing this behavior. And there is this funny thing I always say. If you have an enterprise level company that you claim chargeback against. And you claim 10 of them, and 20 people claim 10, um, chargebacks. Or file 10 chargebacks. You think, oh it's not going to hurt the CEO of that company. He doesn't care. It's going to hurt the the entry level agent. It's going to hurt the managers. So when that business is not doing well, you're not hurting business overall. You are hurting actually your peers, who are working there. Because when there are layoffs, when they are uh elimination of of staff. It's usually starting at the entry level agents at the low level managers, or mid mid managers. So you are impacting your friends, you are impacting your family and your colleagues. So you are not really creating a huge damage and loss to the company. Because they will usually say, oh they don't care. They can take this. This is nothing to them. It is nothing to them in a large in, on a large scale. But when the decisions are made because of the business being impacted, it's going to be impacting your neighbor, your friend, your family. So I think that's something they don't think about. Um, but in what I also see is that, I don't think the rules and regulations are changing fast enough for the environment we have. We talk about AI. AI is changing things so fast on various different fronts. We still have rules and regulations around disputes from 1978 that never changed. So the Reg E Reg Z changed maybe once or twice in in the last 40 years. There was a proposed change around the electronic payments disputes. Um, I want to say 2006. That was never adopted, never fully went into place. It could have been 2016. I'm not 100% sure. I had the data somewhere. But I want to say that we need to change rules and regulations to fit the environment we're in right now. Yes, we want as a consumer, I want to be protected by the bank. If there is a chargeback, if there is a fraud on my account. I want to be able to file the chargeback. But creating these mobile apps where you can just click twice. Allowing people to file 30 different chargebacks on 12 months worth of orders, without checking, without investigating. I know again that there are rules and regulations. That say you have to continue with the process. You have to file it. But let's do the investigation a little bit. And let's see what in this process, or in this regulation can change. That we not only protect the consumer, but that there is some type of accountability for us as an industry. Of of making sure that, you know, this protection does not have to mean lack of accountability. So I I think those two things can go hand in hand instead of being like an opposing forces here. So, um, and and the accountability have to be on all has to be on all of us. To me as a merchant, I feel there is so much more accountability on us. Than on anyone else out there. Maybe finance world feels feels the same. Maybe banking feels exactly the same. Um, but what I do want to say is, I feel sometimes I'm fighting, um, card networks. I'm fighting payment processors. I'm fighting, uh, issuing banks. I'm fighting everyone else, on a daily basis. And trying to appease everyone else, on daily basis. More than I'm fighting fraud. Because there are so many requirements and accountabilities on a merchant. And there is not, there is a lot less elsewhere. So maybe a little bit controversial, um, opinion here. But that's how I feel. Especially with all these changes that are happening lately.

I agree. It's like each each kind of like player in the on the chain is trying to push the ball to someone else's court. So they know, don't have to to deal with it. I mean the VAMP and VFMP, and all of these things. I mean it's exactly that. Um, but yeah, it's like it's interesting because we keep speaking about AI. We keep speaking about AI, as the threat that we fraud fighters need to address, and think about, and prepare for. While maybe, at least on the merchant side. Um, and by the way, side note. Maybe also on the financial services side, if they actually track it correctly. Maybe the the, um, the the the number one threat is still for years now, um, policy abusers and first party fraud. And you know, we we used to kind of like say, when when you wanted to learn how to commit fraud, you used to have, you know, this store uh access point. And you need to go on the dark web, and you need to be like very techy and so on. And today, no. You just need to go on Tik Tok. Uh, to learn how to do fraud, and how to make money. And yeah, I think, I mean someone like David Maimon speaks about it all the time. And I wonder if this is not the number one threat that we actually, uh uh, stand against. Again, especially on the merchant side. And this gets talked about much less.

And you know what? You don't even have to go on Tik Tok. You can just go on Telegram and have someone else commit it for you. Because there there are these fraud as a service groups now. That you bought a computer, and you want a refund and keep the computer. You contact so and so somewhere on the other side of the world.

Yeah. Yeah.

And they just file this claim for you. Because they have the clear instructions. They have step-by-step process on how to call the company, when to call, how to talk to to them, what to say, when to say it, if you need to do it second time, how fast you need to call back. There's there are like a clear instruction manuals for these people how to commit this type of fraud. And you don't even have to commit it. So, you feel good about yourself. Hey, I got the computer. I got $800 back. The fraud as a service kept $200. And we're all doing great. Except the merchant. And then what happens? Merchants overreact sometimes. They're creating all these strict policies. They're creating, um, fees or fees for returning like re return fees. They're creating all these steps that will end up impacting a good user. So either way, merchant is the one that gets the the the biggest impact with some of these, uh, fraud attacks. I think we opened a can of worms here. So I think we should move on.

Uh, I one of the things that you, that you told me, uh, in our previous conversation. Which really surprised me, was how in your current role as head of fraud, um, you actually transitioned more and more into a kind of like head of risk, uh, position. Where considerations like compliance, like AML, suddenly are relevant. And again, you're you're a merchant. Uh, and that took me by surprise. Because yeah, like, I I can see how in certain, uh, in certain environments, especially marketplaces maybe, there are like a bit more compliance issues maybe. Like in in high-risk verticals, you need like age verification and stuff like that. But I heard like, what I heard from you is that you looking at it on a much larger scale. And much more similarly to, um, how a fintech would, uh, look at these topics. And I I I'm curious to hear more about that.

Well, that's probably my own curiosity. Because I like to keep an eye on everything. We do have a full compliance team that handles the full, uh, side of compliance work. Um, they do report to me right now. Uh, and that is more like a practical things of um trying to optimize the performance of their processes and their strategies. So, um, but as a fraud fighter, I think you have to be, uh, aware of what other risks are you facing. And and what other fraud can fraudster commit on your side. That it may not be payment fraud, it may not be credit card fraud. So sometimes you will observe. And I actually had this opportunity, I had this situation in my previous positions, too. Where I would see someone doing something that doesn't look fraudulent at all. But they're doing something that indicates it's off. Something is just not right. And I think a lot of fraud fighters can relate to this, like that bullet nagging gut feeling that tells you something is just not right. So I think, um, you have to follow that gut feeling. And start to digging and start investigating. And and this is where you go into more of a, when when is the risk to your company. So money laundering is, of course, serious issue. And especially in the gift card industry. It's heavily regulated industry. So we have to keep an eye on everything. But these, typically your fraudsters will be the same ones that are committing these type of fraud. So I think that's where the overlap is. And that's where we can actually share our intelligence. Share the data finding and analysis. Which then brought to this natural collaboration between two teams. And I think lines are not just getting blurred between compliance teams and fraud teams. I think lines are getting blurred between, um, cyber security and fraud teams as well. So I think naturally cyber security, information security, cyber security teams, fraud teams, compliance team, naturally that's one progression where we should be all one team. Because we're looking at the same things, at the same people, doing the same damage to our company, Which is financial risk, and that financial risk can grow seriously into some other risks as well. And that's why I think there is this natural kind of, uh, progression into collaboration of these. Another thing I noticed, and I'm going to talk a little bit. Maybe this is a good thing to say about, um, financial industry lines are getting blurred between fraud we are seeing as merchants and fraud financial institutions are seeing right now. It's about identity.

Say more.

Yes. It's it's now fraud is more on the identity and behavior rather than the actual transaction. So I think, don't get me wrong, there is still transactional fraud. It's still happening. But with AI tools, fraud is getting more sophisticated. It is more about behavior. And behavior on the financial institution sides and behavior on the merchant side are kind of looking exactly the same. So this is something that led me to advocate more for let's collaborate more. Not just internally between all these wonderful departments. We have cyber security, and and fraud and compliance. But on the more larger scale with financial institutions as well. Because we are seeing same actors that are causing the same damage to all of us. Which is a financial loss and reputational loss too. So I I think lines are getting blurred here. And we need to recognize that and start thinking how we can take that to our advantage.

Can you give an example. And and this can be also a hypothetical example based on your experience. On like how that would look like. And how how such an anomaly, an attack, um, an occurrence in your system. How that would look like from the different teams perspective. Um, because basically, um, like I would guess that a fraud fighter that is listening right now would say, okay that's a good idea. But like how, what should I actually look for? And why why does that matter? Dajana Gajic-Fisic(46:47): So here is a very basic and simple example. You see a spike in traffic. And spike in traffic comes from three different BIN numbers and you look at the BIN numbers and they all belong to the same issuing bank. Wouldn't it be natural next step to go ahead and call this bank? Or send some information to this bank and say, "Hey, I have x amount of orders." It could be even like a card testing attack. Could be on the cyber security side, which sees it first. It could be anywhere in your process. Um, and then you communicate this to them. Maybe to them, that's really not important. Because it's so small compared to the number of credit cards they had. But maybe it's beginning of something that can grow. Because I'm 100% sure that if I see a BIN attack on my site, I am not the only merchant that sees that BIN attack. So instead of having an opportunity, having a tool, way, protocol, whatever you want to call it. To share this data, to share this intelligence with the industry and get ahead of this. What are we waiting for to happen in order to take next step? We're waiting for a chargeback. Because we're just assuming. We don't know. We didn't have any losses, because we stopped it. Fine, but we didn't have losses. That doesn't mean that someone else will not have losses So I understand the competitiveness. But I think when it comes to fraud we should collaborate a lot more. And sharing the data and intelligence. And there is a serious concern about PII. What we can and cannot share. I think there is a way to share intelligence, without sharing like private information of the individuals, to get ahead of these fraud attacks. And and there is one example. BIN number without any other data, can be shared with the bank or with another merchant. And say hey I'm seeing this. Do you see it? Um, I'm going to give you actually concrete example. And this happened oh maybe five six years ago. I remember it clearly. Because this was the only fraud attack that took me more than two days to detect. Because they were doing it so well. And, um, we accidentally discovered it. We discovered it because of ethical alerts. I'm going to be completely honest with you. Because the fraud attack was so well, uh, organized, coordinated, executed. That I was like bravo. It was really interesting. Um, so I remember getting orders, and I remember addresses changing. But addresses changed just slightly. I, you probably heard of this. Those, I think they called them address manipulators. I think that's how we call them later on. They were adding dashes, dots, changing the spelling of the address, um, of the actual street name. But they did it slowly. There was no huge amount of orders. They were doing it slowly for days. And then they would slowly increase. But what they did, they changed the address slightly. Then they would change zip codes, then they would change name, they would change email. But it was done so at a slower pace. That it kind of looked good. Good, uh, as a good, um, traffic. So we received I think one alert. And we have a process what happens when you receive one alert. Actually when you receive second alert, to me two is a coincidence. Three is a pattern. So I started digging right away. Doesn't matter how big your data, uh, is. So we started digging and then we found this there are some similarities. We took steps. We contacted our, um, fraud vendor. And they said immediately what we see. They actually recognized it. So we took steps and stopped it. I have a WhatsApp fraud group that I shared this with them. I have at least three other merchants that found it right away on their end. And it was like, "Oh my gosh, we just saw it." And of course, they stopped it. 6 months later, uh, we were at the conference. I run into the person who is not part of our WhatsApp group. And she's talking about fraud rates being seven, eight, 10%. And I ask her, "What happened? How did you see this?" This attacker was active on their page. So this was what September, October. Maybe September that we discovered it. So until March, this attacker was active on their site. And they didn't, they couldn't find how. And it increased their fraud rate to almost 10%. What if they were part of this group? What if we had a tool to share this information? They could have stopped it too. And this was not the only merchant that had the high fraud rate from the same type of attack. I think there were some, uh, webinars out there later on in podcast. And there were so many posts later on. Uh, I think they called them address manipulators or something like that. Um, because that was kind of one vector we found. Stopped them. But what if we had a way of share this information in more formal way, and help each other? We didn't. So something I knew in September, someone else didn't discover until they met me at the conference in March. So I think that's unfortunate. So maybe a as a industry we should get organized a lot more around how we can share this data. And I know I'm going to get like million fraud vendors right now and say "hey but we have it." Yes, you do. You have it for your network. If someone is not in your network, they would not have visibility, and you can't expect everyone.

Yeah it's the same. It's not the same as sharing data. It's not like, okay this email is associated to fraud. But really this is the pattern. This is exactly, kind of like, the attack, uh uh uh uh, pattern, that, um, or the the MO that the fraudster is using. This is what you need to look for. Yeah. Dajana Gajic-Fisic(52:44): Uh, have you ever been on Telegram in one of those fraud groups? Have you ever looked through them?

No. I I I always uh worried about doing that. With uh you know with with numbers that I own. And I never went through the trouble of, you know, like uh trying to do it through a burner.

I think on a Telegram you can actually hide your number and your name. That's why fraudsters are using Telegram much more. I'm not sure how it is right now to be honest with you. I haven't used it in a while. But if you go there, and if you go to any other like um site, they share this data. They share the intelligence. They share everything. Because if you get attacked by one fraudster and it's successful, you're going to get attacked by like another five. Because it's successful. So they share this information. Why don't we? I know they're not as regulated as we are. But I think we can find a way of creating more collaborative environment. And share intelligence in a more safe way, without sharing private individuals data. And and help each other. But I I've been preaching this for quite some time, and I am seriously getting nowhere.

Yeah, you're not the only one. Um Dajana, tell me. Um, you we we've spoken quite a lot about the changes that you've seen in the past 12 months or so. And the different drivers that that kind of like fuel these changes. I wonder from now, wearing kind of like the the fraud fighting hat and the practitioner hat. Um, if you can give some advice or maybe share some of the things that you've done internally. Changes that you've implemented to, uh, better, better not not only prepare. But, basically better perform, in in these times.

There are two things that I always do. And especially when I, if I start in a new role, if I start um some new project. There are two things I will always do. One is the method or approach that I created probably, maybe more than 10 years ago. Which is kind of a result of all this experience that I have, and knowledge and working in different organization work. Having different challenges, uh, by being in different teams. Uh, it's my 360 approach. I am not a creative person. I'm a number person. I'm a fraud fighter. So I couldn't come up with any better name. Maybe I should ask ChatGPT. But it's like a, um, I call it 360 approach. Where I think that fraud operations are separated into four parts: prevention, investigation, reporting and uh strategic planning. And they are not separate. It's kind of a circular motion. You have prevention. Whatever you use in prevention, you share with investigation. Whatever investigation comes up with and finds, reporting can get organized, report on to send it to strategy and strategy can use this information for your next fraud prevention. And then it goes like this. So it's like: monitor, identify, mitigate. So I always start with that approach. But what's interesting about that approach is these are four part, four parts of the fraud process or risk management process, whatever you want to call it. But it's not four teams. It can be hundreds of teams in your organization. It can be your vendor's part of this approach.

It could be one person.

It could be one person. It really that's, what actually helps you scale down your teams and handle much more with a lot less resources. And that's what I learned really fast. Because as you know, fraud never have never has enough resources. Um, but you do have to identify who is a part of your fraud prevention process. It's your fraud team. It's your fraud vendor. It it could be your finance team. It can be your customer service team. Your customer service is your front end. They are the first ones talking to the customer and talking to the fraudster. So collect the information from your customer service team, to have better and more impact impacting fraud prevention. Who is your fraud investigation? You don't have to reinvent the wheel here. You don't have to create a whole new process. Look for the processes that already exist, uh, that kind of complete this fraud investigation part. Which was typically in the old times finance team that handle chargeback responses. Then reporting teams. Every organization has reporting teams, or or BI teams, or data teams, whatever you want to whatever they call them. So instead of trying to create reports that may help you do this or that, look what already exists. So instead of wasting time of creating report on some of your order, uh, volume, maybe that report already exists. I'm 100% sure it actually already exists. So we use repurpose existing reports. Resisting an, uh, existing analysis to what it you need. And then once you repurpose what already exists, then try to create something that is already missing. Instead of like jumping into I need a fraud report. Well maybe data already exists somewhere that's not really reporting fraud, but it's indicating fraud. So go ahead and repurpose this. It's saving you resources, is making it faster, and it's including everyone else in the organization. And then strategic planning is, um to me, you pull the important individuals from your organization. Into this monthly, once a quarter, strategic planning meetings. So, but it's not just you. It's your marketing team. It's your customer service team, your finance team, your e-commerce operations team. You pull in everyone. And if they don't want to share with you what they do, which they should, then you share with them what you're doing. So they are aware of what's going on. And what you're going to do. So you can fight fraud better. And if they have any new new initiatives, any new promo codes, any new marketing um ideas, you know about them. So when this marketing idea is launched, it doesn't look like anomaly on your fraud prevention uh side. But it looks like a normal traffic that you expected. So you can adjust to it. So I think it's kind of a very difficult to set it up. But it works well once you set it up. And it's not really have a, it's not really that difficult. It's just maybe time consuming to set it up in the beginning. And that's my first thing. I'm sorry it's a little bit lengthy. My other thing, I'll try to be fast, is end to end monitoring. Fraudsters will usually show its face really really early in a process. By the time it gets to your payment, if it's a fraudster who is attacking your payment place, like using fraudulent credit card. You already missed at least four places in that process where you could have stopped them. So it's very important to look at the end of that end to end life cycle of that transaction and life cycle of your entire traffic. And identify risk gaps where you could have identified that fraudster. And try to move your fraud processes as high upstream as possible. Because then you have at least two, three, four more stops or or gaps where you could stop them. Instead of waiting for them to come to the payment page, and then take the action. I mean, there are so many more benefits to this, uh, than just stopping fraudster. But also you'll learn how your traffic looks like. And then you learn how to spot these anomalies a lot faster. Because you know if you have 100 orders in an hour, and suddenly you have 250 orders an hour something is off. So before this person even gets to your payment and you use your fraud vendor to detect fraud there, you could probably detect them and stop them. So, end to end visibility to monitor, how do I say, monitor identify mitigate.

Yes, that's what I've written down. MIM. MIM.

I didn't think of that. I need to give it a better name. I'm so not creative. I swear I should not be ever in charge of putting name to

Hey, fraud fighters are quote people. That's fine. Um, that is awesome. I mean what I like about it is that, you know, when you ask such a question ,it's very easy to be tempted to answer, um, you know with this new innovative, uh, approach or tool or policy or SOP. And I like how you, like, how you present it in such simple terms. It's just doing the basics right. It's just doing the basics right. And if you do the basics right, it doesn't matter what new technology comes in. Or what new threats, uh, come in. You have the capabilities to manage that. That that's why they are called basics. Um,

Correct.

So yeah, I I love it. I love it. Uh, and and I I must say that it like, it resonates with me so much. Because these are, again, like really the topics that I talk about as well. So I'm I'm I'm glad to see that we are in the same line here.

Well, you are a fraud fighter. So you understand what those basics are. And you understand those pain points. And I think we are up against much more than just fraudsters in this space. And having those basics done right is not just going to help you fight this good fight. It's also help is going to help you showcase how impactful you are to the organization. And how beneficial you are to the organization. So, um, I know that a lot of fraud teams, I hope it's not the case right now but I know it used to be. Every time you ask them about their success, they will tell you what their chargeback rate is. It doesn't mean anything to me. Your chargeback rate can be 0.01. If your approval rate is 50% you're not doing a good job. So I think that as a teams we also have to understand, what is what are the KPIs? What are the metrics that showcase our successes? That showcase our importance in our performance for that organization. And to me, again. That's approval rate, that's um friction, false positive, insult rate, whatever you want to call it. And then risk rate, your judgment rate. And they have to, they have to be observed in a relationship to one another, not just as an independent method.

Yeah, I think it came up with my, uh, it came up on my conversation with uh Gilit Saporta. Where I think, she she said it, that the the biggest. How did she put it? Like that the irony is, that the better you are as a fraud fighter, the more invisible your results become.

Fraud is never a problem until it becomes a problem.

Exactly. Exactly. You're either invisible or, um, you know in line to be hanged. Uh yeah. Um fact. Awesome. Dajana that was super super interesting. Uh, I want to kind of like go back and summarize a few takeaways that I noted down throughout the conversation. Uh, that I think are worth, uh, coming back to. Um so one, we said it repeatedly throughout the conversation. Going back to basics, uh, we'll detail that, uh, in a second. But I think the the main takeaway, um, from this conversation at, least in my eyes, is that going back to basics is not about making sure that you know the the the foundational floor is ready to build all the innovation that you need to, I don't know, like deal with. For example AI AI powered fraud. Going back to basics, and getting the basics right, is how you stop AI fraud. Is how you stop new threats that emerge and, uh, attack your business. So going back to basics is 100% of what a team needs to do. Um, it's not the first thing that you need to do. Um, you also mentioned part of that, and and I take it like the way that you mention it, I take it as, um, maybe one of the points that, uh, that you see, um, neglected, uh, more often. Uh, and I think again it echoes back, uh in in your in your recommendations as well, is visibility, That, um, going back to basics means understanding what your key risk indicators are. And these are probably different between businesses. And that tracking them, monitoring them is how you start. That's the monitor, the the the first stage of the circle of fraud. Um, that's how you start with, uh, the basics. I think one other thing that you, um, that you mentioned, um, I think by the way that that also came up in my conversation with, uh, with Gilit. Is that, um, another part of the basics, which again is often neglected, is collaboration. And I think you you you mentioned collaboration throughout our conversation, in two two types of of of echelons. I would say the first one is internally, uh, within the team. And maybe collaborate, or get closer to compliance teams, to AML teams, but also to cyber security teams. Uh, that by the way, also like very similar, uh um uh, examples were also given when I talked about it with, uh with, Cy Khormaee when we talked about should cyber security teams and fraud teams get closer to one another. Um, so this is one echelon. But there's another echelon, and that is externally. And this can be either the, you know, like banks or other partners. That uh, or or might be not even partners, right, might be that you don't really have an official relationship with. But you see a specific bank coming up in a in a fraud attack. So getting in touch with them might might do you some good. But at the same time, and I think this is even more important, peers. Peers that are, you know, that have the same role, wearing the same hat, in other other kind of like, similar businesses. Be it merchants, be the same, even vertical, uh, but not necessarily. And just exchanging what you see. Um, can sometimes, maybe, prevent a 10% fraud rate for 6 months. Uh so yeah. That can, uh, save your job. Uh, if you, if you collaborate, uh, pro, and I think, by the way, and I think this is also why you're such a prominent voice. I think there's a lot of collaboration that needs to happen here. And a lot of networking that needs to happen here. And networking doesn't have to be done on LinkedIn. It doesn't need to be done through posting on LinkedIn. Networking can happen in a lot of different manners. I think you embody that. Um, and this is such a crucial aspect of the role. That I think, uh, let's say, more newer fraud leaders tend to miss. Tend to really focus on my SOPs, my data, my tooling. And forget that, you know, a a a a real success lever for you can just be your network. Um, we also. Yeah, and this is where we went into like, really like what are the basics. And what are the things that you probably want to get right. So, you talked about your, I I dubbed it circle of fraud. I don't know if you have like a name for it, but that's that's the name,

360. I I call it.

The 360 approach.

360. I'll I'll get GPT to give me a better name. Yeah.

Yeah. Yeah. It needs like some AI dropped in there somewhere, uh, for the hype. But yeah, monitor, identify, mitigate. I love it. It really reminds me of my own concept of, uh, the reaction cycle. Which is pretty much the same thing. Uh yours, I must say, is much more to the point. Um, and um, I think what I liked about your approach is also the fact that, um, it is not necessarily or where you want to start at. It is not necessarily, okay these are the things that I need, let's build them. But actually finding out and mapping what already exists in the, uh, in the organization. Maybe, by the way, it exists like it probably exists somewhere else. Maybe in finance, maybe in compliance, maybe in marketing, maybe in product, uh, that you can utilize. And I liked it. Because not only that it, you know, it saves time and it saves effort and it makes you, uh, you know, not a needy, uh, resource train on the on the organization. But also that if you're using tools, especially by the way when it comes to reports and monitoring dashboards, and so on. If you're using these tools that, uh, serve other teams, you increase the likelihood that others, other other teams would be open to it. That would they would look at it as as well. In like the fraud, uh uh, mindset and so on.

You also create relationships and increase visibility. Because fraud is our job, but it's not only our responsibility. It is a responsibility of the entire company, of the entire organization. To know how to prevent this fraud. And bringing these teams in increases that visibility, and gives them that knowledge of their role. The role they play, and in in this fight

And and that relates directly to the other thing that you mentioned, uh, in that regard. Which is you called it the, or that's what I wrote on, the strategic, the monthly strategic cross functional planning meetings. Where you bring all the relevant, uh uh uh, parties. And I think especially, you know, marketing. You mentioned that product, probably, as well. And you basically align in advance, so you're not surprised by any kind of initiatives that would hit your, uh, hit your side of the of the team as well. Um, that that is like 100% agreed and endorsed, uh, from my side as well. Lastly, uh, you mentioned end to-end monitoring of the user agent. Um, which I also love. It came up in my conversation with Matt Vega when we talked about how do you want to to do fraud prevention in real time. And one of the things that we talked about was that you don't want to do that in real time. You want to do that well in advance. Of, you know, arriving to the transaction, or the monetization event. Um, so that is definitely something I I agree with. Uh, and I love hearing from you as well. Um, Dajana, that was, wow, we covered so much. Uh, I'm glad that we managed to talk about AI, but also not really talk about AI. Uh, that was a breath of fresh air. Yeah. Uh, definitely. Um, thank you very much. That was like super awesome.

Well, thank you so much. It was really enjoyable talking to you. And I hope that we shared some of the information, uh, information to your listeners that they can practically implement and use in their jobs. I hope it was helpful.

Ah, I'm sure it was. I'm sure it was. I have no doubt. Um Dajana, thank you very much. It was a blast. And for all of you listeners at home, I see you next Saturday. YouTube Description

When I speak to fraud teams that have started using Agentic AI, I find that most started in the right place. They're piloting agents inside investigation, enriching alerts, structuring cases, and recommending resolutions for investigators to validate. And for the most part, these teams are seeing clear efficiency gains from using agents. The problem I see is that almost nobody's going further. In a previous video, link is down below, I argued that the master KPI in fraud isn't precision or accuracy. It's the reaction cycle. The time it takes your system to detect a gap, be a new fraud attack or misbehaving in fraud controls, and ship a fix for it. And while fraudsters are deploying AI to adapt faster to new defenses, fraud teams should also use agentic AI to match that speed. Now, automating parts of your investigation process is just the first step in that journey. It is not the journey. Why? Because investigations are just part of the entire process that makes your reaction cycle complete. Even if you automate that one step completely. Which, admittedly is easier said than done, the rest of your links in the chain are still run at human speed. So yeah, your investigations are going to run faster, but you're still running a broken loop. The rules you deploy to flag events in the first place are still degrading. The labels you're feeding your models are still arriving weeks late. Now imagine for a second what it would mean if the entire cycle ran at machine speed. Your system then becomes something entirely different. You make decisions on entire rings instead of single individual events. You're not running faster investigations anymore. You're making decisions at scale. And each of these decisions not only reduce your workload, but they also generate hundreds, if not sometimes thousands of new labels you can use immediately. And if you can generate labels fast and at scale, you can then use those to refresh your rules, your policies, and your models.

And if those are refreshed more frequently, then you can mitigate how fast they degrade. Then in essence, you've built a reaction cycle that runs at machine speed. Sounds great, right? But also pretty complex. And one of the questions I hear most from teams who are making their first steps with AI is where do we start? In this video, I'd like to share the five steps it takes to transform your reaction cycle from running at human speed to running at machine speed. Each step produces an input the next one needs. Or builds the capability the team needs to govern it. So, you want to make sure you first identify where you are in this journey and then follow through the next steps in the right order. Skip any and you'd likely get rule automation running on state labels or policy automation nobody knows how to govern. And that's exactly how AI adoption projects fail.

Step one, the first place we start at is the same one we already mentioned, fraud investigations. After all, there's a reason why most teams start there. It's the most straightforward and natural place to pilot AI agents. This is also where Agentic AI is most mature today, where your team's expertise is likely concentrated and where the savings show up first. Specifically, you want to cover three areas. The first one is handing over data enrichment from the investigator to the agent. The 15 minutes of pulling device data, identity context, account history, and external lookups now gets done in parallel and attached to the case before the investigator even opens it. The second one is to structure case output. The documentation, reporting, suggested next steps are now uniformed across all your human investigators and match perfectly your SOPs. You also get the added benefit that your cases are now logged in a machine readable form, which is what you'll need if you plan on using it further downstream for other agentic flows. And finally, the third area you want to hand over to agents is explaining their reasoning. Not only recommended decisions. A because you anyways need it for the regulator if and when they'll come knocking, but more importantly B because that's when the investigator's role shifts from making a decision to validating it. For many on your team, especially the less senior, that's a new skill and critical skill to learn. And this is the lowest stakes way to start practicing it. When you've done all of that, you should notice individual investigations become not noticeably faster, but the real gains are still invisible at this point. It's making the investigation output better structured, standardized, and machine readable.

Once investigations are structured, we can move on to step two, having AI agents cluster them. As they come in, new alerts get matched against known cases or fraud rings. We do that by matching device fingerprints, funding patterns, or behavioral signature. If we find a match, we can attribute this new event to an existing ring without the human needing to spend time on that. And if we don't find a match, the new alert becomes a candidate for a new case by comparing it to other existing alerts in our queue. In this way, the investigator's queue stops being a pile of unrelated alerts and becomes a curated set of assembled cases. Render expansion happens automatically and unlocks your team's true efficiency as one ruling now labels thousands of events at once. And this is exactly where the structural work we mentioned in the previous step pays off. Teams that skipped it can't build clustering at all.

Okay, so far you may have gained in efficiency, but your reaction cycle as a whole is not meaningfully faster. Step three is where we change that. And I can't stress how important it is. You see, the biggest bottleneck in your reaction cycle is labels and specifically getting fresh labels fast. We all know that chargebacks arrive weeks after the transaction, sometimes months. And by the time they do, the attack pattern either already changed or created big losses. Many times it's both. And that's not only relevant to chargebacks. Confirmed fraud cases take time to appear. Why? Because a human is the one who usually triggers them. Whether this is a customer who files any kind of dispute or an investigator that looked at it and ruled it as fraud. And that's exactly the bottleneck we want to remove from our reaction cycle. We don't want humans to slow it down. But with agentic AI, you can close this gap with relative ease. You already have an AI agent proposing investigation outcomes and by now you should have already gained the confidence needed to start removing the humans from the loop entirely. Maybe not for all the cases, but definitely for most of them. At this point, you are taking the AI recommendations you introduced in step one, combine them with the scale you established in step two, and you start automating it. And the thing to remember is that labels for training don't have to be perfect. These labels shouldn't affect anything other than your training data set. They only fit model retrains and rule back test, not customer facing decisions. Now, can wrong labels potentially undermine your decisions? Theoretically, yes. But in practice, that isn't such a big concern. Rules and machine learning models already work with noisy data. A modest error rate from an agent is no worse than the noise you already have in chargebacks and investigation rulings. After all, humans make mistakes, too. And this tolerance for imperfection is what makes continuous labeling at scale possible. So by the end of this step, the slowest part of your reaction cycle just went from weeks to minutes. And that's when the real possibilities open up.

Okay. So where do we go from here? In step four, we're starting to introduce AI agents into your risk segmentation. What's that? Segmentation is the layer that decides how you'll process and assess the risk of each event. Which rules fire, which vendors get called, or whether step up authentication is triggered. To be frank, many fraud teams tend to treat this step as an afterthought. And even the ones that don't usually resort to set and forget policies. But this is the layer where you actually implement your fraud strategy and ignoring it slows your ability to react to emerging threats. But here also lies the problem of AI adoption. Segmentation likely doesn't have a defined owner, goals, KPIs, or even a process for moving populations between different risk segments. And the thing is that you simply can't automate something nobody's managing manually first. Your team has to learn what good looks like before it can govern an agent that proposes changes. There's also another difference worth flagging. This is a new motion for your team. The earlier steps all follow the same pattern where the agent observes the live data pipeline and processes what goes through it. But recommending a change to your risk segmentation works differently. The agent analyzes historical performance across populations and proposes policy changes, not case rulings. This isn't just semantics. It changes how you test, validate, and observe agent decisions. It's much more data driven and much less expert-driven. This means that the team which will manage it would likely not be the team that runs investigations. But the team that manages rules, models, and other analytical components of your system. Meaning fraud analytics. This is where we're starting to expand the use of AI agents to more teams and processes across your organization. Having said all of that, I also want to stress that this isn't as intimidating as it sounds. Let me give you an example. An agent recommendation can be to take a segment of accounts aged 25 to 30 day old and move them out of the high-risk bucket and into the medium risk because their behavior looks like established users. So, you see, this isn't about finding new fraud patterns or emerging threats. It's more about optimizing existing segments and processes. That makes the problem much more bounded than fraud rewriting for example which makes it exactly what you want to do to build that muscle.

And then lastly, step five touches exactly that. Handing over your detection layer to AI agents. What is the detection layer? It's where you run your signals and the algorithms that use them to detect fraud. Or in simple words your rules and your models. When we are able to refresh these in an automated way, we expedite the last phase in the reaction cycle. Which is actually reacting to emerging threats. Not only detecting them. So why is it the last step if it's so important? Well, there are two reasons for that. The first reason we've already covered. Fuel detection automation needs fresh trustworthy labels to be useful. And without automated early labeling already in place, an agent proposing new rules is just proposing wrong rules faster. We don't want that. The second reason is that governing agents that create and tweak rules is complex. And that is especially true if your team hasn't built the muscle for writing rules themselves or managing agents that run deep data analysis like we've done in the previous step. Because unlike in segmentation where the agent pick between options that already existed, detection is generative. It proposes things that didn't exist before, new rule patterns, new features, and possibly even new machine learning models. Mastering agentic segmentation gives your team the governance experience agentic detection requires. And once you've mastered this step, you're running your reaction cycle at machine speed. Alerts come in, get clustered into ring level cases, get label automatically, and fuel agents running continuous analysis for segment changes and new rule recommendations. And that, ladies and gentlemen, is the future of fraud ops.

Now, let's be honest. This isn't a one quarter project. The teams further along are 6 to 12 months in, and they'd say they have meaningful work still ahead. Each step needs tooling, organizational change, and time to absorb a new way of working before the next step makes sense. And fraud teams are facing a real risk here. On one hand, the pressure to adopt AI and scale back headcount is growing by the day. But on the other hand, there is very little clarity on how to do it safely. In such an uncertain environment, it's crucial to understand your end goal. Not just lower cost, but a better organization. The sequence I just shared isn't only about where you can cut more costs. It's also about clearing dependencies and growing new team capabilities and skills. Ignore these and you might need to hastily rehire the team you thought was redundant. And we all know it's not that simple. YouTube Description

Yeah, thanks. It's good to be here. It's good to see you.

Likewise. David, you and I met, I think, through LinkedIn, right? Around two years ago or so. And I remember us having the first conversation and we had like a very quick kind of like meet and greet call. And I remember, you know, saying to myself, okay, this guy gets it. So I'm I'm super excited to have you on the show. You're a veteran of the fraud prevention industry. You've worked both on the vendor side as well as the financial services side. But for those who don't know you, David, tell us a bit about yourself and your career so far.

Sure, and I'll do it chronologically. So at American Express, I was the, I grew up at American Express. And I spent fifteen years there, about a little less. And I ended up being the owner of fraud prevention for all, basically all of the non-card products end-to-end. And we were a lot. We were the fourth largest bank. We had corporate client onboarding, membership rewards, business loans, savings accounts, checking accounts, SBA loans, et cetera. And then also I was responsible for new account fraud prevention globally as well. And then after that, not chronologically, I worked as the product owner for SoCure's fraud products. Which is about half of them. So those 50% of SoCure's product team, their product responsibility. And then also for Trulio, their fraud products. And then I've also been a consultant for a good number of years, helping both solution providers and operators, all sorts of different kinds of solutions.

That's super impressive, David. I know that you've seen a lot in your career. So I'm super excited to have you on the show, as I said. And very anxious to deep dive into the topic that we have today. So we had, we were kind of like catching up a couple of weeks ago. And we were also talking about you appearing on the podcast. And initially I asked you what do you want to talk about? What's top of mind? And you, I think I don't remember what were the topics, but you mentioned two or three topics. And when we talked, you immediately kind of like said, I want to talk about the future of fraud. And what's coming in two, three years. We said okay.

And even now sometimes. Like the future is now. The things speed up so much.

Yeah, yeah. Things, the time horizon gets blurry. Yes, I agree. And I, and I said, you know, yeah, let's definitely do that. And I really want this conversation not to be about scare tactics, and kind of like, you know. That is something that the industry is too good at, unfortunately. And I, and I do want us to take kind of like practical takeaways that fraud fighters can do something with. But let me start with actually asking, are you worried? Is this a top of mind topic because you're worried about it?

I'm not that worried. Because I think what always happens everywhere, every institution, is things are going fine. Something bad happens. No no, the sky's falling, people scramble, and figure out a way to fix the sky. And then they go, fine again. And that that's BAU. It's just, most of the time you can't describe how the sky will fall. And now, because of the rapid change in technology, it's really clear how the sky will fall. But that's always what fraud has been like.

Um, yeah, I mean, first of all, I agree. It's a pendulum swing. And I would say that, you know, something happens, as you said. We rush and fix it. And then the next thing is, that we block too many good users, and we have too many false positives. And the, and that's the BAU, right? This pendulum shift. But I, I'm curious to hear your thoughts about it. Because you know, you said it's very clear how the skies are going to fall. But I think that for a lot of us, and also including myself, I don't think it's like super clear. And I, and by the way, I will admit that I'm much more attuned to fraud fighter teams, or fraud fighting teams, than the fraudsters themselves and what they are up to. So it's always interesting to talk to people who are, you know, like more in tune, and are looking at that side as well. But share with us, like let, let's get a scary part out of the way. When, you know, you see AI. And you, and specifically Gen AI, and you were thinking about two, three years into into the future. And we already have deep fakes and we already have synthetic identities at scale. What are the unpleasant possibilities that go through your mind? How do you see fraud evolving?

Well, I think we're already. So I'm gonna focus, like I think this podcast is, this particular episode will mostly talk about Gen AI. And what that enables. And it's not just Gen AI in terms of you can use a large language model to do stuff. It can also be, you can use to do attacks and interact with people, but it can also be you can use one to do coding or or to build software. Which maybe in the past would have been much more difficult as a solo fraudster or solo entrepreneur. And right now already people are seeing scams happening, at like greater scale. And more personalization, and phishing happening at greater scale, and more personalization. Like, and we all see this. Because we, our phone explodes. And we're like, this is a better scam than we'd usually see. But I'll give a specific example and then we can maybe jump from my new account example to maybe an account takeover example or a money mule example. But my new account example is, if I were a fraudster today, which I'm not, I would probably use something like an OpenClaw and spend time training the thing. And then give it some high-level instructions. You know, I do my planning in one place. And the planning would say buy a bunch of phone numbers, get access to, like an Equifax data breach, you know. Load it with some kind of digital currency, Monero or something. And find out who used to own those phone numbers. And find out the identity of those people. And then buy a bunch of emails, and these emails should be tenured and not have a name in them. You know, loves to play basketball at gmail.com for your old email. And now that you know that, and sorry, Chen, you're now the victim. So now that I know that this phone number used to be owned by Chen, and I have this tenured email, and I know Chen's PII from a large data breach. I can also say, find a residential proxy. And buy access from the same city that Chen's in. And create a device that's plausible. Use just a common random device. So you can beat typical device ID controls. And now this, this new application for a loan, or whatever it is. Or buy now pay later, or whatever. Becomes really hard to detect as fraud. Because it's Chen's phone and email. Coverage isn't a hundred percent, and it's a tenured email. We can't say it's not Chen's email, and the device signal's weak, the IP address signals really weak. So how do you, how do you see that it's bad? So that's the first thing. You can't even detect that it's fraud.

Yeah, so sorry. I mean it, just because you mentioned OpenClaw. Obviously I heard the name, but can you, can you explain a bit what it is. And why it does, like what are the new capabilities that it provides fraudsters with? Because it sounds to me, like what you just described is something that I can imagine how a fraudster can do that exact same thing. Also today it might be very hard and very expensive to do. How would OpenClaw change that? What is it exactly?

Well, I think it's more about this, and and I don't mean OpenClaw itself, I mean similar agentic frameworks. But it's more the point that the automation becomes dramatically easier. That you can give hints, and then an agent can either through code or interactively as a loop, sort of perform this attack at really great scale. Much easier than it could have in the past. And now I can say, just find a residential proxy. In the past, that might have taken some work to find a residential proxy. That maybe could accept your your digital currency, which is sufficiently clean. You know, just very time consuming. And to do it at scale. And to do it repeatedly. And to do it with a new one, new proxy that matches the city and state that Chen lives in. You know, so each step in this requires a degree of customization, that in the past would have been pretty hefty, and probably not done at scale.

Okay, so you're saying that the new emerging threat here is the fact that for fraudsters, to conduct sophisticated attacks at scale is something that A) they are now capable of. So it's like zero to one, like many fraudsters are now like at a one instead of a zero, and and two, that it is also way cheaper for them to do.

Yeah, that's right.

I can imagine, and I and I know that on the dark web fraud has been industrialized. In the sense that you have service providers. And that's the case for many, many years. And I'm wondering if these capabilities were not always, you know, for sale. You know, for maybe large sums that not all fraudsters could afford. And like, you know, I'm wondering what does it do to the fraudsters' economics themselves? Where maybe, you know, their own services are going out of business. Because now anyone can do that with AI. It's the Suspocalypse.

A fraud suspocalypse. What a funny idea. Certainly I think that there's a lot of shifts in who the players are. So many times I don't think an individual fraudster will want it. Like in general the fraud tools haven't been that expensive. Right? And so as a fraudster, it's probably not worth it to build your own. Because the price of the tool has never been that much. So I don't think that it's like, I'm going to build my own because this tool I used to pay $200 for, you know, I can build it myself. No, it's still not worth the two hundred dollars to build the tool. Instead, what happens is just within, same as within the fraud vendor space, that there's a lot of change that with new technology. And you'll find that one vendor displaces another as technology advances. And so I think the rate of that change is happening.

Yes. Interesting. Sorry. So, you were about to describe how similar attacks can be performed on established accounts as well, right?

Well, and let me go back to something. So you asked a great question. I love the question. You're like, of course I did. Which is that, this thing I described, how is that new? What does that have to do with AI? I love that question. And the answer is like, if I was doing it with one account, and I was gonna spend a lot of time to do it, and I had the skill set, it doesn't have to do with AI. But now, many times there'll be pieces where I don't have the skill set. Or I'm not aware of the control. And AI can help me become aware of the control. And it can help do the thing. And then on the other side, you can say anything in the past that I could have done once, but it would have taken me a long time and a lot of effort, now I can do sort of at really high scale.

Yeah. And that goes also kind of like to the self education, and self training that many fraudsters. I mean, you don't go to school to learn how to defraud, right? You must be an auto deductant. And with AI, and the capabilities that it enables, I'm guessing that it is much, much easier to to do so today.

Yeah, no, that you can turn that into a quote. You should excerpt that, you know, something Chen said in this podcast. This is a very, very good insight. But anyway, where I was going was that, we could come up with different examples of scenarios across the entire customer life cycle. Or different kinds of fraud attacks, where you're like, this fraud attack, this example, it would be really hard to detect. And then on the other side, you could say, well, how do we challenge today? What's the traditional way of challenging? And today the most typical way of challenging is to either send a code to your phone, if we know it's your phone, or to do document verification. Take a picture of your face, take a picture of an ID, do they match? And we know that almost all, probably all of the major document verification fraud solution providers, have had large fraud attacks. Which have beaten them. And then they scramble and they try to fix it. And it's becoming more and more prevalent. It used to be three years ago, let's say one person in a thousand or less, one in five thousand, would beat document verification solutions. And if I today, if I were to speculate, it would be one one fraudster in a thousand. Now if I were to speculate it would be like one fraudster in fifty or a hundred or maybe less, 25. Like it's still a useful control. I mean, it still stops the majority of the fraudsters. But on the other hand, clearly there are fraudsters who can beat it at scale, and they are, regardless of who your provider is. And so you can't, you can't identify it's fraud. And even if you were lucky, and you somehow were able to identify it was fraud, you can't stop it. You can't challenge. It's like a nightmarish possibility. So that's the scary scenario that I'm painting. And then the the obvious question is okay, so if there's such a scary scenario. Well, what do we do about it? And I have some proposals, but I'll throw the ball to you. And why don't you say what we could do about it, and then I'll add on just for fun.

I mean, let me ask you before that just one question. Because the picture of, that you were just painting of fraudsters being able to seep through. Or kind of like penetrate any type of, let's call this family KYC vendors, right? The KYC vendor family. Whether it's Doc V, liveness checks, it's the same. General KYC vendors. And I completely agree with you, like we're seeing exactly the same thing. I'm wondering if you see that as kind of like a new situation, a new status that we're at. Or do you think that this can actually, even get worse, like two years from now? Where do you think are we, at the arms race? Who's winning? Or who's ahead?

This moment in time, as in recent past, I think the financial institutions, the companies, are winning at this moment in time. But I wouldn't be surprised if there was a dip where the fraudsters just started to win. And it took the companies longer to pivot, and then I think the companies will win again.

Just about on one hand who is nimble enough to make quick changes, but once the technology kind of like let's say becomes more stable, I would say, and riches kind of like its peak, yeah. Yeah. Whoever has the more resources.

Yeah, yeah. So let's jump to structural advantages. So I'd say the fraudsters have a lot of structural advantages, even more so today. For a fraudster, they can try once. And if they fail, they learn. And with a financial institution. It, or you know, an operator. You know, if they, if they see one fraud case, they may or may not learn. You know, it it's just the teams are stretched thin. With a, and the attacker can iterate after every attack, especially if you're using AI to automate. Even with AI, most companies can't iterate after every attack because you have to look at larger samples. You have to see the impact on good customers, you have to wait a few days. Maybe you have a, an A B with a control. How long does it take the control to react?

You have regulators

Yeah, yeah. You have to not break the law as a company, where as a fraudster, you don't care. So so I think there's a lot of advantages in terms of speed of learning, sample size, that favor the attacker. Especially today. And that's why I think, along with that bleak scenario I just painted, while there will be a dip and the fraudsters will start to win more aggressively for a while. But large companies also have meaningful advantages. And for all that fraudsters are quote good at information sharing. I put that in air quotes. In the sense that, you know, they'll sell products to each other. And that the life cycle gets coordinated by specialists from, who hand off from one to the next. On the other hand, like the degree of investment from one entity can, at the company scale, like much larger. Than even the largest fraud ring. You know, the largest fraud ring might have, who knows, a few hundred people. Of whom how many are sort of very technologically savvy? I don't know, a dozen, less. Whereas a company could easily have many hundreds of people who are technologically savvy, working on just one specific problem. And so I think the benefit of the people fighting fraud is the benefit in coordination. So we don't share, companies don't share information with each other, that well. But within a company, the degree of information sharing is fantastic. And there's a lot of smart people working on the same problem.

It's interesting. It's exactly the same thing that Shachar Meir brought it up in in our own episode a couple of weeks ago. Where he basically said almost the same thing. I think his point was more the coordination between different organizations, that this is something that we can do through consortiums. But I think your point is actually a very interesting complementary add-on. That, you know, pertains to the fact that we also are coordinated within the organization. We actually have much more firepower, much more resources. Budget to solve the same problem than what frosters have at their disposal.

There was this bleak scenario. We're gonna have this hypothetical attack, what do you do? And I can propose the solutions I've thought of. But I'd be, you know, you can also. How would you like to do it?

No, I wanted actually to ask. What, why is it so scary in the sense, that like in the end. We know fraud today. We have fraud, we have fraud for thousands of years, especially in the digital world. Gen AI is not that new, right? I mean, we we've seen the implications over the past two years. And even agentic, we are starting to see the implications. And I'm not saying that there are no challenges here.But, you know, we are not necessarily seeing fraud rates going, you know, tenfold or even fivefold. So I wonder,

Yeah, yeah, no. Maybe it's even getting a little bit better.

Okay. I would be interested to to hear more about that. But I would ask, why wouldn't the current system, the current kind of like, you fraud fighting system, fraud prevention systems that we have within the industry. Whether this is in financial services or with merchants or with digital platforms. Why would it not be enough? Like where would they break with these new capabilities that frosters are now adopting?

Yeah, so I think in the past, almost everyone says, Well, there's a certain amount of fraud I expect. And I'm gonna budget for it. And so when I have large fraud attacks, where something breaks horribly, I'll lose many millions of dollars. But then there's the day, I'm gonna call it the day-to-day fraud, where I just expect I'm gonna lose X basis points to fraud. And I'm fine with that. And that's most large companies today. And in the past, we say, well, the fraudster attacks. There's a certain manual nature to the attack, they have to iterate. But any one particular fraudster is unlikely to be able to scale up their attack to large volumes in ways that I can't detect. And the argument now is, many of those people who were your three basis points or whatever it is a fraud. Now will be able to scale up their fraud attacks in ways you can't detect. Not many of them, some of them, when in the past they couldn't. Because they didn't have the tools. Or it was like too overly expensive. And obviously it's not all of them. Like, you know, I, if I see your credit card at a restaurant, like that's not something I can scale up. But some of the attacks that in the past weren't scalable, now will be.

And you're saying that, what we would be consider, like what we would consider to be a very sophisticated attack. Let's say, the top, you know, ninety-ninth percentile in terms of sophistication, would basically expand to be the ninetieth percentile in a couple of years' time. Just because the like fraudsters would have more access to the sophistication. Also to the scale.

And especially the scale. Like maybe some fraudsters will get more sophisticated.But I think more the point is, if there were a hundred of fraudsters, and each of them attacks, whatever ten times. The person in the ninety ninth percentile, who attacks ten times, and beats all of your controls, now he has the tooling to attack a thousand times. Whereas in the past, maybe he didn't. Or it would have been, like much more difficult. [Ad Break (23:31): Hey folks, I want to take a quick break to speak about today's sponsor, me. If you're finding this useful, do me a small favor, like and subscribe. It really helps with the algorithm. And if you're not already on my newsletter, The Saturday Fraud Strategist, you should definitely check it out. Every Saturday, I break down fraud trends, real cases, and strategies from my own experience of building and operating fraud prevention systems. Whether you're new to the space or a seasoned practitioner, I'm sure you'll find it interesting. Check the link in the description. Takes 2 seconds. You'll thank yourself next Saturday. Now, back to the video.]

So let's talk a bit about, you know, how things, like in an ideal world. Where we have all the budget, all the resources, all the technology, and we've already gone through the transformation journey. Where we've implemented everything.

Okay, okay. So ten years from now.

It's like this, right? Just a flick of the fingers. What should, it's not a modern it's like a futuristic fraud prevention system, should look like? So it can actually stop these undetectable fraud attacks, or patterns, at a scale that today we cannot really imagine. How should it look like?

Yeah. So first I'm gonna caveat this by saying, I'm not gonna assume any radical changes in society. So I'm not gonna assume every person has a digital ID that they go, when they go to the DMV. And with a fingerprint biometrically bound to all this stuff. So I'm gonna assume, sort of very modest evolution in the broader world. I don't believe that device intelligence will be that robust for that long. That it'll just be too easy to spoof in general. Like you can know it's a unique device, but you won't be able to say, this device is, you know, an emulator or something like that. I think that will be very difficult. And privacy policies make it harder, not easier, right? Regulation makes it harder. But I was talking to Matt, actually who used to be at Sardine, and he gave me this wonderful example. Where he Yeah, yeah, Matt Vega where he was saying that he saw, that the bot clicked right smack in the middle of the button. Like the exact middle of like the submit button or something like this. And I, I think this brings out a broader point. Which is that companies, if we ask how does David apply on a Bank of America website for a new credit card? Like Bank of America has millions of applications, it has millions of data points. This is how new people apply on their website. No larger language model has basically any data on that. And if you tried to train it, maybe you'd have like a few people at most. And those few people wouldn't be actually natural organic applicants anyway. So I think the behavioral biometrics, I think that will be resilient. And will stay applicable for a reasonable amount of time. Like you can add jitter, you can add, you can make things look curved instead of straight lines. But it's a lot harder to make it look like David on the Bank of America website. And the machine didn't know that human beings don't click s right in the middle of the button. But there'll also be much more sophisticated features, which will be also difficult for machines to, to emulate. Because the fraudsters don't have that data. So I think behavioral biometrics will continue to be resilient. And will get meaningfully better over the next year or two at detecting, you know, agents who are controlling devices. Who are trying to emulate humans. So that's one.

But I want to challenge you on that. I mean first of all super interesting. I, I didn't hear this angle. Why? Because usually the angle that I'm hearing, and also an angle that I'm raising myself, is that the main problem is that we will see more and more legitimate users using agents themselves. To, you know, like basically organize or conduct their presence online. And it, you, it would like at the same time that it is you know hard for fraudsters to train their models to behave like legit people, legit people will more and more use agents that would look more and more like bad agents. So how, how do you see that?

I love that. And I like, I like that you challenge it. It makes it much more fun. So I think this is a place where regulation and expectations help us. So this particular tax scenario is at time of no count on boarding. I'm applying for a loan at Bank of America, whatever. And in that context, I think there will be very few institutions, and I don't think many regulators also will like the fact that one could ask an agent to apply for me. And I, and I maybe haven't read the terms and conditions and I maybe haven't clicked that checkbox and I haven't been the one to click submit. So when you talk about checkout, I think it's very plausible that agents will do it for human beings, and that's a desirable state. But for onboarding, I don't think when it's a meaningful financial relationship, that that's something companies will allow or regulators will allow in the near future.

Yeah, that's super interesting. Because it shows that, it will be very much use case dependent.

Yeah. Yeah, that's right.

And yeah, and financial services would probably have very different experience. And maybe even, like between different flows within the same organization, they'll have very different experience with how easy it is to combat fraud. Versus how easy it is to spot false positives. Yeah. Interesting.

But, I mean, let's talk about terms and conditions. Does it count as having read the terms and conditions, if your agent is the one who executed the action? And my guess is most people would say no.

Yeah. I think, I think the question is not what most people say. The question is what the regulators say, right?

Yeah, yeah, yeah. Well, at least for highly regulated institutions at time of onboarding, I'm pretty confident the answer will be no.

Yeah, I agree.

Anyway. So because of all of this, I think behavioral biometrics will be, continue to be really useful. Then after that, there's all the information you put on the application, the phone, the email, etc. And in my hypothetical example, I said, well, we're compromising the phone. And the way we're doing that is by getting a different phone number that was associated with you. And an email, but the email's not associated with you. And I think coverage of email ownership in the US maybe is 50, 60%. So ownership is not good enough, really, to use as a primary control. However, usually the fraudster won't, usually there'll be a change in the behavior of the email. So even if I buy a very tenured email, there are companies out there, you know, marketers, who will say, how often do you open your emails? How often do you click links? What's your typical behavior? And that's not something that's typically provided today by fraud solution providers. But I expect that they will be providing that in the next year or two. Because then the signal becomes, well, this email is behaving very differently than it used to behave in the last three months. So the email's four years old. And the last four months, three months, its behavior has been very different. Then that becomes a red flag. And I can't verify its dated email. So even though it's tenured, I shouldn't treat its tenure as four years. I should treat its tenure as three months. And maybe this is worse. Actually, if it was four years old, if it was four months old and it behaved consistently, that's better than if it's four years old and the last four months has changed. Like usually people will abandon emails and they'll become idle. It's not like I find an old email that I didn't use and now I start to use it a lot. That's actually a particularly bad kind of behavior. So I think you'll start to see more around the usage of the contact channel. So this story around email, you could do the same thing with a phone number. Like, is this phone getting or receiving text messages at the same rate as in the past? So I think usage will become important. And then with phone numbers where ownership is pretty good. You might get 85%, 90%, whatever it is in the US. I think we'll also start to see more focus on like, well, yes, this phone number was David's. But it's not David's currently active phone number. It's not the primary phone number. And so then it won't just be, is this. In today, it's usually sometime we saw this phone number attached to David. Like that's the solution today. The solution in the future will be, we saw this phone with David. But we stopped seeing, notice that it was attached to David nine months ago. And to some extent, you might get this from like a phone porting date or something like that, from some solutions. But many solutions not. And then not only will you say, is this David, was it when did this phone number stop being associated with David? You'll also get the phone's activity. Like has the phone's behavior changed in X time period? And then finally you'll get what's David's primary phone number, or primary phone numbers, primary email and primary emails. And so then if you see something that was associated, but it was a long time ago, or it's a very tertiary email or phone where the pattern of behavior has changed, then all of a sudden you can say, well, I don't completely trust that phone. I trust it much less. And these are signals that aren't surfaced today. But the data is absolutely there. And now, that I think the need will be coming soon. I think solution providers will start providing these.

Yeah. Basically you have to do much more data mining, to extract much more context and insights. And especially when it comes to whether it is established accounts or establishing your ownership on specific like assets. Like phone numbers and and emails. You're saying consistency and like measuring consistency is key here basically. Because this is something that fraudsters cannot train, even with AI. They cannot train to act and behave like you.

Or at least the cost to do so is exorbitant.

Yeah, yeah, yeah. Um, I want to ask something. I mean, that made me very curious when you mentioned it a couple of minutes ago. You said I believe that device fingerprint, or device intelligence would be much less reliable in the near future. Can you, can you elaborate on that? Like why, why do you feel this is the case?

Well, I mean, even now, like this is not a new thing. Ten years ago. Actually. So it was easier, I think, to produce device intelligence ten years ago before the browsers started restricting access to a bunch of stuff, right? And so the fraud cover, coverage from a device solution, a very good device solution, ten years ago, or the ability to identify re-returning users. Was let's say fifty percent better than it is today. And that's because the browsers keep restricting what's available. You know, you hear all these speculations about like Google's privacy sandbox and privacy initiatives. And the EU, and what you're allowed to do. But beyond that, even ten years ago, there were many fraud attacks. Which would have a unique device that seemed plausible. That didn't have any dramatically risky signals. That beat at least many name brand device ID vendors. You know, I saw three of the top five. All of them had this thing, had happened to them. I'm sure the other two I didn't see had the same phenomenon. So I don't think device, typically device ID was used either. Because this device ID we saw with 20 different people. Therefore it's high risk. Or this device ID we saw with David, and now we saw him use the same device ID six months later, therefore it's low risk. That was overwhelmingly the typical use. And yeah, there's a whole bunch of features that device intelligence providers provide of like, oh, maybe this is a RAD attack. Maybe this is, maybe this is an emulator, blah blah blah. But I mean in terms of model predictiveness, all of those features for the major device ID providers I know, accounted to for like less than six percent of model predictiveness. You know, these weren't that important.

Interesting.

So I mean it's not a new thing actually. This is an old thing and it's just made worse.

I cannot say that I'm surprised. Because I think indeed device ID used to be such a strong, I mean it I think it still is. It is such a strong, let's say, feature. Exactly when you are able to link it, either to other users or to basically the consistency. Again, going back to consistency, of the same user behavior. When it comes to like extracting the rest of the intelligence, I'm not sure if the fault is with the fact that this intelligence doesn't matter. Or that like with, you know, what we just discussed about email and address and phone, that organizations never really bothered to really extract the full potential from that intelligence. Especially with high quality, and especially with accuracy, of the on the road data layer. Because it wasn't needed, at least so far.

I mean, I think I know of at least a couple organizations that looked at a lot of the variables. You know, like if there was an output of like three hundred variables, they'd throw the whole thing against some automated rule creator or something like that. And just most of the variables weren't that predictive. Now and to be clear, like if you say, I think this device is using, is from an emulator. Is that likely fraud? Absolutely. Or at least much, much elevated risk. Should companies use those signals? Absolutely. But on the other hand, the prevalence of that signal will be pretty rare. So there's a lot of signals, they'll be very predictive, but they'll each one will be pretty rare. And so it's not predictive enough. But to be clear, like even IP adjust geolocation, like everyone knows you can, you know, use a residential proxy. Appear like you're another place. IP address geolocation in theory shouldn't work at all. And yet it's still like a top ten variable in lots of models. So just because the solution is beatable doesn't mean you shouldn't use it. Like you should stop the stupid fraudsters. They're making it easy for you. Use it.

I agree. I agree. Yeah. okay. So we talked about, we have all biometrics and that this is going to be still prevalent in stopping fraud. We talked about extracting more intelligence specifically from email and phone. What else would a futuristic fraud prevention organization should look at when it comes to stopping these undetectable fraud attacks?

Yeah, another signal, I so when we talk about phone or email, then that's talking about making these traditional signals more useful. Because we have, you know, is it David's, is it David's primary. What's the the kind of behavior over time? But there's also the passive signal. So there's behavioral biometrics. Another one that I think is really nice is that there's a a number of different data providers who make available passive signals, saying this IP address or this device is attached to David. And is attached to this address and this name and this identity. And so this becomes then a positive signal. And like I mean a a classic example is, is silent network enrichment. Where the cell phone carrier will tell you this IP address is, this Verizon IP address or whatever. T Mobile IP address is. You can ask them. This is the phone number I got, and they can say, yeah, that phone number, that phone number is paying for this IP address. And then you can say, and that phone number is David's. Therefore, David, who's paying for this data, is the one on the other side of the screen, and that makes it meaningfully lower risk. So that's it one example where you can use passive signals to deterministically link David to this IP or device. And there are a few providers who do this in different ways, using marketing data, using telco infrastructure, using consortium data. And I think the problem in the past is each one only had so much coverage. Like, you know, maybe they'd have 20% coverage, 10% coverage, 30% coverage. But I think there will be a future state where there's some aggregator who puts all these things in the same place. Where you can start to get 70 or 80% coverage saying, I know this is Chen. And I know it the moment his browser hit my website. Or or maybe I know it after he entered his phone number and his name. And then I ask, and they verify that that's correct. And once again, this doesn't stop the fraudsters, but if you can start to say I have really strong positive signals on 80 or 90% of the population based on, you know, these passive signals. Based on the phone ownership, whatever. Then the fraudsters are, instead of occupying this large space, are compressed into the small space. And it's a lot easier to find them.

Interesting. I think I'm starting to see more and more such players in Europe. I don't know how it looks like in the in the US. I'm guessing it's less complicated in the US. But when you're when you're going after regions which are more fragmented, obviously it becomes a problem. Because when you are in Europe you have a few dozens countries. So yeah, that's definitely interesting, by the way. An interesting business opportunity as well, I would say as a new vendor.

Yeah. Who knew that subscribing to Chen's podcast would be so profitable?

Yeah, yeah, yeah. You owe me money. Tell me, I mean, we've discussed a lot, the data layer. Like what features you want to extract and how to do so, and in what use cases. Do you see something about the basic technology that the system is running? Whether this is rules, models, or AI, or do you see something more on kind of like the policy or the team maker that would need to change to accommodate this new wave of fraud.

I think it will be helpful to have the rate of change, to have org structures accommodate a higher rate of change. Already most fraud institution, most fraud fighters exist within organizations that can act like much faster than credit. Like you want to change a rule in credit, it takes you know years. Use a new variable, and two years later, if you're lucky, you have the new variable and fraud. If you're lucky, maybe it takes you a month. I think the existing org structure should be able to accommodate it. When we talk about, like for example, soups is posting about, you know, this AI agent that can create rules and that can find fraud attacks and stop them in real time. Or almost real time without needing manual review. And so certainly I think that's a cool innovation. Um, there are places that rebuild models on a daily basis. So you're like, we don't need custom rules. The model was just rebuilt last night. Now I think things like that will become more common. Um, there are these articles about foundation models, which just ingests data from so many different customer touch points. And it's not just that it's a foundation model, it's that the philosophy becomes different. So in the past, if you had a fraud model, it would be really unusual for a new account opening fraud model to have, for example, your call center data. And yet it's really believable that we'll end up in a future world where it does. And then you'll say, oh this phone call into my call center was marked as bad and came from this phone number. And you know, so by having just more complete data available. And by having data lengths being a standard, I think that will help us as fraud fighters.

On one hand I must say that I still don't know myself, like what would be the part that foundation models would play in in risk prevention? Is it more like a feature engineering layer? Is it more replacing machine learning? Which I don't really see happening, at least not in use cases. Or maybe, in very specific use cases that, you know, are not that time dependent. Or maybe where you're more lax, right? For example, around login where the friction is not that bad. Maybe there you can use it. Um, but I think the the interesting bit is indeed around extracting features for free, and maybe from a varied data source pool that maybe we haven't looked at so far in fraud prevention. I think that's an interesting, like that is the most interesting bit, and the biggest promise that I see in foundation LLMs. And sorry foundation models. But yeah, I haven't seen it yet.

I think also there's, so my hypothesis is that our old traditional machine learning will continue to be really good. If you want, if it's based on the attacks of the past that are cleanly labeled. And if it's you, and assuming both have access to the same data. So I think a lot of the discussion about foundation models also, like the foundation model is given access to a bunch of data that the traditional machine learning wasn't given access to. So, like naturally, it will do better because it has more data. But on the other hand, I think the the change in the way that it's supervised also matters. So if you talk about like, unsupervised machine learning versus supervised machine learning. Traditionally in fraud, you'd have some sort of tree-based supervised machine learning. And I think there's independent value that can be generated by something that was not treebased machine learning with labels. In particular the labels, right? So you don't know that this is fraud. I think the, that they're, that these new models have an ability to extrapolate more.

Yeah. And then, and then to let's say reinforce the anomaly detection layer. Uh yeah. This also goes hand in hand where I think anomaly detection. Because of, you know, its lesser accuracy. Also with machine learning was traditionally, kind of like human in the loop, like the first human in the loop semi automated process that you had in fraud prevention and yeah, I I'm I'm guessing that foundational models can can fit there quite well.

So I'm going to jump then to the other side. So we talked, okay how do, you how do you avoid detection? And then okay, what could the future look like? So that the frauds can no longer avoid detection. But then there was the other side which was, well even if you know they're fraud. If you're going to challenge them they'll beat your challenge. Uh so now what about the step-up authentication? How does that change? And on one side uh some of that we already got. Because like, okay we know that phone isn't trustworthy. So you just don't send the text message to that phone or to that email. So we already addressed some of that. Um I think more generally, uh the old is becoming new again, that we start to rely more on delivery. So we know that you're not a fraudster because you ship that big screen TV, or at least not a third party fraudster. Uh because you ship that big screen TV to your known and tenured and trusted address, right? And so the way that the consumer interacts with the company. Like I log in using my email and I get emails and I have to click a link or something. Uh so because I have to have access to this email because the email is David, and it's trusted. It's low risk because I got a code to my phone and the phone is David. It's low risk because the TV was sent to my address. It's lower risk because the bank account is my bank account and that's where the funds were sent yada yada. So I think that will remain robust. And often times these, I'm going to call them contact channels. Uh have attached to them some history. And that's helpful. And you have ownership, you have regularity. On the other side, I think there is a big growth in digital IDs and eIDs. And states or countries issuing some credential that's bound with key exchange and encrypted. Uh, which will be, continue to be, like really hard for fraudsters to break. Uh, and so not now, but in our, in our future world. That's not that far off. Uh, and you could imagine that I use my Apple wallet to authenticate myself. And you know, you're just not going to be able to spoof that. And maybe you're, and asking people to go to the DMV and be mules and say, "Go, I show up at the DMV and I say I'm Chen." You know, I don't think that's going to happen at scale.

Yeah. Being able to, to basically match electronic IDs, to devices, to physical devices, that are known to be owned by, uh by you. That were acquired with, yeah, um yeah. Let's see. And I think the main question here is not whether this would become feasible, but whether this would become an acceptable part of a digital experience in the eyes of a customer, right? I think this is, this is the question that remains to be seen.

I mean, I think it will get there. And I say this just because for example, document verification, taking a picture, rolling your face around in a circle, taking pictures of the driver's license, like that's much more frictionful. It's meaningfully, less trustworthy. And basically, there's broad-based acceptance today. And so, we're taking this experience and replacing it with something meaningfully better.

That would be interesting to see. How that would shake things up. Uh because, I think, I mean, I think that we're pretty far from it. But this can also change quite quickly, right?

Yeah. Well, I mean. We, we're pretty far. But on the other hand, I think 7% of people have their driver's license in their Apple, of in the US, have their driver's license in their Apple wallet. So on one side we're far, and on the other side that's like non-trivial adoption. And not many merchants are using it today. But uh some are. And I think we'll continue to see the adoption grow rapidly, or you can go back to like Zelle. Where, you know, it went from nothing to something. Because the customer experience is just so good.

Yeah. And and I can

Or chip readers. You know, you put chips on credit cards. For a long time, Americans didn't have chip readers. And then, you know, they did.

Exactly. I can see how suddenly, you know, Apple comes up with a new schema that is unrelated to fraud or risk. Or even, you know, like finance. Um that would just, you know, like encourage, um encourage, adoption, right? With, uh, with users. That, you know, uh, as a side effect will also completely change how, like what kind of possibilities are open to fraud fighting teams. So yeah, I can definitely see, you know, that we will talk again in five years time. And, you know, that world would be completely different. Or not, we don't know. But this can happen, I don't want to say overnight, but this can happen relatively quickly. We've seen similar things. I mean, I'm just thinking about mobile, right? And how mobile affected, um the, uh, the kind of like purchasing behavior of users. And that changed very quickly. That changed, like within 3 years. It was, you know, there was a very big shift in around 20 I would say 2010 to 2013. Or right around that time. Uh, there was a very massive change.

I mean, actually just speculating. Since we're talking about far future. So, I'm going to give, like two random scenarios, which are more far-fetched. I have less confidence, but I, I'm super curious. Um, so I think typically, if you're a large conservative institution. Uh, you typically don't want to give other companies massive control over you. And you typically don't want to trust other companies. But, and here's my but. But, and as an example, for the longest time, if you were going to log in through Google, no one who did financial transactions would ever allow such a thing. But now, you're starting to see some companies that, you know, are smaller, are, you know, maybe not as high dollar, maybe that are lower risk, allowing you to log in through Google. Where it involves money movement. Albeit not the most extreme cases. And if you're a smaller company or a less mature company, there's a very good chance that Google does a better job fighting fraud than you do. And so, just to pick on, like you can imagine, a state like let's say. I don't know the fraud controls in the different states in the US, but there are varying levels. Often times, it's older. There's regulation and acceptable vendors that gets in the way. But I bet there are many states where if you want to log in, and get some sort of state benefit, the controls are really weak. And if they said you can login with Google, that the people who log in with Google will be way safer. Um, and what's more, I don't know, but I bet you could trigger this login with Google. You could make Google think it was a high-risk transaction. So for example, let's say I wanted to do login with Google, and it was a high-risk transaction. And I said I'm going to make this request. I'm going to try to see if I can tell Google this request is coming from Zimbabwe. And I'm not saying this is the right way to do it. It's clearly the wrong way to do it. But my point is, they have internally some way to step up, and if you have a way of signaling to them that this is a high-risk transaction, can you force them to step up? Or let's say medium risk. So that they step up, but only if it's not a recognized IP address, or device. And you know math 65% customer penetration, great solution, lots of history, continuous investment, huge organizational scale, um should that be the only solution? Is that defensible in front of a regulator? Probably not. If it was one piece of a layered set of layered controls, maybe it is defensible. Maybe it does become a useful positive signal. Or another example. Uh let's say, uh you have a CapitalOne credit card or American Express credit card. And uh you can have a merchant account, and do an off, and say I want to spend money on this credit card. Uh but not actually charge them. You just just place a hold, brief, for a moderate amount of money. Uh, just to trigger their authentication path. And then, okay these companies have a long history on you and you're you're once again doing a $500 transaction, from a risky merchant, uh and your IP address is on the other side of the world, blah blah blah. Like they'll have pretty good risk assessment. Now they probably won't approve of people abusing their controls. It's probably against terms and conditions, because they don't make money if you actually settle I don't think. But I think there are large institutions which will have long history with you as a customer, which have really good sophisticated fraud controls, and it would be really interesting to create some mechanism to allow people to piggyback on those fraud controls.

Yeah. Well, I think this is exactly the the Achilles heel of, you know, of the industry. That on one hand we're saying coordination is our biggest strength, but at the same time,

It's not at all.

Yeah, I would say, but not not for the lack of trying. I think it's just very complex from a regulatory and also technological perspectives, right? So it's, it's just not that easy to do, uh these sorts of uh partnerships and cooperations. Yeah. And it's very times, it's just very very challenging. Just to share data. Just to share data between organization. It's already a big challenge in an offline manner. Um, and without you know, like using it for any kind of decisions. Um, but yeah. I agree um if if the regulator would, uh think of how to do something like that. And for example, you can see how I think it's a bit different. But in the UK and in Australia, where the regulator is, I think a bit more advanced. Or a bit more, I would say, forward thinking. Than for example, in the US. You do see these kind of, uh, initiatives. But at the same time these are also like much smaller markets, in terms of how many players are there/ And, you know, you basically, you capture four players. Then get them into some sort of a consortium, or some sort of network. And you have, you know, 80 95% of the market. And and that's pretty much done. So it's also a bit of a different game

I mean open banking I think is another example. Like not many places use open banking as a step up. And the challenge with using open banking, as a step up is, no I mean first of all you have to have coverage. But also maybe after I link my bank account, it doesn't clear the concern. It's like yeah, I see your bank account, you don't look any better than you did before. But I think there will be many people for whom if you did open banking as a step up, and you see a long history, and they can log into the bank etc. that meaningfully reduces risk. So I think there's a lot of ways to piggy back on other institutions potentially. That have like wide coverage, and decent fraud controls, that we're just not doing today.

Yeah. Yeah. Open banking is definitely a good point. Um, I want to ask you, David, I mean, we've been talking about how, you know, how fraud would look like in the future. We've been talking about how fraud systems should look like in the future. Um, now for our audience that, you know, are concerned about the here and now, but are also maybe a bit concerned about what's coming. What would be your kind of like top three recommendations? I just made up the number three. So use whatever uh whatever end figure that you are comfortable with. What are your top recommendations for teams to be busy with in the next 12 months, so they are better prepared, if and when these uh sophisticated attacks start to scale within their system.

I mean I think it depends a lot on the institution size. So if the institution is relatively smaller, um probably just use a very good orchestrator. That can orchestrate not just uh vendors, and data. But also can do some amount of models. Can also perhaps orchestrate solution providers, uh which involve device intelligence, or behavioral biometrics. You know, just because it gives you a lot of flexibility and ability to move more rapidly. Um, if it's a really large organization. Many times with large organizations, it's very, it takes a long time to do anything. And things are super complex. And having a good foundation becomes really valuable. And all of them know this. So I'm telling things they already know, but I would say have a high quality data link. Where all the data is available, uh, from any kind of control point. From any decision engine. Uh and, uh, would be, have general wise, uh uh, feature creation platforms. Which are available across your customer life cycle. Have ideally, have models which can be trained, uh retrained, every night rather than every two years. Um so, I think for the large organization, it's more about creating an infrastructure which allows, uh rapid iteration. Which makes the tech cost of change lower. Uh, and I'd say the same thing for small institution. Smaller institutions, you do that through, uh third parties. Larger institutions, you do that through in-house build. But oftentimes the cost of change is just so large, and so slow.

I love it. Because I think in the end, I must say, I very much agree with these, with these two approaches. Uh, I mean, basically it's the same recommendation. It's just about how you approach it. Yeah. And to me, you know, it kind of like touches the most basic fundamental principle of fraud fighting. And that is, in my mind, and that is that you are just as good as how quickly you can learn. And I think it loops really nicely back

And that's a great quote. You can excerpt that, excerpt that as a quote.

Uh it, loops back to something that you said at the beginning of the conversation. I'm trying to remember the exact words but, um you said how you know fraudsters can iterate after the first time that they experience failure. Whereas, uh, organizations large or small, it takes them more time. And it's more costly. And the, like the closer you can be to a fraudster's, uh, speed of learning. And I don't know if you can be as close, or even faster. But the closer you can be to a fraudster's, uh, learning speed, the better off you are.

Yeah. It's funny. I like this, uh, an analogy. And it's almost, like a large institution might have 20 people learning. But each person, and I mean the people are learning, but each person represents, let's say a function/ And so you might have, 20 functions that learn. And each function takes a month to do one unit of learning. Whereas, maybe a fraudster might have two or three functions that learn. But they each might learn in the order of a couple days.

Yeah. By the way, I want, I want to say about that, is that also your, uh, learning speed is as good as the weakest chain in the link, right? You mentioned, hey, uh, it it's best if you update your models every night. But what happens if your labels are coming in only every week or every month through, you know, like an acquired chargeback, uh, file. Or what happens if, you know, whether it gets, uh, to you every month. Or every day. Maybe these labels are pretty old because up until you get the chargebacks, you know, time has passed. So there many times I find that when, uh, organizations try to, uh uh, kind of like, um increase their reaction speed. Uh, they tend to obsess and focus on very specific parts of the, of the chain. Uh, many times that, you know, they control directly when there are some kind of, like hurdles. Or kind of like bottlenecks, that they don't necessarily think about how they can solve them as well.

I agree. It's, this is very good insight. I'm going to go on a tangent that you have sort of inspired me, which is, as it relates to labels. I think there are a lot of places where you can get labels that are much more rapid, that people just don't use. So, the normal label is, they talk to someone. The person says they didn't do it, and that maybe takes a week, or a month, or maybe they see a fraud loss. Maybe that takes three months. Um, but you can say what's the rate at which, uh, challenges are passed and that's almost real time. Or you could say after sign up, what percentage of this population does activities which indicate very high trust, like maybe afterwards I did open banking. And it was a really pristine account, and clearly this is me, and maybe that happens 5% of the time, and maybe or 10% or 20%. And then you can say, well, on a larger population, if you see that number meaningfully less, maybe that's an indication that it's higher risk. And you know, is it enough to be deterministic? No. But as a leading indicator or, you know, whatever they send in a bank wire, they pay a utility bill that you can find activities, which become really positive, or meaningfully, uh, riskier, uh, as fast labels. But no one does this. It would be really interesting to, if I, if someone if, someone does this. Please write to me on LinkedIn.

Yeah, I must say in general, I think that, you know. From everything, all the problems that I've ever encountered in fraud prevention, labeling and cleaning labels is probably the hardest problem of all. And I see very few organizations that not, not that tackle that, but that are even aware of that. Uh, so that's that's yeah. I uh, I'm with you on the, on the labels tension. Um, awesome, David. I want to take a moment and kind of like summarize, quickly, the harrowing journey that we went through, uh, in this conversation. Where we were trying to figure out how fraud would look like in the near future, 2 3 years from now. And I think the, like what you outlined, what you portrayed. Is basically that the same sophisticated attacks that we are already seeing today, and maybe over the last few years, um we are going to see uh plainly more of them. Uh why? Because um honestly they are just cheaper to execute, right? With uh with genAI, with agentic capabilities you are able as a fraudster to basically produce 10x the amount of attacks for the same uh for the same investment. And that's something that, uh, is definitely scary. And we also talked about the fact that this is exactly why fraud prevention systems would break. It's not necessarily because they would encounter a challenge that they haven't encountered in the past, but because they would encounter more of the very difficult fraud cases, um, that they are just not able to detect. Um, and that would lead to elevated losses. Maybe not a fundamental, uh system breakage, but definitely more losses. And by the way, we already see today, uh in some industries, and in some uh use cases, that this is already the case. Then we also discussed, um how fraud prevention systems should look like, uh in the future. So they are able to meet this kind of sophisticated fraud attacks, especially at scale. And I think you mentioned a few things. You mentioned, uh first of all on the data side, you mentioned, uh 1) that usage will be more important than ownership. I really like that point where you said it's more important to understand how you use certain assets, um, for example email or phone rather than whether you can establish your ownership on them.

Or let me just say not, I won't say more important, but let's say it will be an added dimension that will be really critical as well. Carry on.

I take it. Thanks for the, for the uh fine tuning. Um you also mentioned consistency. That consistency will remain a very important dimension. Especially when establishing the risk of established, uh accounts. Where the, um the behavior of the legitimate account owner would not be something that fraudsters could easily, or cheaply, uh mimic. Uh, and that goes to everything from behavioral biometrics, to how you like literally the type of actions, uh, that you, uh, that you actually, uh, place on your account. Uh, last, you also talked about the fact that, um starting to look into aggregators,like data aggregators. That can match, uh, different dimensions of your identity. For example, your um, device and your IP. Or your IP and your, uh address.

Digital physical linkage.

Exactly. Yes. Um, this would become, uh even more, I think, they were always important. But I think even more important than before. And look at, and looking at other industries, I think especially the telco data angle, I think is is very interesting. And I think I am already starting to see, over the past couple of years, more and more cooperation between telco data aggregators and fraud uh teams. Um, like exactly around this use case. I think that that was also very interesting. We pondered without, you know, like with a question mark not an exclamation mark, um around both foundation models as well as authentication methods. And we tried to understand how this would, um, shape the future of fraud prevention systems. I think especially foundation models, um to me, it remains to be seen exactly what would be their role to play. Um, we talked about, uh maybe feature engineering. You mentioned, uh them being used for anomaly detection, uh which I really like. So there are definitely places where foundational models could play a part. Which are not necessarily straight out removing or replacing machine learning models. Um, and I think it would be interesting to see how this would shape up. Um, and lastly we talked about what should fraud teams do, already today, so they are better positioned to A) become that uh that uh better future fraud prevention organization. And also meet these uh future fraud attacks or fraud threats. And I really like your point.

You have the same point.

Yeah, I just agree with it. Um, that it's all about increasing your reaction speed. And that how you actually go about it might look differently. Uh, for different organization sizes. You mentioned that smaller teams, that have less resources and have less their capability to invest in house, should probably go with an orchestrator of sort that is able to give them um all the different capabilities in one place. And I think the important thing here is to be able to kind of like control the reaction cycle across all of these point solutions. Uh, but more importantly I think for organizations that have their own uh, their own teams, and their own resources, to really invest more in how quickly they react. And we also talked about the fact that how quickly you react doesn't necessarily mean how fast, uh can you deploy a rule. Or how fast can you retrain a model. But the entire chain of learning. And I think that is so so so crucial.

I, I'd also add one other thing. Do you mind? Um, which which is that we walk through one detailed example for one kind of fraud. But you could do the same exercise that we went through today. And you could do it for account takeover, you could do it for scams, etc.

Yes. And if you're, uh if you're having trouble, uh doing it, uh yourself. I'll put David's uh LinkedIn profile, uh in the podcast uh episode description. And you can, uh and you can message David. That's exactly the kind of engagements that he's, uh been, uh involved with, uh financial services today. David, I want to thank you. And I hope, uh dear listeners, that um other than scaring you, maybe I hope a bit. Uh just a bit. Uh and not a lot. Uh we also gave you, especially David, gave you some practical, uh ways in which you can improve your fraud prevention systems right now. Uh, so you are better positioned to whatever comes next. What comes Next, uh, we have guests, but, uh, well, we'll we'll all find out soon. David, it's been such a joy, such a pleasure having you here. Um, thank you very much.

Yeah, thank you, too. I think this is the most in-depth conversation we've had. And it's been super interesting for me, too. You know, you are also, you're very quotable.

I try to, I try to.

We can have a, you can have a section, you know, quotes by Chen or something like this.

I try to. Uh that would be my next book. Uh once more, it's been a pleasure having you here. I super appreciate you joining us. And for all your listeners, I hope you enjoyed it as well. And I see you all next Saturday. YouTube Description How to beat undetectable AI-powered fraud, with David Liu August 15, 2026 Episode #17 Guest: David Liu, Fraud Strategy Advisor Okay, this episode made me rethink something I've been hearing more and more over the past year: "Our fraud stack is AI-powered, so we're ready for AI fraud." Honestly, that's a comforting story. It just isn't necessarily true. Because the problem isn't simply that fraudsters have AI now. It's that AI-generated fraud is becoming increasingly difficult to distinguish from legitimate customer behavior. The old tells are disappearing, and many of the assumptions we've relied on for years are starting to break down. In this episode, I sit down with fraud strategy advisor David Liu to discuss what undetectable AI-powered fraud actually looks like, why traditional fraud detection systems are struggling to keep pace, and how fraud leaders should rethink their fraud prevention strategy before today's attacks become tomorrow's baseline. This conversation explores everything from deepfake fraud and synthetic identity fraud to autonomous fraud attacks, AI-powered cyber fraud, and the growing role of AI fraud detection in modern fraud operations. More importantly, we discuss how organizations can build fraud prevention systems that continue learning as attackers evolve. What you'll hear in this episode: ● Why undetectable AI-powered fraud represents a fundamental shift in fraud risk management ● How AI-generated fraud is changing the effectiveness of traditional fraud detection models ● Why deepfake fraud and synthetic identity fraud continue to become more convincing ● How AI fraud detection can improve real-time fraud detection without relying solely on static rules ● Why fraud operations automation should focus on accelerating learning rather than replacing analysts You should listen to this episode if you: ● Lead fraud operations or fraud strategy teams ● Are evaluating AI fraud detection solutions ● Want to modernize your fraud prevention system ● Need a stronger fraud prevention strategy for AI-enabled attacks ● Want to understand how autonomous fraud attacks are changing the fraud landscape Episode notes & key takeaways: (Accordion dropdown) Undetectable AI-powered fraud is becoming a scalability problem, not an intelligence problem When people picture the future of fraud, they often imagine fraudsters suddenly becoming much smarter. Honestly, I don't think that's the biggest shift. The uncomfortable part of this conversation is that AI doesn't have to invent entirely new fraud techniques to create a serious problem. It simply has to make existing attacks dramatically easier to execute, cheaper to run, and almost infinitely more scalable. David walks through a hypothetical new account fraud scenario that is surprisingly realistic. An AI agent doesn't need one magical capability. It just orchestrates dozens of existing ones: purchasing phone numbers, mining breached identity data, acquiring aged email accounts, selecting residential proxies, creating believable devices, and assembling them into applications that look remarkably legitimate. Individually, none of these techniques are new. Together, they create undetectable AI-powered fraud that challenges many of today's fraud detection systems. Some of the biggest shifts include: ● AI-generated fraud becomes highly personalized at scale. ● Existing fraud techniques become dramatically easier to automate. ● Fraudsters can launch thousands of attacks where they previously launched dozens. ● Traditional fraud detection signals become much less reliable when every individual signal appears legitimate. The concern isn't that fraudsters suddenly become brilliant. It's that AI gives average fraudsters access to capabilities that previously belonged only to the most sophisticated attackers. Why traditional fraud detection systems begin to struggle One of my favorite parts of this discussion is that David doesn't argue today's fraud prevention systems are broken. He argues they were built for a different world. Historically, fraud detection models have relied on independent signals that were difficult for attackers to fake simultaneously. Device intelligence. Phone ownership. Email tenure. IP reputation. Identity data. The problem is that AI-powered cyber fraud allows attackers to assemble believable versions of all of them. You end up with an application that looks legitimate across almost every traditional fraud detection rule. That's where the phrase undetectable AI-powered fraud starts making sense. Not because it literally cannot be detected. Because the evidence becomes increasingly difficult to separate from legitimate customer behavior. That changes how we think about: ● AI fraud detection ● Fraud detection models ● Fraud decision engines ● Fraud risk management The question shifts from "Is this signal good?" to "How do all of these signals behave together over time?" Behavioral biometrics may become one of the strongest fraud signals One insight I hadn't heard framed this way before was David's argument around behavioral biometrics. Okay...that one made me stop for a second. Fraudsters can buy identities. They can buy devices. They can buy phone numbers. They can buy aged email accounts. What they cannot easily buy is years of genuine behavioral history on your specific website. David's point is simple. Large language models don't know how your legitimate customers naturally interact with your application because that behavioral data only exists inside your own fraud prevention system. That creates an important structural advantage. Behavioral biometrics may continue improving because fraudsters lack the proprietary interaction data needed to convincingly imitate genuine users. Some examples include: ● Mouse movement patterns ● Click positioning ● Navigation behavior ● Session consistency ● Human interaction timing None of these signals are perfect individually. But together they may become increasingly valuable inputs for real-time fraud detection as AI-generated fraud becomes harder to distinguish using static identity signals alone. Context will matter more than identity One of the bigger themes throughout the conversation is that ownership alone becomes a weaker fraud signal. Instead, context becomes the differentiator. David explains that an email address being four years old isn't necessarily reassuring if its behavior changed dramatically three months ago. The same applies to phone numbers. Ownership is useful. Consistency is better. Future fraud analytics will likely place much greater emphasis on behavioral continuity than static identity verification. That means fraud prevention systems will increasingly evaluate questions like: ● Has this email suddenly changed its usage patterns? ● Is this still the customer's primary phone number? ● When was this device last associated with the user? ● Does this communication channel still behave like it historically has? These aren't entirely new data points. What's changing is how much value they'll carry inside modern AI fraud detection and machine learning fraud detection models. Winning against AI-powered fraud requires faster learning, not just better technology The discussion closes on a surprisingly optimistic note. Yes, fraudsters gain structural advantages from AI. They iterate faster. They automate faster. They don't worry about regulators, false positives, customer experience, or compliance. Not exactly a fair fight. But companies have structural advantages too. They have larger engineering teams. Better coordination. More investment. Far richer proprietary datasets. The organizations that succeed won't necessarily be the ones with the newest vendor or the most sophisticated fraud detection rules. They'll be the ones whose fraud operations learn fastest. That means investing in fraud operations automation, improving fraud prevention strategy, strengthening AI fraud prevention capabilities, and designing fraud detection systems that continuously adapt instead of relying on yesterday's assumptions. Because the uncomfortable reality is this: AI isn't just changing how fraud gets committed. It's changing how quickly fraud evolves. And if your fraud prevention system learns slower than the attackers do, the question isn't whether they'll find the gap. It's how long it takes before they do. (Accordion dropdown close) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with David Liu | LinkedIn Fraud Executive, Strategy & Product Co Founder, Dispatchly Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Episode Title: The Rise of Agentic Fraud Ops, Part 1: Your fraud team is running at the wrong speed Meta Title: Agentic Fraud Ops: Fraud Teams Must Learn at Machine Speed Meta Description: Agentic fraud ops is reshaping fraud prevention by enabling AI agents to accelerate learning, reduce losses, and transform fraud operations... URL: https://www.sardine.ai/meia/the-saturday-fraud-strategist/agentic-fraud-ops Keywords Primary Keyword: agentic fraud ops Secondary Keywords AI agents for fraud detection, fraud operations automation, fraud prevention AI, AI-powered fraud detection, fraud detection system, fraud prevention system, fraud detection automation, fraud risk management, fraud decision engine, fraud analytics, machine learning fraud detection, fraud detection models, fraud detection rules, fraud prevention strategy, real-time fraud detection, adaptive fraud detection, fraud orchestration, chargeback prevention, AI-driven fraud prevention, fraud operations transformation

Every fraud leader I've spoken to this year has gotten some version of the same memo from finance. Cut costs. You've already had the argument. You've already explained that fraud isn't a typical cost center. That the cuts being proposed will cost more in losses than they save in headcount. And none of it has changed the answer. That cuts are still happening across the company and they're happening in your team. Whether you like it or not, for all the fraud leaders I talked to, this was bad news. But I want to offer a different perspective. It's only bad news if you don't use this pressure. Here's what I mean. Your fraud team is going to look very different 2 years from now, whether or not you actively shape it. The system you're running today was designed for an adversary that moved at human speed. And that adversary is rapidly being replaced. The team you've built around that system is going to come under strain regardless of what finance does this quarter because the system is what's breaking not the budget. So the cost cutting exercise and the breakage of your fraud stack are two symptoms that signal the same thing. It is time to rethink the future of your fraud prevention system. And if you treat cost cutting as the forcing function for the rebuild, you can actually use it to fund this transformation. But if you treat it as a fight, as a status quo which needs defending, you will lose the fight. Your headcount still gets cut and the system you're left with is still breaking. That's the reframe I want you to walk into your next paying meeting with. The question isn't how do we cut fraud ops costs. The guiding question every fraud leader needs to have in front of their eyes, both 5 years ago and certainly today, is how fast does our system learn? Because everything that matters, including cost, losses, false positives, approval rates, and what your team looks like two years from now, is downstream of that one metric. But here's the thing, the answer to both questions, how do we cut cost and how do we learn faster is also the same one with the help of agentic AI. Look at any fraud stack and you'll find the same architecture. Machine learning models at the bottom scoring the general population, rules in the middle catching what the models miss, and manual review at the top handling the gray cases that are left. I call it the three layer cake, and I've seen versions of it in pretty much any mature fraud system I ever encountered. And for a good reason, well-maintained, it works pretty well. But here's what gets glossed over.

Every layer of that cake starts degrading the moment it goes live. Rules degrade the fastest. The minute you ship one, the adversary starts probing around it and the pattern it was designed to catch mutates within days. Block lists rod the same way and many times even faster. Machine learning models decay slower, but they're far more expensive to refresh and a model refresh has a side effect that most teams underestimate. When a new model ships, the score distribution shifts, and every downstream rule that was calibrated against the old distribution suddenly needs retuning. So potentially hundreds of rules can break or misbehave unpredictably overnight, which is exactly why most teams don't even try to release models more frequently. Interestingly, there are two components of your fraud system that degrade very slowly, if at all. Manual review and AI agents. How come? They are both running on fresh data. Simply put, human investigators are reading what's in front of them right now, and agentic AI is doing the same in trying to mimic them. But everything else in your stack is working from a snapshot taken weeks or months ago. So, what is the main takeaway here? Pretty simple. A fraud system's effectiveness at any given moment is a function of how recently it learned, not how sophisticated it was when it was deployed. But in my experience, most leaders evaluate fraud systems mainly based on their sophistication. Better models, more features, smarter rules, tighter thresholds. They treat effectiveness as if it were determined at the moment of design. But effectiveness is actually determined at the moment of use, when the rule is fired, not when it is tested. And the gap between those two moments is the variable that controls performance. In most cases, against an attack that emerged only last week, a rough rule ship this morning will outperform a sophisticated one shipped two months ago. So the conclusion is straightforward. To avoid decay and ensure high performance, our systems need to learn more frequently. So how do we do that? We start with rethinking how we define performance. If I could measure one thing about a fraud team and one thing only, it would be the time it takes to learn about a system gap and deploy a fix for it. And I use the term system gap on purpose because it doesn't have to be a new fraud attack. It can also be a misbehaving rule or a sudden production bug that degrades data quality. And I call this process finding a gap and fixing it the reaction cycle.

And it has six steps split across two phases. The first phase, learning, covers detecting the threat, scoping it, and finding the root cause of the issue. The acting phase covers designing a fix, testing it, and deploying it. Everything we do at a fraud team is part of that cycle. The technology we run, the processes we follow, the skills on our team, the tools we use. Want to cut chargebacks, you need to react faster. Want to reduce false positives? You need to react faster. Cut down your cost, you need to, you guessed it, react faster. There is no version of improve fraud outcomes that doesn't run through this loop because every other metric you report on precision, recall, false positive rate, loss rate, conversion rate is a downstream manifestation of one number. How fast you close the gap between observing something and acting on it. But ask a fraud leader what the reaction cycle speed is right now and you'll get a long pause then a guess. Almost no team measures it explicitly. They measure everything that depends on it instead. And that's exactly why this is so easy to miss. Now let's consider what's happening on the other side. Fraudsters didn't get smarter over the last 2 years, they got faster. We've now seen multiple cases of a new category of attack behavior. We call it polymorphic fraud. What's polymorphic fraud? Essentially, these are attacks where we observe subsecond pattern adaptation. A defense goes live, starts blocking the fraud campaign, but within milliseconds, we see it mutate its behavior in a way that our fresh defenses aren't able to block as effectively anymore. What powers these attacks are AI agents which were weaponized by fraudsters to redenile signals in real time, test variations, and scale up whatever works. All in an autonomous manner, so there's no need for a human-in-the-loop. I've seen teams that had been forced into blunt mitigations like shutting down cards from entire issuers for weeks at a time or blocking whole geographies because their defenses couldn't adapt fast enough. But even setting aside the most sophisticated polymorphic attacks, we need to internalize that the baseline has shifted. Frosters are using AI agents for the boring parts of their work. Analyzing campaign performance, tweaking scripts, answering customer service emails, as funny as it may sound. And the time those agents save is now being used to outpace you. My point is this. The system every fraud team is currently operating was designed for a world where the gap between attacker adaptation and defender reaction was manageable.

Your reaction cycle was slow, but so was theirs because the adversary was a human working with humanized tools at human speed. But that's no longer the case. And that gap that used to be manageable keeps getting bigger. Now when you hit this kind of pressure, the instinct is to optimize. Shorter meetings, better dashboards, more analysts, cross functional task forces, or looking to edit another vendor to the stack. I've watched teams pour energy into all of those and none of them produce the kind of speed the moment calls for. How come? Because they're optimizing the wrong layer. Your reaction cycle isn't slow because the people running it are slow. It is slow because it was designed around human speed learning. Models refresh on quarterly cycles. Rules get reviewed in weekly cadence meetings at best. And insights from your frauds teams rarely make it back to the detection layer in any structured form. And even when it does, it arrives days if not weeks after the attack has already subsided. Best case, you can squeeze 20% improvement out of these steps. But no amount of incremental optimization fixes the fundamental design assumption. Running at human speed just doesn't cut it anymore. What's the solution for all of these issues? AI. But of course, now I'm very conscious of the fact that AI is beyond hype right now. It's the solution for all mankind's problems. It's what every vendor or service provider sells. It's also the reason why I'm even talking about this topic in the first place. It's what drives the trend of headcount reduction that is surging all around us. I'm not saying that you should use AI to automate tasks done today by humans. This framing is too simplistic. It lacks direction and it's likely setting organization up for failure. What I do want to point out is something much more specific. Your reaction cycle itself needs to run at machine speed instead of human speed. This calls for more than just replacing headcount with AI. It's about redesigning the system itself and how it functions. To run reaction cycles at machine speed, you need a different architecture, one that enables fast learnings. And automation alone would not get you there. So what breaks if you stop at automation? Your team will get faster at the work it was already doing. But what about the things that do not happen today at all?

Things that required so much effort that no team invested in them at all. Nobody on your team is labeling fresh transactions, clustering alerts across thousands of accounts to find the ring they belong to, or continuously retuning segmentation thresholds as user cohorts change their behavior. These aren't tasks to automate. These are capabilities most human teams never had the bandwidth to deliver. Make case review three times faster and you save investigation cost. That the reaction cycle itself doesn't move because what limits the cycle was never how fast investigators worked. It's that labels arrive weeks late. Rules are researched manually and model updates are pushed live long after the attack has moved on. All of these are things your team can and probably should start looking at when you're thinking about how to best utilize agentic AI. Agentic AI can cluster alerts into ring level cases. So, one ruling labels thousands of events. It can generate labels from fresh data continuously, closing the feedback loop in hours, if not minutes. It can propose new rules and surface model refresh candidates with back test results already attached. The reaction cycle speeds up because the loop is closing in places where it used to be ultra slow and you're now reacting at machine speed because you added capabilities your team never had. And so this isn't about automation or at least not only about automation. The teams that would win understand it's mainly about introducing new processes to their system. And this mainly means that in order to truly transform your team and your stack, you'll need to invest. It won't happen with a flip of a switch. But the rebuild and the cost cuts don't have to be separate projects. Cut frauds 30% on its own and you're running the same broken cycle with your investigators. But rebuild the loop first and the same 30% drop in headcount comes out of the other side as a consequence rather than the cause. And here's the thing, many fraud leaders are about to spend the next 18 months either resisting the cost pressure or breaking under it. The third path, using it deliberately as funding for the rebuild your system needs anyway, requires you to walk into your next leadership meeting with a different argument than the one you've been making. And what you get is a real argument, not for sparring your team from businesswide cuts, but a clear road map of what needs to be done in order to facilitate it. So, it's not about being right, it's about being smart. The fraud team you have today is going to break under the next 18 months of pressure either way.

The only question is whether you replace it with something designed by you or assembled by accident. YouTube Description The Rise of Agentic Fraud Ops, Part 1: Your fraud team is running at the wrong speed Date Episode #16 Every fraud leader I've spoken to this year has heard the same message: cut costs. You've probably already had the conversation. You've explained why fraud isn't just another operating expense, why reducing investigators today often means higher fraud losses tomorrow, and why your team is already stretched thin. And yet the budget cuts are still coming. Here's the question I'd rather ask. What if finance isn't actually your biggest problem? In this episode, I introduce the idea of agentic fraud ops and explain why the real issue isn't shrinking budgets. It's that most fraud organizations are still operating at human learning speed while their adversaries have already moved to machine speed. We talk about why fraud detection systems decay over time, why fraud rules and machine learning fraud detection models become less effective the moment they're deployed, and why the future of fraud prevention AI isn't about replacing analysts. It's about building systems that continuously learn. Honestly, optimizing a slow system is still optimizing a slow system. This episode explores what happens when AI-powered fraud detection becomes part of the reaction cycle itself instead of just another automation project. If we're going to redesign fraud operations, we first need to rethink what performance actually means. What you'll hear in this episode: ● Why agentic fraud ops changes how fraud teams should measure success ● Why every fraud detection system begins decaying the day it goes live ● How fraud detection rules, machine learning models, and manual review age differently ● Why reaction speed is becoming the most important fraud metric ● How AI agents for fraud detection can compress learning cycles from weeks to hours ● Why fraud operations automation should focus on new capabilities instead of replacing people ● How fraudsters are already using AI-driven fraud prevention techniques against defenders You should listen to this episode if you: ● Lead a fraud operations or fraud strategy team ● Want to modernize your fraud prevention system ● Are evaluating AI agents for fraud detection ● Need to reduce costs without increasing fraud losses ● Are planning the future of your fraud operations transformation Episode notes & key takeaways (Accordion dropdown) Agentic fraud ops starts with measuring learning speed Most fraud teams optimize outcomes like chargebacks, false positives, approval rates, or fraud losses. Those metrics matter, but they're all downstream of something more fundamental: how quickly your organization learns. That's the central argument behind agentic fraud ops. A fraud prevention system isn't defined by how sophisticated it was when it launched. It's defined by how recently it adapted. If your fraud models refresh quarterly, your fraud detection rules change weekly, and your analysts only feed intelligence back into production after days or weeks, your system is already behind. Every fraud detection system is decaying Fraud stacks usually share the same architecture: machine learning fraud detection models, fraud detection rules, and manual review. The problem isn't the architecture itself. The problem is that every automated component starts degrading the moment it's deployed. Rules become predictable. Models drift. Thresholds become outdated. Fraudsters adapt. Manual review and AI agents are different because they're operating on fresh information rather than historical snapshots. That changes how we should think about adaptive fraud detection. Fraudsters have already reached machine speed One of the biggest shifts discussed in this episode is that fraudsters aren't necessarily becoming smarter. They're becoming faster. AI agents can now test attack variations, interpret denial signals, and evolve campaigns automatically. That means defenders can no longer rely on human-speed reaction cycles. Improving meetings, dashboards, or staffing won't close that gap. Only a fundamentally different operating model can. AI should create new capabilities, not just automate work This episode argues against the common idea that AI exists primarily to replace investigators. Instead, fraud prevention AI should introduce capabilities that most teams never had the capacity to perform: ● Continuously generating labels from fresh transactions ● Clustering alerts into fraud rings ● Recommending new fraud detection rules ● Identifying model refresh opportunities ● Accelerating fraud analytics and decision making That is what agentic fraud ops actually means. It's not faster analysts. It's a faster learning system. Final takeaway: Most fraud leaders are preparing for another round of cost pressure. Some will spend the next year defending headcount. Others will simply absorb the cuts. But there may be a third option. Use that pressure as the catalyst to rebuild your fraud prevention system around machine-speed learning. Because the question isn't whether your fraud organization is going to change. It's whether you design what comes next, or inherit it by accident. (Accordion dropdown close) Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: LLM analytics: When a data exec walks into a fraud team Episode Title: When a Data Exec Walks Into a Fraud Team, with Shachar Meir Meta Title: LLM Analytics for Fraud Teams: What Can Go Wrong Meta Description: LLM analytics can help fraud teams move faster, but only with strong data governance, clean foundations, and better questions before AI adoption... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/llm-analytics Keywords Primary Keyword: LLM analytics Secondary Keywords self-service analytics, AI analytics tools, AI agents for data analysis, fraud analytics, risk analytics, data governance, semantic layer, data quality, data foundations, AI data governance, AI hallucinations in analytics, fraud team AI adoption, fraud operations, risk team data strategy, LLM-powered self-service analytics, AI data analysis, fraud analytics tools

Welcome everybody to another episode of The Saturday Fraud Strategist. And today I have a special guest. Actually, more than a guest, I have a very dear friend joining us, Shachar Meir. Shachar, I think we've known each other for about 15 years or so. And all of this time you've been working on the other side of the fence, specifically in the data space, data management space. So it's a pleasure having you here.

Yeah. It's my pleasure. Thank you so much for having me.

The pleasure is all mine. So Shachar, I'm guessing that most of my audience isn't really familiar with you, as you're coming from outside of the space. So before we talk about why I invited you and the conversation that we'll have today, maybe you can tell our listeners more about yourself and your career so far, and what have you been up to lately.

Yeah. So hi everyone. My name is Shachar. I'm a data executive with over 20 years of experience. I've been leading and managing data teams in smaller companies as well as bigger companies such as PayPal and Meta. At Meta, I was London data engineering site lead. I've grown the site from four data engineers when I started to more than 200 at the peak. I managed the data for a portfolio of products in ads with a run rate of $7 billion a year. And in my very last role I was a director of data engineering for all of Facebook, Instagram, and Messenger's trust and safety problems. And I left Meta about three years ago because I realized that over the past 20 years, I've seen a lot of companies that had a lot of data. They had good data teams, they had decent data platforms, and nothing was working. No one was happy. It was a complete disaster. So I took a step back. I asked myself, why is it that in 2023 we're still failing as an industry, and why do so many companies struggle to get value from the data? And I investigated that, and I realized two things essentially. One is that I have the playbooks and I know how to think to fix it, because that's what I've been doing my entire career. And secondly, that I can achieve a lot more impact if instead of being chief data officer one company at a time, I would work with multiple companies helping them implement my playbook. So that's what I do today. And the interesting thing is I come from a data background, but throughout my entire career I've actually been working with risk teams and teams that deal with adversarial environments in different places, both at PayPal and WorldRemit, which is another fintech. But interestingly also at Meta with trust and safety, which is a different type of adversarial environment, but it's essentially the same kind of playbooks and methodologies and ideas. And then in my consulting career as well. So I've seen it a lot from the other side.

Yeah, I'm sure. You mentioned adversarial environments and it made me laugh because I was thinking, who's the adversary? Is that the risk team or the fraudsters? But we'll get to that in a second. I can only say that I definitely like how you described your career switch. It definitely resonated with me. And I think that one of the things that I truly appreciate in our relationship is that we've met about 15 years ago, and I think we've worked there for probably like two or three years. Then we lost touch for probably nearly a decade, and we had very different careers. You just described your career at Meta. I went and started Frockster as a startup. But we both ended up really in the same place at the same time, basically starting our own consultancy firms. And this is exactly when we got back in touch. I think since then we are probably in touch every week, maybe sometimes, most of the time, daily. And Shachar, by the way, now that I'm thinking about it, he's one of the main sources of memes that I post on LinkedIn. So if you like those memes, know that Shachar is the one to thank for.

Guilty as charged.

Yeah, it's been a journey, man.

Definitely a journey. And I appreciate that.

Yeah. Same here. So Shachar, you mentioned that already, that you yourself, you're not a fraud fighter, but you've been working with fraud fighters for long periods of time in all sorts of different places in your career, both in-house as a manager, but also as an advisor. And actually, that's the main reason why I wanted to have you on the podcast, because I think that data is such a core pillar to the work of every risk manager, and especially fraud managers. And that's also an area where many, many times we have, let's say it nicely, challenges or hurdles to overcome. And I wanted to invite you to the show to basically discuss exactly that, both the relationship, but mainly how can we leverage our peers and how can we leverage our own tools or infrastructure so our lives as fraud fighters are easier. That's the aim. Let's see if we can grab some takeaways out of that.

Yeah, sure. Actually, maybe I can just start by telling about my journey starting from PayPal, because I think that was the inflection point when it comes to fraud fighting. It was the first time I was actually part of a fraud team. And PayPal, back in the day, acquired a startup company that became the PayPal office in Tel Aviv, and I was essentially part of it. It was a group of brilliant people, absolutely brilliant, who couldn't get anything done. And the reason was that they were really struggling with PayPal's data platforms and infrastructure. They were trying to build things, and they managed to build some solutions, but they couldn't get anything into production. It was a complete mess. And I was hired. I don't think that I was hired for a very specific role in that sense, but I sort of identified that as an opportunity. And I thought to myself, okay, I'm going to become the bridge between the data technologies team and the risk team. And I tried to investigate and understand why it is not working, and why these people are not able to deliver anything, and why are the people in data technology, why do they keep complaining about risk? And they had plenty of complaints because risk was so data-intense and data-heavy, and they were killing the clusters and running processes that the data team could not even understand what these people were trying to do. So it was fascinating to see this relationship. And I thought, okay, if I can bring the data team a little bit closer to risk and the risk team a little bit closer to data, and I can understand the gaps on both sides, I can make the situation better. And I think that specifically, unlike other professions like, let's say marketing, sales, finance, where things are, I don't want to use the word straightforward, but you can get away with not having perfect data, and you can get away with not being super data-intense. In risk, it just doesn't work like that. First of all, a lot of the decisions you have to make are in real time and in the moment, sometimes with very limited history, sometimes with a lot of history, but then you're looking for anomalies that are not always very obvious. I think the signal-to-noise ratio is very, very bad in most cases. And sometimes understanding the difference between a stolen account and somebody who's just traveling the world and happens to be in a different country trying to make a transaction that doesn't look like their normal profile or history, there's a very thin line between these two and it's hard to distinguish. So I think that these people were actually some of the most data-savvy and tech-savvy people that I've seen, but at the same time, I think there was a constant misalignment. And I noticed two very interesting things. One of the mistakes the risk team was doing is they kept asking for very pointed solutions. So they were working on a process to detect returning malicious sellers and then returning offensive buyers, and they kept working on specific pointed solutions for specific risk problems that they've identified. Whereas what they should have really done, or what would have been good, and this is what we did when I came in, is instead of asking for those pointed solutions, we started thinking about what is the infrastructure that we need to enable all of that. And if you think about it, the infrastructure for different risk solutions is often the same. You want to calculate some flags in the back end, using the entire customer's history, keep them in a front-end system, and then be able to access that in real time. It's the same infrastructure. So if you build it once and you model it in the right way and you create yourself a pipeline or a channel that you can send data to your front-end system, you're 80% done from a data perspective. So I think that understanding when is the point that you need to switch from thinking about the individual solutions to the infrastructure that enables all of that is something that really accelerated the fraud team there. And the second thing is that I think people often boil the ocean, in the sense that they were trying to build the most prolific solution that they could to a specific fraud problem. Whereas if they started simple, proved that it works, and then iterated on that, it would also give them something to work with and it would also give them the infrastructure to iterate. And I think people spent months and months building something that they could have shipped the simple version of in a couple of weeks, see if it works, and take it from there.

And it probably solved 90% of the problem. I say it all the time. It's hard.

Most likely. And actually, if I can add one last point, I think it was very interesting to see the risk teams, the way they worked when they were building solutions. There were some things that they really thought through all the way and they came up with very interesting heuristics and ideas, and they managed to create pretty good models. But at the same time, sometimes they would end up writing a rule that says, okay, if the model score is greater than 800, then decline the transaction. And every time I would see a static number somewhere, I would always challenge that in a nice way. I would just ask, hey, by the way, how did you come up with this number? And almost always, it was random. So they spent all this time building this model and coming up with something that was, as far as I could say, state of the art. And then the last mile always seemed a little bit like guesswork. Not always, but sometimes. And then I would say, but 800, is it the same 800 for a customer in Europe and a customer in Vietnam? And I would start challenging that and people would be like, yeah, that's a good point. So it would be like, why did you spend so much time building the model, but then so little thinking about how you're going to use it?

Yeah. I must say you touched a nerve for me here, because what you just described is one of the main things that I've been speaking about for ages now, I feel. And it seems that many times we invest so much, as you said, effort and sophistication in the model, and we measure the model in, for example, AUC, area under the curve, right? Like how curved the model performance is. While in the end, the curve and the AUC doesn't matter and doesn't dictate performance and business performance at all. In the end, you need to ask yourself how much losses did I stop versus, or for the cost of, how many false positives did I create? And around this question, what we call the strategy layer, how you basically implement the model and bring it to production so it touches users and influences business performance, in that area, most teams spend very little sophistication and very little effort. And not only that, but also they very rarely look at that decision periodically and validate that it is still true. And yeah, all of that effort that was invested in the model many times just goes to waste because you didn't really look at the last mile.

We should actually change the name from AUC to PAUC, right? Potential area under the curve. Which is, if you use the model correctly, then this is how much you can get. But if you just do whatever, then you'll get whatever.

Yes, yes. Potential, probable, maybe area under the curve. Yeah. I want to first of all say that both of these things that you've seen very much resonate and feel familiar. I want to take a step back and maybe talk a bit more, because you mentioned that risk teams are different than your usual or your other clients, right? The marketing team, sales team, customer support, and so on. I'm just wondering, as a fraud fighter, how should I think about it? I'm not thinking, why should I care, but how is the fact that I'm different, or what about this relationship is different? How should I transform that into a different way in which I align with a data team or communicate with the data team? So how does this difference matter in reality?

I would say there are two ways I would like to think about it. One is that from a technical perspective, I think risk teams have extreme technical complexity, because you can run these batch processes offline in the background for hours and hours to crunch years of historical data, but then in the end you do that because you need the data to be available to make a split-second decision, under 200 milliseconds or 100 milliseconds, and make a decision on a specific transaction or specific account. So I think this is a different level of complexity, I would say, from most other use cases and most other teams. And I think the other thing is the cost of mistake. In marketing, if your marketing is inefficient, then your CPMs might be a little bit too high or the cost of acquisition might be a little bit too high, or you might be spending money acquiring the wrong customers and then churning them, not achieving the lifetime value that you could be achieving. So I'm not saying that's not a bad thing. It is, but it's not the end of the world. But if you think about the consequences of a risk team getting it wrong, then you're essentially funding the next, you know, if you're allowing bad transactions to come through, you might be funding the next 9/11, or you might be helping build terrorist groups in Africa. Or on the other hand, the impact of people traveling the world, trying to buy something not in their country, and then getting a transaction declined when they're in the middle of nowhere and they have no other means. It's a terrible experience for the end users. So either way, the implications and repercussions can be pretty bad. And it has implications on everything. If I use PayPal, for example, and my transaction is declined once, maybe I'll try again. But if it keeps getting declined all the time, guess what? I'm going to stop using PayPal. So you have implications on marketing, and you have implications on your company's finances, and you have implications on everything. And while you're supposed to be an enabler for the company and you're supposed to save the company, doing a poor job is actually hurting the company more than people realize. And in most cases it's very difficult to measure that. It's very difficult to measure the false positives. There are some other examples, right? I know you talk mostly about fraud fighters in fintech, but if you think about some of the problems that we dealt with at Meta, like suicide and self-injury, the cost of getting it wrong is somebody committing suicide. Or bullying and harassment or these kinds of problems, the cost is horrendous. So I think that it's a higher level of complexity than any other problem, and the cost of getting it wrong either way is going to be more difficult. And I think that what does that actually mean for the risk teams working with data? I would say a couple of things. First of all, I think that it means that the collaboration must be a lot tighter. And I also think that there's an opportunity here to connect them to the mission and to what you're trying to do. Help them understand. Don't just throw jobs and solutions over the fence and say, hey, can you just run it in production for me or whatever. Try to bring them on the journey. Try to make them a real partner of yours. They really want to understand what are all these things that you're building, and they really want to become enablers for the business. So bring them on the journey. I think that's the most effective way most risk teams can get the support that they need from the data teams they work with.

That's super interesting. First, it comes to my mind that, you know, you spoke about the complexity. But as you were describing how data teams work with sales and marketing organizations, for example, if you're measuring CAC or CPM, these are very known definitions and there are no half measures here. Either you know your CAC and you're calculating it properly, or you don't. And with risk, first of all, you're dealing with undefined situations. You know, I want to run a batch that catches all the accounts that made these and these transactions in this and this period, that look like this and that. And it might be that I can do something a bit more simple, as you mentioned before, kind of like a simpler solution, a more basic solution that would be 10x simpler to the data team to achieve. And I think, you talked about bringing the data team into the journey, and I think one big aspect here is that the data team, as any type of engineering resource that you usually deal with, they have a lot of demand, very little supply. I haven't seen a data team that is bored yet, or an engineering team, and AI didn't change it. I don't think it will change it soon. So there are a few things here. Can you actually make sure that the cost or the resources that you need are minimized? And that is something that you need to not negotiate, but align with the data team. How can you create the most, or how can you ship the most efficient solution? And the other thing here is also to have them bought into the result, into the mission. So they are on your side and at least they are interested in this project as well. So I completely see it. I wonder, speaking, you know, I mentioned AI. You know what, before AI, you talk a lot about self-serving, and not in the context of fraud and risk teams in particular, actually very generally speaking about data teams. And I think that is exactly where, in this conversation, this topic can come in.

Yeah, so I talk a lot about self-service analytics, right? The ability of users or stakeholders to use their own data, like non-technical users, non-savvy users, getting access to data without the need of learning SQL or Python or other languages like that. And I think that self-service analytics is not a new concept. It's been around for over two decades, where BI teams in the past were building dashboards with lots of selectors where you could slice and dice the data in whichever way you wanted. We've always tried to give business users or stakeholders the ability to get their own data. And it always failed, right? So it's not a new concept, but it's making a massive comeback now with AI and LLMs, because now you got to a point that you can ask an LLM a question in plain English and get an answer. It will get translated to SQL or whatever, and then...

Yeah. But before we go into LLMs, because that's another whole thing that I want to discuss, why originally was it the case that it always failed?

Yeah, it's a good question. I've been pondering on that a lot because in my career I've tried several times to introduce self-service. It didn't work, so I kind of gave up on that idea in a way. Or it worked in very specific cases, but I sort of stepped away from that. And now all of a sudden it's coming back and everybody's asking for it. It made me think, hang on a minute, that's not a new idea. It's always been around. So why did it fail back then, and how do we know it's not going to fail again? I thought about it, and what I realized is if you think about self-service as a concept, for self-service to work, you're making a few assumptions that you assume are true. Number one, that users know what they want. Number two, that they know how to put it into words, and they can explain what they want in words. You assume that words have deterministic meaning, so that revenue always means the same thing and not six different things depending on who in the organization you talk to. You assume that users understand the complexity and nuances and the intricacies of the data, which is very rarely the case. I had a friend who was an analytics manager, I think in Wix or one of the website-building platforms. And he said, Shachar, our analysts, whenever users would ask for reports, in some cases they would just automatically exclude all traffic from China because that created a lot of noise. And as soon as you take it out, it kind of sorts out the report. But they would do it in only some cases, and they knew when they needed to do it. So it was kind of like tribal knowledge. And the users were asking for reports, they didn't explicitly mention anything about China, and they didn't know that this is what they're getting, but this is what the analysts knew. So understanding the nuances and the complexities of the data is not something that most users understand. They might have an idea of what it is that they want, but they cannot put it into words. And the last thing is that you actually assume that users want to be self-served and be held accountable for pulling their own data. And I've seen so many cases where I've built teams dashboards, and I said, hey, here's all your data. You can answer 80% of your questions in this dashboard alone, thinking that it's going to remove a lot of the ad hoc questioning. And actually, it didn't. I was very surprised. People still insisted that they want the analysts to pull the data. But then I realized that if they go and present a number to the CEO and the CEO says, this number is wrong, where did you get that from? If they can throw the analyst under the bus, it makes them look better. But if they are responsible for pulling this number, it makes them look like a bunch of clowns. So obviously they don't want that. And you also assume that people need a high degree of flexibility in the way they consume data, which is not always the case as well. Sometimes you do if you're a fraud analyst and you're now trying to understand a certain modus operandi and trying to investigate something. Fine. But if you're the chief risk officer, you probably want your scorecard or list of metrics that are always there, and it always looks the same so you can manage the business. So I think that if you look at the assumptions behind self-service, many of them just don't work. It is not obvious to me that it's an interface problem and that LLMs and chatbots and free-text, plain-English questioning will make it much different.

Okay, so maybe just to step back. So when you're describing a self-service system in the old days, right, three years ago, we're talking about systems that are essentially dashboards that allow, in this case, fraud teams, with a lot of features, sorry, not features, filters, with a lot of filters and a lot of different dropdowns and so on, to ask data questions without needing to either have SQL skills or have SQL permissions even to go into the data and query themselves. And you're saying that you actually never saw that being a successful solution, not in general and not specifically when it comes to risk teams.

No, so I won't say never. Self-service can work, but I think where people get it wrong is I think unbounded self-service doesn't work. In the sense that people can just ask whatever about any topic, and they just get it. I don't think that's something that's going to work. At the same time, bounded self-service, if you bound it to a very specific domain, to a set of questions, the underlying data sets are highly curated and very specific and purposely created for that mission, I think this is where it can work. And we've seen examples of that. By the way, self-service, when I say self-service in the old days, it doesn't need to be dashboards. It can be purpose-built tools. So at PayPal, for example, you remember we had a review tool that the fraud analysts were using, where they could put in an account ID and they would get all information on the account, and they had a bunch of ways they could explore the data. But it was a very purpose-built tool that gave them specific views, and they had specific workflows around it. But again, you had to invest a lot of time learning how to use it and to use it correctly. They kept enhancing it all the time, where every time you want to support more views and more ways of people looking at fraud, I think that's something that you have to keep building all the time.

Yeah. And that's something that because of the adversarial nature and the fact that the problem keeps shifting, a data point that was interesting yesterday would not necessarily be interesting tomorrow. It's very hard to bound these kinds of use cases for fraud teams.

Exactly. And because of that, the fraud analysts would always know SQL on top of that. But that was their go-to tool and that was always their starting point, if it was good enough and if their data was there. But it was meant to answer very specific questions in specific ways, with specific views and specific information. So I think that somebody did spend a lot of time thinking about what it is that they need and structure the data in a way that would be useful to them. I think that just open-ended self-service for people who are not very technically savvy and not very data-savvy and don't understand these nuances, my view is, it's a little bit delusional. But maybe some people were more successful than I have.

I wouldn't bet my money on that. But I want now to speak about the self-service concept as we are hearing about it today. And today self-service means having an LLM be your basically bionic extension and replace your querying knowledge, capabilities, skill, and querying the data directly and not through a dashboard. I want to share my view and see how you see it. So I'm sharing my view as a data and/or a fraud analyst, right? I think it's even worse. Honestly, it scares me. It scares me a lot because in the old days of self-servicing data tools, at least I know there's a single definition to where the tool gets the data from and how it processes it. There are very clear definitions for each data point that I'm seeing on screen. And this doesn't change. Obviously, it's all hard-coded, so there are no hallucinations. And supposedly now in this new age, the major advantage is that as a user, I can ask any question that I want in plain English, and I will always get an answer. And that's seemingly very powerful. But as a data analyst, I'm like, okay, I'm not sure exactly where you took the data, I'm not sure exactly how you joined the data, and I anyway need to go through the SQL myself and the data scale myself and see and validate that you've done it correctly. And what scares me about it is not that, okay, you can write a query faster and better than me, maybe. That's great. But most users, and that's exactly the point, most users don't have SQL skills. So this is what scares me about it. What's your view?

Yeah. First of all, I'll say that I think that we're observing the biggest revolution of our lifetime right now, technological revolution of our lifetime. What we're seeing is probably unprecedented, and it happens at a scale that is probably too fast for comfort for most people. I think that the ability to have a chat with a bot or whatever in your own language, just explain what it is that you want, and then get a result, is mind-blowing for most people. And if you look at sci-fi movies from five, 10, 20, 30 years ago, that's always how humans wanted to interact with computers, right? And it's finally happening. So in a way, that's amazing. We just describe some sort of an outcome and we get it. But I think when people think about using AI for self-service, there are two camps. There's one that says, I'll just give my AI agent access to my BigQuery or my Databricks or whatever, and then I would be able to ask whatever questions I want. I think that's setting themselves to fail, and big time. And I'll explain why. And the other camp, and if you've seen Anthropic's recent paper on how they do data analytics, and they say they do 95% of their analytics with conversational interfaces and all of that, the basis of that paper is you need to have very, very strong data foundations, good semantic layer, clear definitions, like you need a lot of data governance. So basically what they say is you really need to get the basics right to be able to do that. And that's a company that's presumably already doing it. I think that a lot of the way other people are interpreting the situation is, hey, I have this amazing capability, let's just give it access and see what happens. Now, these models have one very big flaw. They have many flaws, but one of them is they never say, I don't know what the answer is. They just give you the most probabilistic answer, even if this answer is delusional or has no grounds in reality. So I think that we have to be super careful. Now, when can that work? It can work where you define a semantic layer, you define clear metrics, you have very good definitions, you give it a lot of metadata and a lot of context about these are my tables, this is the information that I have, this is what everything means, and all of that. So it can basically do the mapping between your questions to the underlying data. And if you don't have that layer in between, and if you don't have the governance and controls and monitoring and all of that, then you're just leaving yourself to chance, which is not a good way of doing things. Now, the one thing that people also don't realize is that this is all great assuming that you know how to ask the right questions. But actually, if you ask the wrong questions, even if you get the right answers to the wrong questions, that's not helpful. And I think never in the history of mankind was it so easy to ask the wrong questions and get the wrong answers at such a scale. So I think we're getting into a point that every person, and I've seen it a lot managing analysts and data scientists and analytics teams, is often the best service you do for the business or to your stakeholders is not actually running the queries for them, but doing the framing and the ideation and challenging their question. Saying, you're asking for this, why do you need this data? Okay, so this is what you're trying to do. This data is not going to help you. You need something completely different. So that's the kind of detailing conversation. And people can say, yeah, but I can build a skill in Claude and have Claude ask me these questions and whatnot. Possibly. We need to prove that. But people assuming that they will just ask whatever and it will be the right question, and they will get the right answer, and they will know what to do with this information, there are a lot of assumptions going on here.

This is great because I always say that, in my mind, the job of an analyst is not to answer questions. It is actually to ask the right questions. When you ask the right questions, you will get the answers that you need. I want to double-click on this for a second because I think it is truly, truly important. When we, data professionals, I would call myself a junior data professional for a second, talk about this, it is very, very clear to us. It's very clear to us that you need to ask the right questions, and it's very clear to us what is the right question. Well, how do you know whether a question is correct or not? Most of our listeners, especially I would assume the ones that would benefit the most from this self-service, LLM-powered oracle, are not necessarily people who were trained and qualified and educated. And this is not something that you are born with. This is something that you need to learn through many, many years, to know how to ask the right questions and how to identify bad questions. And you managed hundreds of data professionals. You mentored hundreds of data professionals. What would be your advice for someone who is not a skilled data person? How to ask the right question?

It's a good question. I don't know if I have an answer to this, because I think that some people just naturally have that and some people don't. I think that it's something you can learn. I think you can learn that from being in an environment where you get challenged a lot and you get a lot of external feedback on everything that you do, and you're a person that can learn from that feedback and reshape their framing. There are some things that people who ask good questions naturally do, and they keep challenging themselves, right? How do I know this is the right thing? How do I know it's going to work? What am I trying to do here? Is the question I'm asking actually going to help me achieve the goal? I think that a lot of people go into this process being very unclear on what their goals are in the first place. And then they don't actually sense check and say, okay, this is the goal. Is this approach right now going to help me achieve the goal? And we talked about asking the right questions. I think that there are three things here, which is: is the question right? Am I asking the right question? Is the answer correct? Is this answer that I'm getting correct? And then, is the decision correct? Is my interpretation of the answer and what I'm trying to do with it correct? And I think one of the canonical examples for that is the survivorship bias story. I'm sure you know that from the Second World War, where the German Air Force, I think, were losing a lot of fighter airplanes. At some point, they tried to approach it from different angles and engineers tried to fix it, and a lot of people tried to fix it until somebody thought, hang on, maybe it's a statistical problem, right? Some airplanes come back, some don't. There's some statistical angle to it. So they brought a statistician and asked him, hey, can you help us fix it? And he said, yeah, of course. So what used to happen is they would shoot the airplanes and then there would be bullets hitting the airframe and leaving holes in the airframe. So he drew a picture of an airplane and said, just put a dot every place you see a hole, like a bullet. And they did it, and then there were some clusters of areas where there were no bullets at all, and then the rest of the airframe was full of bullets. And their conclusion was, that's great. So now we need to reinforce all these areas where the bullets are and we'll be fine. And he said, no, you need to do the opposite. These are the airplanes that came back despite the fact they had bullets in these places on the airframe. So if you have it in the tip of the wing, that doesn't matter. But if you have it in the root of the wing, then that's a problem for the airplane and it cannot generate lift. So you need to reinforce the areas where you don't see the bullets. I think it's a marvelous example of people asking the right question, getting the right answer, and then reaching the wrong conclusion.

Yeah. I think one of the things that, at least how I operate, and this is how I'm trying to mentor analysts around me, is to actually, before you get to the question, work back your different options. What are the possible decisions that you can reach? Which answer would you need to get that would make you choose each of these decisions, each of these options? And then based on that, ask the question that would actually be able to tell you which of these decisions you need to make. And many times I see that people not only ask the wrong questions, they ask the wrong questions in the sense that all of the answers lead them into the same decision, picking the same decision without even thinking about it in advance. Yeah, it's very difficult.

I love that. And I think an interesting example is that sometimes just reframing your questions and your goals... Let's say on Facebook ads, there are obviously people who use that as a tool, fraudsters who use ads to recruit victims and all of that. So there was an entire business integrity team whose purpose was to find these fraudulent ads and take them down, and fraudulent advertisers and that sort of stuff. It's an amazing channel for fraudsters, by the way. I'm sure you know that. So I think that there are two ways you can think about this problem. One is, okay, they're going to create all these fraudulent ads. How do I find them? How do I identify them at a high accuracy? How do I block them? And that sort of stuff. Which is assuming that they're going to do it. But the other approach could be, how do I prevent them? How do I make it so that they don't want to do it in the first place? And I believe there is actually a very simple solution here, which is not data and has nothing to do with technology. Just changing the policy and saying every advertiser needs to put in $50 in their account before they can start advertising, or $100 or whatever. And then maybe the payment terms are net plus 30 or something. So you basically ruin the economics of being a fraudster on this platform. Make it not worthwhile for most fraudsters. The problem is, yeah, you're also hurting the business, right? Because you're also hurting the economics for a lot of small- and medium-sized businesses in foreign countries, where for them, $100 is, I don't know, 10x their marketing budget or anything they can afford right now. So there's a real tradeoff here. But I think this is a really good way. And if your approach in the first place is, how do I detect that, you're already going down a certain path and you're completely eliminating this path, this branch of how do I make it not worthwhile for them in the first place?

Yeah. Analysts tend to try and solve all of their problems through the data set. And sometimes the solution is outside of the data set. Look, I just want to say that this is not easy. We're both sitting here and trying to come up with maybe some guidance on how to ask the correct, the right questions. And honestly, there is no trick or tip that we can give that would just make you now understand the framework with which you need to approach these AI self-service tools. And I think, in my mind, that's also part of my conclusion, that there isn't a simple way to upskill or train employees, especially in fraud, to work with self-service tools. And for me, that is more of a cautionary red flag than, this is the right way to do that. But we bashed on AI for a bit. That was fun. Still, there are many, many ways, and you said, we're living in exciting times. There are many ways in which fraud and risk teams can leverage LLMs and AI, GenAI in general. And I can share what probably most of our listeners have experienced: that the promise is very enticing. Just run this agent on your data and get everything you've ever imagined or fantasized on. The reality, when you already paid and booked and signed the contract, is that it is not that easy to make it work well, and many times exactly because of data integration issues. And I wonder if you have some words of wisdom and perhaps even tips about how fraud teams should think about preparing for implementing AI and even realizing or testing whether they are ready to adopt AI.

Yeah. I think I'll go back to what I said earlier. Don't boil the ocean and focus on the basics. I think that humans need good data sets, good data foundations, good consistent metrics, high data quality to operate. AI needs that even more. And the idea that we don't have very good data foundations, so we're going to let AI do that, that's going to make things worse. So if you're a fraud team, and by the way, I appreciate that most fraud teams sit in a very tough spot because they're very data-intense, they're very data-savvy, and they're usually not the owners of their data platforms. So they do need to work with the data teams, the data organizations, to get things done. So I would say, first of all, get the basics right. Get one table or one data set that you can trust 100%, that you feel like is highly curated, high quality, very clear definitions. Make sure you have good metadata around it. Don't try to open the floodgates and ask any questions at any point in time, but choose one use case, questions that are highly repeatable, and you want to try and solve that and do a proof of concept and show that this actually helps you, that this new tool is actually helping you operate better. That's one thing that I would say. The second thing is I think that AI is really great for external context. Just to give an example, I was working with one of my clients. They produce electronic appliances, and they showed me their sales dashboard. Basically, from the top of funnel to the bottom of the funnel, for people who did the checkout, the number was about 1% to 1.5%. And I asked the person that was showing me, is that a good number or a bad number? I don't know what it means. I don't know, from the number of people who visit the website, who click on a specific product, who go to the add to cart and then checkout and complete payment, I don't know if 1.5% is good or bad. And AI is actually very good at giving you a sense of that external context. It's not scientific, but it's actually very, very good. Like, what should I consider? I asked Claude, I gave the context, I explained this is the situation, these are the steps, this is what's happening, these are the type of electrical appliances they sell. And it said the typical range is between this number and that number. Now, do I trust it 100%? I'm not sure. Is it fully accurate? No. But it kind of gives me an idea of the lay of the land. So I think that's pretty good. I also think that AI is very good for brainstorming, for ideation. If you just need somebody to challenge what you're doing, to tell you what are the things you're not thinking of, I think these are really good use cases. To basically detail your plans or help you sharpen your thinking process. That's good. But don't let it do the thinking instead of you. Use it as an augmentation tool.

That's super interesting, and I've written all of that down. There's one thing that you said that surprised me. I mean, it didn't surprise me, but I think it might surprise a lot of our listeners. And I think that you glossed over it quite quickly. You said get one view, one table correct. And I think many organizations that I see, in general on an organizational level, think about AI as the duct tape, as the pipelines. We don't need to build a view because we can just give it access to all of these different tables. And usually these are not different tables, these are different databases, different environments, different technologies, and let AI figure it out. But you're saying, if I understand you correctly, no, you want to prepare everything on an infrastructural level. So all the data for one use case is collected in this one table, one data view, and you plug AI on top of that. Did I get it correctly?

Yeah, that's absolutely how I would do that. Now, it doesn't have to be one table. It can be a set of multiple tables. I don't know if you know about medallion architecture, where the idea is that it's an architecture that's become very prevalent in the last few years, where you build your data in three layers. So the first layer is usually called bronze, silver, gold. The bronze layer is basically all your data coming from all your data sources and goes through very minimal adjustments or permutation. So it's basically what we used to call a data lake. And then the silver layer is your model data. So you clean the data, you model it, you validate it, it goes through quality checks and all of that, and you structure the data in a way that is useful for the business. But this data, which is what most companies have as their data warehouse, still has a lot of variance. There are a lot of different definitions, there are a lot of different ways of interpreting this data. So then on top of that, you build the gold layer, which is what you use to power your dashboards or your AI use cases and machine learning use cases. By the way, when you train machine learning models, you didn't just train it on your dim customer table or fact transactions or whatever. You'd always build a smaller data set that is highly curated, and you spend a lot of time defining the metrics and validating them. And sometimes you apply all sorts of different things on them to remove anomalies and reduce noise and all of that. So I think that using your data for LLMs is no different. You want to give it a data set that you trust, that you know what's in it, to reduce the variance and the errors, because there's going to be a lot of errors. So you want to make sure that at least on the surface, you give it a data set where it's very clear who is the customer, what is their history, how many transactions, how much total payment, how many declined transactions, etcetera. So you give it all the information, you give it context and metadata that says this is a customer, this is how many transactions, this is the definition of declined transactions. It's declined because we decided to decline, not because the card issuer or whatever decided to decline. So you give it all this extra information to reduce the variance, and then when you ask questions, the answer is going to be in a smaller range of error, which makes it more trustworthy than in a massive range where you just don't know what you're getting.

Yeah. That is massive. This is so, like when you say it, it sounds so trivial, but I don't think that even I had it in my mind. Of course. If you're not sure that all of your team can ask the right questions, at least make sure that the AI can give you the best answers. And for that you need simplification.

Yeah. And when you're a fraud team or a risk team that doesn't own their data foundations, then agreeing with the data team, building together a data set or set of tables that are highly curated with the right set of metrics, and working on that, is actually a good project. And you don't need to start with what's the ultimate version. Start with what can I produce in one week. Build a table with them. Build all the validations on top of that. Build the context layer. Build everything. So you have the entire infrastructure end-to-end on just one dimension or just one metric. Show that it can work and then gradually start incrementing from there. I think that's a much more sensible approach than coming up with a two-year project that nobody understands why you're even doing it.

That's awesome. So true. I really love it, Shachar. Wow, we went through a lot. So I want to take a moment and gather my thoughts and replay all the takeaways that I've written down. We started a conversation speaking about how fraud teams and risk teams in general are a different client, or a different stakeholder, when you look at it from the perspective of the data teams. And you mentioned two common mistakes that you see in this relationship. The first one is that fraud teams tend to request point solutions, and you say that a better approach would be to actually align on the infrastructure and not necessarily a specific use case. Because when you align on the infrastructure, you have a way better chance to first of all get it right, but also to make other requests much cheaper. The second mistake that you mentioned was boiling the ocean, basically going for a full-fledged, highly sophisticated, highly advanced solution, when you can probably deliver an MVP, basically a pilot, in a fraction of the cost and time, and probably still achieve a significant amount of the value, 80%, 90%, and so on. So that was the first thing that I've captured. We also talked about, in general, that data teams are a coveted resource, like any engineering resource in an organization. And so you want to align not necessarily on the project goals, but at least on the architecture and the proposed solution, because that can really cut down the resources needed and increase the ROI, so increase the chances that you will get your project prioritized. And secondly, also to sell the result, to sell the mission, to get the data team bought into the project so they are on your side as well. Then we talked a lot about self-service. We talked about the fact that in general, this approach fails often, and especially with fraud teams, because of the adversarial, or let's say dynamic, nature of the problem. So there are very few use cases and questions that you can ask that are really repeating themselves in exactly the same way, especially in fraud. And we talked about the fact that this is now getting way worse because of LLMs and the way that a lot of tools nowadays offer self-service data querying in the form of LLMs. And we talked extensively about all of the red flags and the issues that you may encounter here. But we mainly touched upon the point that the core skill that you need here is to ask the right question. And that is not something that you can just understand or learn very, very quickly, despite all of our best efforts. And that this should serve as a warning that a lot of your efforts here can lead to, let's say, not the results that you are expecting. And I would say one thing, you mentioned it later, but I think it's very important here to mention it as well. You need to think very clearly about the impact of the decision that you make. If you're shooting to a range, or if you're brainstorming, as you put it, or if you're doing maybe base research analysis of your population, okay, maybe you can use it and see if it leads you to a ballpark that is within the expected results. But if you're going to deploy a solution that eventually would take action on real users, and you are using LLMs for that without any sort of sanity checks, that can become very, very dangerous. You mentioned all of that as part of speaking about how to get ready to adopt AI, not necessarily in a self-service context, but in general. And other than this point that I just mentioned, you also mentioned two other points. You said start with one simple, clear use case. And for that use case, prepare one table that you can implement your agent or your LLM or whatever you want to run there. And this is so important. I see so many businesses skipping this and basically going, as you mentioned, trying to boil the ocean with their first go, and they get stuck and disappointed with AI. That's basically the cycle that I'm seeing in the last 12 months repeating itself over and over. I hope, dear listeners, that maybe this conversation has helped you gain some sort of glimpse into what goes on on the other side of the fence in the data team, but also how to approach these projects in a way that is more realistic and that has a better chance of success. Shachar, did I miss anything?

No, I think that's good. And maybe I can just add one small thing, which is I have this framework or idea of four versions of every project that I shared with my friend. He's managing the analytics in a company that's building software for psychotherapists in the U.S. We looked at his dashboards and I said, show me your top-line company dashboard, whatever. And he showed me a dashboard where the top-line metric was number of users. And I said, look, this metric is pretty useless. It's interesting to know, but what I care about is the market penetration rate. You're showing a number with no context. I don't know if this number is 1% of all potential users, or 20%, or 70%. I think that's what matters. And he said, you're actually right, but it's actually very difficult to get this number. And I said, well, it's not very difficult to get this number because there are four versions of every project: the one-hour version, one-day version, one-week version, and one-month version. So in one hour, you can go on Claude and ask how many psychotherapists are there in the U.S. You get a number, and you go to your dashboard, you divide your metric by this static number, and you call this estimated market penetration rate, where the word estimated is the key word here. And you've already made progress in the right direction. It's not going to be highly accurate, but it's probably going to be within the right ballpark, and it gives you an idea of whether you're closer to the 1% or to the 70% penetration rate. And then in one day, you can do a little bit more research and maybe there is a database, because I'm sure all these practitioners need to register somewhere. So there must be some database where you have a list of all of them, and you can say, okay, let's download this list once, tidy it up a little bit, get a more accurate number. And what this one day of work allows you to do is remove the word estimate. So now it's not an estimated market penetration rate, you can just call it market penetration rate. So the one-week version, you can probably build an automated process that will download this list periodically, update the number and all of that, so your dashboard is always correct and it doesn't drift. And then the one-month version, and also some of these practitioners are probably registered but inactive and that sort of stuff, so spend a lot more time cleaning it up and getting to a more finely accurate number. Now the one-month version is a little bit more complicated because maybe some of these practitioners work in multiple clinics and they probably have to pay a license for every clinic that they work. So one person can have multiple licenses, and that changes a little bit your market penetration formula and all of that. So in one month, you can probably scrape and bring external data. You can get other data sources like their websites, their LinkedIn, look at big clinics and get the list of all practitioners and look for duplications, and come up with a factor of how different your number is from the real potential one. That's one month worth of work. Now, the interesting thing is that a lot of technical teams, and I would say a lot of risk teams in that sense also, operate in the mindset that when you put a problem in front of them, what they tend to do is to think of the one-month version. They think of all the edge cases and all the complexities of the problem and everything. Whereas what you can do is really ask yourself, how can I create enough value? How can I create progress or create some value in one hour of work? And I think that people usually start with this version, start working on something, realize that it's even more complicated than they'd thought, and they stop in the middle and the project fails. It's not a new concept. You call it MVP. I think it's the same idea, or just agile. But the same project has multiple versions, and start from the small and then build it from there, rather than starting from the biggest one and fail.

Yeah. Or agile. Just agile. Be modest. Just be modest. Go back to basics and be modest. Awesome, Shachar. Thanks a lot for all of these valuable insights. For our listeners that wonder where they can hear more from you, or if they want to work with you, where should they go?

Yeah. So first of all, I'm on LinkedIn. I post on LinkedIn every day. Check me out. If what I say is merely interesting to you, feel free to follow and connect. I have my YouTube channel where I talk mostly about data career growth, but also some of these concepts that can also be applicable for a lot of risk professionals. And most importantly, if you're struggling with your risk team or with your data team, or something about this connection of risk and data doesn't work for you, I'm always happy to explore and see if there are ways I can help. This is what I do today as an advisor. I help companies get value from the data, and that includes risk teams. So of course, any problem you have, feel free to reach out and let's have a quick chat.

Shachar, thank you very much. It was a pleasure.

Pleasure is all mine.

And for the rest of you, I'll see you next Saturday.

Thank you. YouTube Description When a Data Exec Walks Into a Fraud Team, with Shachar Meir August 1, 2026 Episode #15 Guest: Shachar Meir - Data Advisor / Former Director of Data @Meta So this episode starts with a data executive walking into a fraud team. Which sounds like the setup to a very niche joke. And maybe it is. But honestly, it also describes a real problem most fraud teams know too well: we rely on data for almost everything, but the relationship between fraud teams and data teams is often messier than anyone wants to admit. In this episode, I'm joined by Shachar Meir, a data advisor and former director of data at Meta, to talk about LLM analytics, self-service analytics, data infrastructure, and why fraud teams cannot just throw AI on top of messy data and hope it becomes strategy. Because yes, LLM-powered self-service analytics sounds amazing. Ask a question in plain English, get an answer, move faster, avoid waiting three weeks for a data team with 900 priorities. Great. I want that world too. But then you've got to ask yourself: where did the answer come from? Which tables did it join? What definition did it use? Did it understand the business context? Did it hallucinate? Did you ask the right question in the first place? Not a small detail. This conversation is really about the gap between the promise of AI analytics tools and the operational reality of fraud analytics. Fraud and risk teams work in adversarial environments. The problem keeps changing. The signal-to-noise ratio is bad. The cost of getting it wrong can be massive, whether that means letting fraud through or blocking legitimate users. So if you want LLM analytics to actually help, you need more than a shiny interface. You need data foundations, governance, semantic clarity, and the humility to start small. What you'll hear in this episode: ● Why fraud and risk teams are unusually complex data customers ● How fraud teams can work better with data teams instead of requesting endless point solutions ● Why self-service analytics failed so often before LLMs entered the picture ● What changes, and what does not change, when LLM analytics becomes the interface ● Why asking the right data question matters more than getting a fast answer ● How data governance, semantic layers, and data quality shape AI analytics results ● Why fraud teams should start AI adoption with one curated table, one use case, and one measurable outcome ● How to think about the one-hour, one-day, one-week, and one-month versions of a data project You should listen to this episode if you: ● Work in fraud operations and depend on data teams to ship risk or fraud analytics projects ● Are considering LLM analytics or AI agents for data analysis inside your fraud stack ● Have been promised AI data querying that sounds too easy, because it probably is ● Need better ways to align with data engineering, analytics, or risk data infrastructure teams ● Want a practical way to think about AI data governance before deploying tools that affect real users Episode notes & key takeaways: (Accordion dropdown) Fraud teams are not normal data customers One of the main points Shachar makes in this episode is that fraud teams are different from most business stakeholders. And I say that with love. Mostly. Marketing, sales, and finance teams absolutely need good data. But in fraud and risk, the complexity is different. You might run huge batch processes over years of historical data, but the result still needs to support a decision in milliseconds. Is this account legitimate? Is this transaction risky? Is this behavior normal, or is it the start of an attack? That is a very strange operating environment. The signal-to-noise ratio is usually bad. Fraud patterns shift. Legitimate users behave in ways that look weird. Fraudsters behave in ways that look normal. And sometimes the difference between a stolen account and a real user traveling across borders is not obvious at all. This is why the collaboration between fraud teams and data teams has to be tighter than a standard request-response relationship. Fraud teams cannot just throw jobs over the fence and hope the data team runs them in production. Data teams need context. They need to understand the mission. They need to know why a decision matters and what happens when the system gets it wrong. Otherwise, everyone is technically doing their job, and somehow the system still fails. Not ideal. Point solutions create long-term data problems Shachar talks about one of the most common mistakes he has seen in fraud and risk teams: asking for pointed solutions instead of infrastructure. A fraud team sees a problem. Returning malicious sellers. Returning abusive buyers. A specific scam pattern. A new risk flow. So they ask the data team for a solution to that one problem. That feels reasonable in the moment. But if every request becomes a one-off project, the entire system becomes harder to maintain, harder to scale, and harder to reuse. The better question is: what infrastructure would make many of these solutions easier? A lot of fraud analytics problems need similar foundations. You need to calculate backend signals from historical behavior, store them somewhere accessible, and make them available in real time or near real time. If you build that pipeline properly once, you are not starting from zero every time a new fraud pattern appears. That does not mean every problem has the same solution. Obviously not. Fraud would be much easier if that were true. But it does mean that fraud teams should align with data teams around reusable infrastructure, not just individual requests. That is how you make the next project cheaper, faster, and less painful. The last mile of fraud models matters more than teams admit There is a very familiar moment in the conversation where Shachar describes teams building sophisticated models and then using a static threshold almost randomly. Okay, the model score is greater than 800, decline the transaction. Why 800? Often, the answer is not great. This is one of those things that can make a fraud person quietly stare into the distance. A team spends months building a strong model, measuring performance, tuning features, and evaluating statistical outputs. Then the actual business decision layer gets much less attention. But the last mile is where the model touches users. It is where you decide how much fraud you stop, how many false positives you create, and what tradeoffs the business is actually making. AUC, or area under the curve, can be useful. But it does not tell you whether the model is being used well. It does not tell you whether the threshold makes sense for different regions, segments, products, or user types. It does not tell you whether the business outcome is improving. That is why the strategy layer matters. If you do not periodically validate how the model is being used in production, a lot of the work that went into the model can go to waste. Self-service analytics failed before AI for a reason Self-service analytics is not new. It existed long before LLM analytics. Dashboards with filters. BI tools. Purpose-built internal tools. Dropdowns everywhere. The dream was simple: let business users answer their own data questions without needing SQL, Python, or an analyst. And yet, it often failed. Why? Because self-service analytics depends on assumptions that are rarely true. You assume users know what they want. You assume they can explain it clearly. You assume words have deterministic meaning inside the company. You assume "revenue" or "decline" or "active user" means the same thing to every team. Cute idea. Often false. You also assume users understand the complexity of the data. They usually do not. And that is not an insult. Data has tribal knowledge buried everywhere. Sometimes an analyst knows to exclude traffic from a specific country for a certain report because otherwise the results are noisy. The business user may never know that decision happened, but the report only makes sense because someone made it. Self-service also assumes people want to be accountable for pulling their own numbers. Sometimes they do not. If a number is wrong in front of the CEO, some people would very much prefer the analyst to be the person under the bus. Human behavior. Not elegant, but real. So when we say LLM analytics will solve self-service analytics, we should pause. The interface changed. The underlying problems did not disappear. LLM analytics can make bad questions more dangerous The scary part of LLM analytics is not that it can answer questions in plain English. That part is impressive. The scary part is that it may answer the wrong question very confidently. With traditional dashboards, the definitions are usually hard-coded. You may hate the dashboard, but at least it is pulling from a known source and applying known logic. With AI data querying, the system may decide which tables to use, how to join them, how to interpret the request, and how to present the answer. If the user does not understand SQL, data structure, metric definitions, or business context, how do they validate the answer? Exactly. This is where AI hallucinations in analytics become a real operational risk. The answer may look polished. The query may look plausible. The result may even be close enough to feel right. But if the question was poorly framed or the data context was missing, the output can still be wrong. And in fraud operations, wrong answers are not just academic. They can lead to bad rules, bad reviews, bad model decisions, bad policy changes, and real impact on users or losses. This is why Shachar's point is so important: the job of an analyst is not just to answer questions. It is to help ask the right questions. Honestly, that is the part we should be automating last, not first. Data governance is not optional for AI analytics tools One of the most useful parts of this conversation is the distinction between two approaches to AI analytics tools. One approach is: give the AI agent access to the warehouse and let people ask anything. That is the version that makes everyone anxious, or at least it should. The other approach is: build strong data foundations first. Define the metrics. Create a semantic layer. Add metadata. Clarify what each field means. Curate the tables. Monitor quality. Then use LLM analytics on top of a controlled data environment. That second approach is less exciting in a demo. It is also much more likely to work. AI analytics tools need the basics even more than humans do. A human analyst can sometimes notice when data looks strange, remember tribal context, or ask a follow-up question. An LLM may just produce the most probable answer. And if the underlying context is messy, that answer can be messy too. So if a fraud team wants to adopt LLM-powered self-service analytics, the first project should not be "connect the entire warehouse." It should be one trusted use case, one highly curated dataset or set of tables, clear definitions, good metadata, and a way to validate whether the tool is actually helping. Very boring. Very necessary. Start with one use case, not the entire data universe Shachar's advice for fraud team AI adoption is refreshingly practical: do not boil the ocean. Start with one use case. One repeatable question. One trusted dataset. One table or one small set of curated tables with definitions the team actually understands. Build the validation layer. Build the context layer. Add metadata. Define what "declined transaction" means. Is it declined by your fraud system? Declined by the card issuer? Declined because of insufficient funds? These distinctions matter. Then test whether the LLM analytics tool helps. Does it answer the right question? Does it use the right data? Does it reduce time to insight? Does it support a safer decision? Does it make the fraud team better, or just faster at being wrong? That last question matters. If the first use case works, expand gradually. Add dimensions. Add metrics. Add more questions. But do not start with the fantasy that AI can duct tape together every messy database, environment, table, and definition in the company. That is not a strategy. That is a very expensive shrug. AI can help with context, brainstorming, and sharper thinking This episode is not anti-AI. That would be lazy. Also, wrong. Shachar gives a useful distinction: AI can be very good at providing external context, brainstorming, ideation, and challenging your thinking. If you need a sense of the typical range for a business metric, AI may help you get directional context. If you want to think through what you might be missing, AI can be useful. If you want to pressure test a plan, it can help. The key is to use it as augmentation, not delegation. That means AI can help you sharpen your thinking, but it should not replace your thinking. It can suggest questions, but you still need to know which ones matter. It can help frame options, but you still need to understand the decision. It can help draft a query, but you still need to validate the logic if the result affects real users. This is especially true in fraud and risk, where decisions can trigger declines, reviews, friction, user harm, financial loss, or compliance exposure. So yes, use AI. But maybe do not hand it the keys to production and walk away. Seems fair. The one-hour version is often better than the one-month version Toward the end of the conversation, Shachar shares a simple but useful framework: every project has a one-hour version, a one-day version, a one-week version, and a one-month version. This is worth stealing. Respectfully. A lot of technical teams, including fraud and risk teams, see a problem and immediately imagine the one-month version. Every edge case. Every integration. Every perfect data source. Every complexity. And then the project becomes so large that it stalls, gets deprioritized, or dies quietly in a roadmap document. Instead, ask what the one-hour version looks like. Can you create directional value quickly? Can you estimate the metric? Can you make progress without pretending it is final? Then improve it. The one-day version is cleaner. The one-week version is automated. The one-month version is more complete. This is not new. It is MVP thinking. It is agile. But apparently we all need to relearn simple things every few years under a new name. Fine. For fraud teams working with data teams, this mindset is especially useful because data resources are limited. If you can show value quickly, validate the direction, and then build from there, you are much more likely to get support than if you propose a two-year project nobody fully understands. Final takeaway: LLM analytics can absolutely help fraud teams. But only if the foundations are not a mess. If your data definitions are unclear, your semantic layer is missing, your metrics mean different things to different teams, and your users do not know how to ask the right questions, then AI will not magically fix that. It may just make the confusion faster. That is the slightly annoying but useful takeaway from this episode: fraud teams should not wait for perfect data, but they also should not pretend AI removes the need for data governance, data quality, and smart collaboration with data teams. Start small. Build one trusted use case. Align on the infrastructure. Ask better questions. Use AI for context and thinking support, not as an accountability escape hatch. Anyway, if your LLM analytics strategy is "connect everything and see what happens," I guess technically that is a strategy. Just not one I would want to explain after it breaks. (Accordion dropdown close) Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Shachar Meir | LinkedIn Data Advisor Former Director of Data @Meta Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: False Positives Masterclass Part 4: Building a safety net over your fraud stack with fraud override logic Episode Title: False Positives Masterclass Part 4: Building a safety net over your fraud stack Meta Title: Fraud Override Logic That Reduces False Positives Meta Description: Fraud override logic helps reduce false positives with known good users, high-exposure networks, geo-chaining, low-risk items, and shadow testing... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/fraud-override-logic Keywords Primary Keyword: fraud override logic Secondary Keywords false positives, reducing false positives, fraud prevention logic, fraud rules, fraud detection rules, fraud override systems, fraud system override, device intelligence, manual review, shadow mode testing, fraud model precision, fraud model recall, false positives in fraud detection, payment fraud, fraud operations

In the previous part of this masterclass, we talked about how to tune your solutions so they create less false positives. This sounds great in theory, and it's often the case that it would work great in reality as well. But even if you do all of that perfectly, you still face one more problem. Your fraud system is a maze. You have rules, models, AI agents, manual review queues, partner responses, payment routing decisions, KYC checks, device intelligence, and dozens of other components That are each capable of blocking a user. Even if each component is reasonable on its own, their interactions can still produce false positives in places you never intended. A user might bypass one part of the system only to be caught by another. This is why mature fraud teams eventually introduce a different mechanism altogether, a high-level override layer that sits on top of the entire stack to help the system speed through users you already have reason to trust. Think of it as the mirror image of your fraud safety net rules. When any actor in the system, be it a rule, a model, an agent, or even a human, tries to block a user, this safety net checks whether there is strong evidence that the user is actually a good one. And if so, it reverses the decision. This idea is extremely powerful when implemented correctly, but it requires thoughtful design. So let's walk through how it works.

There are two questions you must answer before designing any override logic. The first one touches on the parts of your system that shouldn't be overridden, and definitely not by default. Why? Well, frankly speaking, not all solutions should be treated equally. If a stolen card blacklist says a card is compromised, you shouldn't automatically override the decision just because the device location looks good. After all, the data intelligence vendor can be wrong. That doesn't mean it has to be complex, and there is no need to pair fraud logics with false positive logics. Just mark the solutions in your system that are particularly strong and should never be overridden. The second thing you want to consider, obviously, is which heuristics are strong enough in detecting false positives. Keep in mind, you are searching for good users in a population that is inherently high risk. You already chose to block them. So even if this can be more accurate, the fraud rate in this segment is much higher than your baseline. So your false positive heuristics need to be especially airtight, much more than a normal fraud rule. The question then is, where do you start? I'd like to suggest four key heuristics, or logic families, that usually prove to be effective at sifting through high fraud rate populations and finding false positives in them. Which of them you implement, and how exactly, depends on your specific business context.

Let's start with the simplest and most intuitive category: users you already know and trust. These signals might include long-standing accounts with clean behavioral history, or users who passed high friction verification. And I'm not talking about your generic KYC, but something more meaningful. Now, I know what you must be thinking. Hold up, this is the most trivial piece of information ever. Surely I didn't waste my time watching this for this. But here's the catch: while flagging established accounts is straightforward, the logic changes entirely when you need to assess new accounts or one-time users. In many cases, these new users can be returning users. And if we manage to associate them with their past activity, we might exclude them from high friction risk mechanisms. For example, I myself have three PayPal accounts, as I've lived in three different countries, and PayPal requires users to open an account in each new country they move to. This is not the ideal user journey, but I understand the compliance requirements they need to uphold. At the same time, I also know that whenever I open a new account, They immediately link it to my previous ones. How?

Simply because I'm using the same device to open a new account with the same name as an account I already own. That's me. The key insight is that trust can be inferred, not just collected. You don't have to wait for someone to build a long history on one account if they are the same user behind multiple accounts, and you verify the legitimacy of any one of them, the new one inherits that trust. Now, using the same device and name to open a new account is not exactly a groundbreaking heuristic, even though not all teams have even that in place. But you can easily find similar logics that don't rely on the same device and are still as strong. Here are some examples: card plus IP address, email plus the same, item bought. And you can also infer across family members, for example, with last name plus exact geolocation. Just remember, it's not about finding random links. It's about linking a new event to an airtight, proven good event.

Fraudsters avoid environments where their real identity or real affiliations could be exposed. Legitimate users, on the other hand, often operate from those environments naturally. This leads to a surprisingly powerful heuristic. If a user transacts from a highly controlled or highly traceable network environment, the likelihood they are a fraudster drops dramatically. What could be examples of that? One example can be corporate, government, or military IP networks with strict access controls, where the user is likely an employee. University IP networks, where the user is either on staff or a student, are also a good example. And some large nonprofit organizations, think about like the UN, Where the user is likely an employee as well, can also be considered. Committing fraud from such a network is highly unlikely, as they are monitored for security breaches and can lead back to the individual user. Similarly, sending packages to a highly controlled shipping address, let's say a military base, for example, is another indicator the user is highly unlikely to be a fraudster. Now, keep in mind, it's not necessarily about identifying and prosecuting the fraudster. Most of them don't fear that. But it is about the fear of being sanctioned by their own employer that would deter them from utilizing these assets to commit fraud.

When we think about geographic data, we often think about rudimentary fraud detection logics that look for geo mismatches. And as rudimentary logics, we also know they are many times the main culprit when it comes to causing false positives. We just talked about it in the previous part, where I gave the US-Canada example. But we can also use geographic data to identify false positives quite accurately using a technique called geo chaining. Specifically, I'd like to point out two types of geo chains. The first one deals with geographic proximity in specific locations. Here's what I mean. If an IP address resolves to a location extremely close to the legitimate billing address, that on its own is not necessarily a redeeming indicator. For example, if the address is located in New York City, finding an open proxy IP that is less than 10 miles away from the billing address you stole is probably not so hard. But what if the address is in a small town or a rural area? When you get a strong match to the IP location, let's say less than 10 miles, this can be a stronger good user indicator than most people realize. Fraudsters can spoof large cities easily, but it is not that easy to do that with obscure villages in the countryside. The second thing you want to look for is nationality-consistent mismatch. And again, let me share an example from my own personal experience. When I use my Spanish card in Israel, I often get blocked. Why? Because of the Spanish BIN, Israeli IP mismatch. But it shouldn't be the case. Analyzing my name should mark it as Israeli. The technology exists, trust me. And matching it to the Israeli IP should be easy. That's not to say that when a name shares the same nationality as the IP address, it means it's a good event. But when we see a very specific pattern where the card mismatches the IP, but the name matches the IP, we find a correlation that is too obscure for a fraudster to mimic. Think about it. When a fraudster sees a Spanish card, they simply go for a Spanish IP. There's no reason for them to overengineer it. This is subtle and contextual, but when the logic is vetted properly, it can overturn a surprising number of geo-based false positives while introducing minimal additional fraud risk. As a side note, by using this heuristic, we basically identify travelers. And you can probably think of other ways to identify travelers. For example, identifying hotel or airport Wi-Fi IPs. Are travelers by definition good users? Not always, but it helps explain that rudimentary geo mismatch logic we mentioned earlier.

The final heuristic applies mainly to payment fraud. Usually fraudsters don't steal goods so they can use them themselves. They steal it so they can later resell it and pocket their earnings. This means that their economic model depends on transferring the stolen goods, whether digital or physical, to someone else. If the item being purchased has no resale market, or only a very weak one, then declining it for fraud risk is often counterproductive. Here are some examples. Educational goods and services, niche and hyper-personalized items, like think about customized family gifts, or services that require personal consumption, like a therapy session or a personal training session. If you think about it, we're basically reversing the heuristic that makes high-value products riskier. Why do we perceive electronics, auto parts, or gift cards as high risk? Because it's very easy to resell them. So in a well-designed override system, low resale value can help your system approve things fraudsters don't even want in the first place.

Let's make it simple. You do not turn on override logic for 100% of your traffic on day one. You test it robustly. A typical rollout might look like this. First, deploy the override rule in shadow mode only. Second, you measure the hits and compare them to your analysis. Is it hitting according to your expectations? Third, you lower the shadow mode rule to 80% of the population, and introduce a challenger rule that actually overrides declines for the rest of the 20%. These numbers are, of course, placeholders, so you can decide what's the level of risk you want to take here. Fourth, you watch the results over, say, 30 to 60 days, and this should show you how fast fraud matures in the undeclined population. If it looks good, you can ramp up to, let's say, 50%, and repeat the process. If it's not really clear yet, you want to wait another 30 days. And if fraud spikes, you go back to 100% shadow mode and to the drawing board in general. You go through several iterations, again, based on your risk appetite, and gradually increase it until you reach full deployment. One thing that you might find out throughout this cautious ramp up is that the fraud you do see penetrating these rules can come from a handful of fraud detection solutions. This is pretty common, and in that case, you should move these fraud solutions to that tier we mentioned that doesn't get overridden. And if you've gotten to this point, essentially developing exclusion logics to exclusion logics, you know you're leading the pack. YouTube Description False Positives Masterclass Part 4: Building a safety net over your fraud stack July 25 2026 Episode #14 Guest: Here is the uncomfortable thing about fraud systems: even when every individual part looks reasonable, the whole thing can still behave like a maze. You fix one rule. Great. You tune a model. Nice. You clean up a manual review flow. Very responsible. And then a good user still gets blocked somewhere else because another rule, partner response, payment routing decision, KYC check, AI agent, device intelligence signal, or some forgotten logic from three quarters ago decided to step in and say, absolutely not. In this episode of the False Positives Masterclass, I'm talking about fraud override logic, which is one of the more powerful tools mature fraud teams can use when reducing false positives across a complex fraud stack. The idea is simple in theory: build a high-level safety net over the system that can recognize users you already have strong reason to trust, even if one actor in the stack tries to block them. But simple in theory is where many bad fraud ideas are born. So we need to be careful. A fraud system override is not a shortcut. It is not a "good vibes" approval layer. It is not an excuse to ignore bad logic underneath. It is a controlled, evidence-based mechanism that asks: before we block this user, do we have airtight evidence that they are actually legitimate? That sounds obvious. It is not. Otherwise, more teams would do it well. What you'll hear in this episode: ● Why even well-tuned fraud prevention logic can still create false positives ● How fraud override logic works as a safety net over rules, models, AI agents, manual review, KYC checks, and partner responses ● Why some fraud detection rules should never be overridden automatically ● How known good users and inherited trust signals can help reduce false positives ● Why high-exposure environments can be useful false positive indicators ● How geo-chaining can help distinguish travelers and legitimate mismatches from fraud ● Why non-resellable or low-risk items can support safer payment fraud approvals ● How to deploy fraud override systems safely using shadow mode testing and gradual rollout You should listen to this episode if you: ● Work in fraud operations and your stack has too many independent blocking points ● Are trying to reduce false positives without weakening fraud detection rules ● Need a safer way to identify trusted users across accounts, devices, cards, or flows ● Want practical examples of fraud override logic beyond generic allowlists ● Are evaluating when to use device intelligence, geo-chaining, manual review, or challenger rules to improve decisioning Episode notes & key takeaways: (Accordion dropdown) Fraud override logic is a safety net over the whole stack The main idea in this episode is that reducing false positives does not end when you tune individual fraud rules. That helps, of course. But even if every rule, model, AI agent, partner response, manual review queue, payment routing decision, KYC check, and device intelligence signal makes sense on its own, the full system can still create bad outcomes. That is because users do not experience your controls one at a time. They experience the whole maze. A user might pass one rule, then get blocked by a model. They might pass the model, then get held by a partner response. They might pass KYC, then run into a payment routing decision. Somewhere along the way, one part of the system says no, even though the broader evidence says this is probably a good user. That is where fraud override logic comes in. It sits above the stack and asks whether there is strong enough evidence to reverse a decline or block. Think of it as the mirror image of fraud safety net rules. Instead of catching fraud that slipped through, it catches good users who were incorrectly stopped. Okay, this sounds dangerous. It can be. That is why the design matters. Not every fraud decision should be overridden The first design question is simple: what should never be overridden? This is where teams need discipline. Not every solution in the fraud stack deserves the same treatment. Some signals are weak. Some are directional. Some are useful only in context. Others are strong enough that overriding them casually would be a bad idea. For example, if a stolen-card blacklist says a card is compromised, you probably do not want to override that decision just because the device location looks nice. The device intelligence vendor might be wrong. The network might look clean. The behavioral pattern might even look normal. But if the payment instrument itself is known to be compromised, that is a very different category of signal. You do not need to overcomplicate this. You do not need to pair every fraud rule with a matching false positive rule. Just mark the solutions or logic families that are too strong to override by default. That one step prevents a lot of future pain. Because if your override layer can reverse anything, eventually it will reverse something it should not have touched. And then everyone gets to have a fun meeting. By fun, I mean not fun. Good-user heuristics need to be stronger than normal fraud rules The second design question is which heuristics are strong enough to identify false positives inside a blocked population. This part matters because the population you are searching inside is already high risk. These are not random users from your baseline traffic. These are users your system already decided to block. So even if an override rule can be more accurate than the original decline, the fraud rate in this segment is still higher than normal. That means your good-user signals need to be especially strong. More airtight than a normal fraud rule. You are not asking, "Does this user look kind of okay?" That is not enough. You are asking, "Do we have strong evidence this user is legitimate, despite the fact that something in the fraud stack tried to stop them?" That is a higher bar. It should be. The episode breaks this into four useful logic families: known good users, high-exposure environments, geographic chains, and non-resellable or low-risk items. None of them should be copied blindly. But each one gives fraud teams a practical place to start. Known good users can carry trust across new accounts and events The simplest override family is known good users. Long-standing accounts with clean behavioral history. Users who passed high-friction verification. People you have strong reason to trust. Now, yes, I know. This sounds very basic. "Good users are good users." Fantastic insight. Maybe put it on a slide. But the interesting part is not old trusted accounts. The interesting part is what happens when those users show up in new accounts, new countries, new flows, or one-time transactions. Sometimes a "new" user is not actually new. They are a returning user your system has not connected yet. If you can link the new event to a prior proven-good event, you may be able to safely reduce unnecessary friction. The example in the episode is a personal one: having multiple PayPal accounts because of living in different countries. The user journey is not ideal, but the compliance requirement makes sense. The important part is that the platform can link the new account to the previous ones using signals like device, name, and past account history. That is the real lesson. Trust can be inferred, not only collected. A new account using the same device and name as a previously verified legitimate account is one example. Other combinations might include card plus IP address, email plus same item bought, or even family-linked signals like last name plus exact geolocation. The point is not to find random connections. Random connections are how you accidentally build bad logic with confidence. The point is to link a new event to an airtight, proven-good event. High-exposure environments can make fraud less likely The second family is high-exposure environments. Fraudsters usually avoid environments where their real identity, employer, institution, or affiliation could be exposed. Legitimate users, on the other hand, often transact from these places naturally. That creates a useful false positive signal. If a user is transacting from a highly controlled or traceable network environment, the likelihood that they are committing fraud may drop. Examples include corporate networks, government networks, military IP ranges, university IP networks, or large nonprofit organizations. This is not because every person on a corporate network is automatically legitimate. Obviously not. Let's not get carried away. But committing fraud from a monitored environment tied to your employer, school, or institution is risky in a different way. Many fraudsters are not especially afraid of prosecution in the abstract. But being sanctioned by their employer, university, or organization? That can be a more immediate deterrent. The same logic can apply to controlled shipping destinations. A package going to a military base, for example, may carry a different risk profile than a package going to a reshipper-friendly residential address. Again, this is contextual. It is not magic. But when vetted properly, high-exposure environments can help fraud override systems identify legitimate users inside a blocked population. Geo-chaining can reduce geo-based false positives Geographic mismatch rules are a classic source of false positives in fraud detection. Everyone knows the logic: card country does not match IP country, billing country does not match shipping country, account country does not match login country. Sometimes this catches fraud. Sometimes it blocks normal human behavior because humans insist on traveling, commuting, using VPNs, and generally refusing to behave like clean database rows. Very inconsiderate. Geo-chaining is a way to use geographic data more intelligently. Instead of treating all mismatches as suspicious, you look for contextual geographic patterns that explain why the mismatch may be legitimate. The first type is geographic proximity in specific locations. If an IP address resolves very close to a billing address in New York City, that may not tell you much. Fraudsters can find open proxies near major cities. But if the billing address is in a small town or rural area, and the IP resolves within a tight radius, that can be a stronger good-user indicator. It is harder to fake obscure geographic proximity than generic big-city proximity. The second type is nationality-consistent mismatch. The episode uses another personal example: using a Spanish card in Israel and getting blocked because of a Spanish BIN and Israeli IP mismatch. But if the name appears Israeli and the IP is Israeli, that context matters. A fraudster with a Spanish card would probably just use a Spanish IP. There is no obvious reason to over-engineer a Spanish-card, Israeli-name, Israeli-IP pattern. That is the value of geo-chaining. It finds subtle, contextual correlations that can explain legitimate behavior. Used carefully, it can overturn geo-based false positives without introducing unnecessary fraud risk. Non-resellable items can support safer payment fraud approvals The fourth family applies mainly to payment fraud: non-resellable or low-risk items. Fraudsters usually do not steal goods because they personally want the item. They steal goods because they can resell them. That is the economic model. If the item has no resale market, or only a weak one, then declining it for fraud risk can sometimes be counterproductive. Examples include educational goods and services, niche or hyper-personalized items, customized family gifts, therapy sessions, personal training, or other services that require personal consumption. This is basically the reverse of why we consider electronics, auto parts, or gift cards risky. Those items are easy to resell. A customized family mug with someone's dog on it? Less exciting for an organized fraud ring. I mean, probably. In a well-designed fraud override system, low resale value can become part of the approval logic. Not as a standalone reason to approve everything, but as a supporting signal that helps the system recognize transactions fraudsters are unlikely to want. That distinction matters. Low-risk item logic should support the override, not carry the whole thing by itself. Safe deployment matters as much as the logic The final section of the episode is about deployment, because this is where good ideas can become bad incidents. You do not turn on fraud override logic for 100% of traffic on day one. You test it. Carefully. A safer rollout starts with shadow mode testing. Run the override logic without letting it affect decisions. Measure whether it hits the population you expected. Look at the cases. Compare the results to your analysis. Then you can introduce a challenger rule to override a small portion of declines while the rest remains in shadow mode. The transcript uses 20% as an example, but the exact number depends on your risk appetite and business context. Then you watch the results over 30 to 60 days, or long enough to understand how fraud matures in the population you allowed through. If results look good, ramp up gradually. If they are unclear, wait. If fraud spikes, go back to 100% shadow mode and rethink the logic. This cautious ramp-up can also reveal which fraud detection solutions should never be overridden. If most of the fraud that penetrates the override comes from a handful of rules or models, those solutions may belong in the "do not override" tier. And if you get to the point where you are building exclusions to your exclusions, congratulations. You are either leading the pack or very tired. Possibly both. Final takeaway: Fraud override logic is powerful because it recognizes something important: false positives are not always caused by one bad rule. Sometimes they are caused by a system where too many reasonable components interact in unreasonable ways. A high-level override layer can help mature fraud teams approve users they already have strong reason to trust. But only if the logic is precise, the signals are airtight, and the rollout is controlled. Known good users, high-exposure environments, geo-chaining, and low-risk items can all help reduce false positives. Shadow mode testing and gradual deployment help make sure you do not solve a false positive problem by creating a fraud problem. Anyway, that is the balance. Build the safety net, but do not pretend gravity stopped existing. Am I being too cautious? Maybe. But in fraud systems, cautious usually ages better than clever. (Accordion dropdown close) Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. This is part of a larger series, so if you landed here first, you may want to go back and listen to the previous episodes. We've already covered quite a lot that will make this one much easier to follow, so get caught up here: False Positives Masterclass: How to measure FPs in systems that hide them False Positives Masterclass Pt. 2, Turning data into intelligence False Positives Masterclass Part 3: How to reduce FPs inside your system Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: Picking Fraud Vendors Without Falling for the Wrong Promises, with Holly Sandberg Episode Title: How to Pick the Right Fraud Vendor For You, with Holly Sandberg Meta Title: Picking Fraud Vendors: Vendor Fit & Risk Checklist Meta Description: There are many traps to avoid when picking fraud vendors. From black-box tools to weak SLAs, how fraud leaders can choose partners with fewer regrets... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/picking-fraud-vendors Keywords Primary Keyword: picking fraud vendors Secondary Keywords: fraud vendor POC fraud vendor implementation fraud vendor SLA fraud vendor relationship management fraud vendor black box black box fraud tools chargeback vendor fraud prevention platform fraud detection platform identity verification vendor doc verification vendor

So, welcome back everybody to the Saturday Fraud Strategist Podcast. And with me today is Holly Sandberg. Holly is someone I got to know a couple of months ago, and we even had a chance to meet in person in London at the MRC. That was, I think, around April. And Holly, correct me if I'm wrong, you've got like around two decades of experience in trust and safety, as well as payment fraud, mainly on the merchant side, but not only. So I'm super excited to have you on the show. We'll get to the topic in a minute, but maybe before that, for those of the listeners who don't know who you are, maybe you can tell us a bit about yourself.

Sure thing. It's good to be here with you. I think you've just solved a mystery for me actually, which is why I often say I stopped counting at 15, because, oof, to hear it characterized as two decades, you just made me feel old right there.

Yes, I just dated you, I'm sorry.

That's okay. It's okay. I'm embracing it. It's just easier for me to say fifteen plus ish. But yes, entirely on the merchant side. Primarily I've worked in risk, fraud, trust and safety in two-sided marketplaces, also quite a bit in event ticketing previously, so around concerts, live music, sports. Working with professional sports teams, college athletics programs, all of which get hit with some pretty interesting types of fraud, particularly the two-sided marketplace experience. A lot of insight into the way that fraud is evolving. And, like you said, it's been a lot of years, and I fell into it sideways, as most people do, and got the bug, the thrill of the hunt, or whatever you want to call it, and have really loved doing it ever since.

That's awesome. And what are you up to these days?

So I most recently was in a two-sided marketplace, as I mentioned, and doing some consulting work right now, back in event ticketing, before I have a reveal coming up pretty soon about what my next role is going to be.

Sounds both exciting and mysterious. Awesome, Holly. I can't wait to dive into our topic today. But actually, just before that, I've got to tell you about the night I just had. Basically, about a year ago I started running. And, like my stupid self, I injured myself, and I didn't take care of it, and I ignored it, and I let it simmer. Until a few weeks ago, I had enough, and I went to a physical therapist who diagnosed whatever thing in my tendon, my Achilles tendon, in my right leg. And I started going to this physical therapist for these weekly checkups. And I was there yesterday. And just before I left, he asked me, are you putting any cream on it? And I'm like, I don't know what you're talking about. We never talked about the cream. So he gave me a prescription for a cream. I got it, came home, took a shower, and just before settling down, I applied this cream to my leg. And after about like 15 minutes, 20 minutes, I started feeling this tingling sensation. And I kind of said, okay, I didn't expect that, but all right. Gave it two hours, at around 11 p.m. I went to bed. And next thing I know, I wake up at 3 a.m. and my leg is on fire. It's on fire. And I probably put too much of it. And, as I'm trying to wake myself up and figure out what to do, I'm kind of asking Claude what's going on, and basically understood that I need to go and take a shower. And then I understood that I probably put too much on, and I just needed to wash it off.

So here I am at 3 a.m. and I'm taking a shower, and obviously I get back to bed, and I cannot fall asleep. And I think I only fell asleep at around 5 a.m. And the entire point of this story is to basically say, good news, my leg is currently okay, and I didn't have to amputate it or anything. But not all of my faculties are with me today. That's basically the bad news. The good news, though, is that the topic that I wanted us to discuss today is actually a topic that, I don't know about you, it gets me going. I get more than excited, get emotional about it. And that is, in general, vendor selection, or fraud vendor selection, and all the happiness and joy that it brings all the involved parties. So, Holly, somewhere between 15 and 20 years of experience, I'm sure, on the merchant side, you've seen a lot of these processes yourself. So let me start by asking, is there a right fraud vendor? When we're thinking about how to pick the right fraud vendor, is there such a thing?

No. And for podcast purposes, you probably want a lot more words than just no. But rule number one that I think is most important to recognize, and I have to say, I'm as fascinated as your telling that story. My mind was going in a very different direction. I thought you were gonna say you used this stuff and had an epiphany and said, where's this stuff been all my life? I feel like I have a brand new leg. Didn't think it was gonna go to my leg's on fire and maybe imperiled, but I'm glad that you still have it, and that you're doing okay.

Ha ha.

Okay. Almost every kind of story about something I think going sideways like that, could for you and I, people who've been around for a long time, somehow, I'm sure we could six degrees of separation our way back to it relating to a vendor story. And there's this huge influx of marketing. Like our industry, so to speak, like risk and fraud and chargebacks particularly, where there's a lot of this happening right now, has been pegged.

Me too.

By folks looking for exits, and as being an underserved industry. And what's super interesting to me is, with an underserved industry, this might be where I sound like a cranky old lady who's about to say get off my lawn, is that sometimes there's a lot more that goes into assessing vendors because there's this huge influx. There's an influx of venture capital, there's an influx of new players, there's an influx of marketing promises.

Mm.

That you, me, especially being on the merchant side, and others on the merchant side, have to really consider and really do their due diligence as to what is marketing fluff and what is substance. And that gets that much harder when, you know, when I first started doing this, there were a handful of, I think, what we needed was solution partners and solution providers, and there were very few. Now it's a very different story. There are many. And it makes that much more important to do your due diligence. And part of the due diligence, I think, going back to your original point, needs to be, what do I need? What are the challenges specific to my business? If you speak to someone and you get a referral from someone in a very different line of business, what worked for them and what they said, this is exactly what I needed, might not work for you. So knowing what questions you need to ask, and that has to start at home, so to speak, with really understanding your challenges, your business, your data, your resources in terms of engineering, and I could go on and on. So that's the longer answer to the short two-letter with a period, no. There is no such thing as a perfect vendor.

It's interesting, because just before we went live, we were talking about how you brought up how trust and safety is a term that can mean completely different things for different people. And then I said that, yeah, the same thing can be said about fraud as a term. And what you just said reminded me of that in the sense that sometimes when we get these referrals, we get it from people who really mean well. And maybe both of us think that we're in the same line of business, or that we have the same problem, or that we're looking for the same type of solution, because we're looking for a trust and safety vendor or for a fraud solution. But actually we mean two very completely different things.

Definitely. And that goes back to, where do you need protection? Where are you? What are your pain points? Where are you protected right now? Are you protected at all? That's a big one. I was, in preparing for having this conversation with you, thinking through some things that, as you have a lot more experience, I think some things just get sort of built into the fiber of the way that you think. And the danger is, you might not even think, if you were talking to someone very new who came to you and said, I really could use the advice of someone experienced in this,

You might just have those things so built into your thought processes that you wouldn't even think to call them out to someone who was a beginner. So understanding where you need to be protected, where you are protected now. One thing that, and you and I definitely use the word rant, and a couple of words probably similar to rant, in talking about having this conversation, is the promises that you see from vendors are often, when they've come in where there was no one else previously, which is fine. I think you really need to understand that as the merchant. When someone says, we reduced fraud by 97%, I would bet you good money that in most cases this is a newer company that either had maybe some in-house processes or some manual processes, and probably didn't have, even if it was a solution provider that they weren't crazy about that they knew they needed to move on from,

It's probably still doing something. So going from something to nothing, and I say this frequently with fraud, if reducing fraud when it's, my gosh, stop the bleeding, is kind of easy. Easy's the wrong word for it, the best one I've got right now. So getting from that "it's on fire" to "it's below one percent" is easier, I would say, than getting it down ten basis points once you've already got something in place. So understanding that sort of what is your landscape, current day, of where you are and are not protected in terms of touch points throughout the user journey, anything from account creation.

No, that's such a good call out. Okay, but let me ask you this. I mean, today's market is pretty much commoditized. If I'm buying for a device fingerprinting solution, there are of course a lot of vendors and they are doing pretty much the same thing. If I'm looking for an IDV, or a doc verification solution, there are a lot of solutions offering the same thing. If I'm looking for a fraud platform, then there are plenty of fraud platforms out there that offer rule engine and machine learning models and so on. So supposedly, if all of that is kind of the same, more or less, if we kind of clean out all of the marketing fluff that you mentioned, supposedly, what's the harm? What can go wrong? How? Why does it matter so much, picking the right vendor?

It matters, well, it matters in terms of what you want to accomplish broadly. In general, if it is that big reduction that I talked about, because maybe you've got some real problems, and obviously that's gonna be focus number one. Even if you don't, I would say if you're in that more optimization sort of mode that I just spoke about, where you've got things relatively under control, but you know you can get better performance, better service, better relationship management, better a deeper integration from the technical side. So I think it matters both in terms of the specifics of the product that you're using, but also uptime and reliability and pricing. I've seen certainly people make decisions where they've really, really wanted something and said, I would love to have this, but I don't have the budget. It's a really expensive vendor that I can't work with. So there are always trade-offs. And I think for anyone to assume that there aren't, they may be going down a little bit of a dangerous road. And it's okay to say maybe if two things are very similar, there are lots of businesses who win. What I would question about that is, are they similar at a 30,000 foot view? When you really get down into a granular, into the weeds view, are they still similar? That's where a lot of the work needs to be done, I think, to understand what you're getting yourself into. But if you made a decision and say went with one where you thought it's gonna perform well enough and it's a lot more affordable, certainly lots of businesses have made that call, or a lot of fraud leaders will tell you they had that call made for them, even if it's maybe not the call that they wanted to make. So that speaks to the why it's important and the what can go wrong. You could have fantastic tech and a terrible customer relationship management team. You can have someone who doesn't answer your calls if

You can have someone who says, yep, we're down and we can't tell you exactly why or how quickly we'll be back up. So these are just a couple of the things that come to mind for me immediately, that either I've experienced or in mentoring folks or talking to other folks in the merchant community, you certainly hear horror stories. The things that go wrong will always travel further than the things that go fantastically well. So anybody who's been around has probably heard some of those stories about being throttled by their fraud vendor. That was one that kind of blew my mind, where I won't name names, but it was, you know, you've got too much fraud for us to fight your fraud. That's a very specific example of something that still I can't say without my head exploding. That is the need to really get into the weeds in terms of speaking to other customers of a particular vendor that you're going to work with, and also really making sure that you go through that contract line by line.

Wow.

By word, by word. And you understand, hopefully nothing ever goes wrong, but you're prepared if something does, and you need to be.

Well, without naming any names, I'm guessing, you've done, you went through several leadership roles on the merchant side. And I would assume that it did occur that when you stepped into one of these roles, you probably inherited a system that was poorly designed, and maybe to a large extent because of a suboptimal, let's call it like that, vendor selection. And I'm wondering, just without kind of analyzing what led to that decision by your predecessors, I wonder what were the implications? Basically, what would that mean for you as a fraud leader, that you need to work with a vendor with a tool that is not fit for purpose? What's the worst that can happen?

It happens a lot. So before I go to what is the worst that can happen, it's important for me, I've been thinking, because we're in conference season, and sometimes I also will get a little bit ranty about conference season, although I see the value in it, and I've gained a lot of insight, a lot of important connections at conferences over the years, but by necessity, they're talking almost about either what is emerging or what is three ticks past emerging. And that used to get on my nerves a bit from a having-inherited-existing-fraud-tools perspective. You're talking about the next big thing, whatever, back in the day it was big data, then it was machine learning, now it's AI. I have to fight the fraud today. But I think I've come to realize that it serves a purpose, that there needs to be something that drives us forward, and having those conversations that get all of that attention and LinkedIn posts and everything that comes with conferences is necessary. Oftentimes in the hallways and the happy hours, I think is where you'll find the, okay, but I have to fight what's happening today, not what's happening a week from now.

Mm-hmm.

So, in terms of having to basically sort of MacGyver your way through what you've got, it's okay, first of all. I used to, and the reason why I started down the conference road, sorry not to go off on a tangent, was when I was brand new to the industry, I would go and see all these people talking about this and assume they all had it, and go home kind of dejected, like, well, I don't have the budget for this, or I couldn't possibly get this on a product roadmap to get this best in class that everyone is talking about. Then I've come to realize that a lot of the people talking about best in class are doing it for that sort of moving us forward as an entire community that fights fraud, rather than that meaning that everyone has the thing that they're talking about. So first and foremost, I think for your listeners who are newer, I have to say

Mm-hmm.

That if you inherited whatever you inherited, if it was some bubble gum and a paper clip and some tinfoil, you've gotta make that work, and that's perfectly okay. I'm gonna pause for one second just to clear my throat.

I'll take a sip.

And I have been in that situation. And you segued me actually nicely into something that I feel like on your last question I forgot to say, which is, black box. I think a whole episode just on the words proprietary and black box, which was my first experience really with comprehensive fraud tooling, which I inherited. And that was before, one thing that I love actually about the sort of AI world that we're in now, is we're talking so much more about explainability, and explainability has always been something that's been important to me, because going back to your what's the worst that can happen, what can go wrong, I have had black box fraud tooling go rogue. And without getting too specific, this was many years ago. You can track back to where I worked at the time, and it was in live events. It was during a major sale event, and it started canceling, I think it got up at its worst to approaching 90% of all orders. And everything here, I think, is this sort of web of associations that ties back not just to the tech and to the capabilities of the tool, but also to the relationship management, because I needed to get a hold of someone immediately who said, how do we turn this off? We'll deal with, if we have to do post order manual review, whatever the case may be, this is what the fire drill really looks like. And when you're dealing with something black box, and there is no explainability factor, and I have to go to my leadership, or I have to go to a partner in this endeavor where things went the wrong way, it's not an acceptable answer for me to say, well, our fraud partner says that's proprietary. I'm gonna get my head absolutely chewed off in that scenario, and someone's gonna tell me to get immediately out of the room and come back with a much better answer than, they say they can't tell us why they rolled over and died when we needed them most.

Absolutely. I'm so with you on that. I must say I really like the fact that you mentioned MacGyver, because I think honestly, I think that is the right word that I was missing all of these years to think about how fraud and risk leaders in general should perceive their role. Because I think I'm totally with you on that. I think that, by definition, you will never have an ideal state. I don't know any fraud leader that would say, yeah, my team is perfect and my stack is perfect and we're on top of things, and there are never fires, and my life is just sweet. Things are always broken and things always need to be patched up. And you never have, I haven't seen it, at least. You never have the privilege to wait for someone to just kind of tie it up in a bow and present a perfectly working, smooth system. You always need to use gum and a paper clip, as you put it. And I think this is so important. And I'm not sure that all the fraud leaders that I've met throughout my career accept that, that this is actually their job. This is not a bad thing. This doesn't reflect bad on you or your organization or your team. That is the job.

It is absolutely the job. And I was thinking, maybe I'll be a smidgen more optimistic than you and say, maybe you achieve perfect, but you achieve perfect for a day, and then the fraudsters figure out what perfect is, and then tomorrow you're right back in it. So you find out very quickly, I think, in this industry, whether you truly love that, and you're along for the ride and up for the ambiguity, and up for the cat and mouse game, whatever you wanna call it. And I think the very best fraud fighters do

Yeah. Yes.

That, where people need to learn, and this also ties back to vendor selection, is how good a partner is your vendor in going on that journey with you, and talking about, here's what we have to stop today, but here's what's coming. There are a lot of questions, and I'm sure we'll get to that, in terms of how you ask them, what they're doing to sort of future proof against what's coming next while protecting you for today. But it is okay wherever you're at. It's shocking to me. Maybe not shocking is the wrong word, but I know some very large companies who still have some processes that people would be surprised. I think the me who was at my first MRC with my notebook thinking I was gonna get all the answers would go, this like Fortune 50 definitely isn't still doing things with spreadsheets. They might be. There's a pretty good chance that there's some process somewhere actually where they actually are doing that. So that is, I think, what makes the job lovable for those of us who love it and want to stay in it. It's where people fall out very quickly. And I used to, once upon a time, manage accounting teams, even though I am not an accountant in any way. No one would want to make me one. But it was interesting, and the way that I would phrase it was, when you're in that line of work and you have that mindset, two plus two is always four. In fraud, two plus two might be four today. It might be 17 tomorrow, and you're either gonna love that, or you're gonna hate it. And if you love it, you're going to say, well, I'm backed up against a wall and I've got a paper clip and some chewing gum, and I've got to make it work. Then maybe you should always have one eye on, what would this be like if I actually, or maybe to move away from the MacGyver reference, I would, if I was driving a Toyota Corolla, and you have to think to yourself, well, I have to get somewhere in this Toyota Corolla, some percentage of your mind should be thinking about what it would be like to get there in a Ferrari. But you've got to balance that. What am I working towards versus what am I working with right now?

Yeah. Hey, dreams are always great to have. A man can dream. So tell me, I mean, I'm completely with you. You mentioned the fact that the tech can be very shiny. But one thing that you want to look at is, of course, the kind of service wrapper. The black boxiness, the SLAs, the support, and so on. I also wonder what's your view on the layer that underlines technology, which is data. Because one of the things, for example, that I see is, sometimes organizations would take up a vendor that just doesn't have enough experience in their industry, and is used to manage completely different fraud typologies, or completely different payment methods, or completely different use cases. Maybe fraud is not their main use case. And usually, I don't know why, but they are surprised to see suboptimal results. Let's call it like that. So what's your experience with how to look at the data layer and the experience that the vendor actually has in your industry?

Yeah, and I guess for starting out answering this one, I will say something that's not entirely beating up on the entire vendor community, which is there is a mutual responsibility in the success of these partnerships. And that sometimes gets lost on merchants. If you are putting in a vendor, and I'll get to your sort of the experience of a vendor in a particular vertical in a second, but also, it's not just about, there's a lot of kind of catchy phrases that we talk about. We talk about build versus buy, we talk about build and buy, we talk about layering different solution providers. All of that is really dependent on how well you build it. And a lot of vendors, I think, maybe sometimes shoot themselves in the foot by saying, like, we're a light integration, we're easy, we can be up and running fast. Those are all not to say that those aren't important things and that should be part of your evaluation process. But if given a choice between two more weeks, or a couple more sprints during an integration, and what actually gets built, are we rolling out a minimum viable product for fraud, is always something that gives me some pause. And vendors, I think, sometimes need to push back a little bit more on merchants specifically, and say, we've got the deal signed now. And then say in comes the team that's doing the integration and saying, well, we want to stand it up as quickly as possible. We don't need to send this piece of data. We don't need to send that. We don't need to send something at this particular inflection point in the funnel. Every single one of those things is like, you're starting out with this bright, shiny vision of how great it's going to be, and you're like death by a thousand paper cuts, undermining exactly what that vision is that you've built for yourself along the way. So the data is exceptionally important. What are you sending? When are you sending it? Is it pre-auth, post-auth, is it real time? And a lot of those considerations, the work starts, to me in my mind, when you kick off an implementation, not, okay, we've selected a vendor, and we're gonna wait till we flip the switch and turn that vendor live and everything's gonna be great.

It very seldom, I don't know anyone that it's ever worked out for in that way. So the underlying questions about, how do you understand my business, my vertical, nuances specific to the type of fraud that we get. If you were to take a luxury goods vendor right now versus a subscription streaming service, who's running recurring charges for 15 bucks a month, those things look, that's a material difference in how you fight that fraud, even though all fraud is fraud, and it needs to be a consideration. Not meaning that if you're that luxury goods merchant, you don't talk to that fifteen dollar streaming service. I think you do, but you do it with some caveats and understanding what the business differences are.

Yeah, absolutely. I love it. I mean, I don't love it, I hate it. All of that really reminds me of the times that I was a vendor, and I saw firsthand exactly that dynamic that you just described. And it was always so frustrating. And by the way, I always beat myself up because I was the enabler of this behavior. I always say, no problem, no problem, no worries, just do the most basic integration and we'll take care of the rest. And you find out, like six months down the line, that you're missing some labels, or the labels are not coming in time, and it screws up the entire model training process. And now go explain that to the client. So yeah, I know that I'm gonna have some nightmares tonight. So yeah, definitely with you on that. So okay. So you definitely want to pick the right fraud vendor. I hope no one is surprised by this conclusion at the halftime of this conversation. But why do you think, what are the most common reasons that you've encountered for teams or fraud leaders to pick a suboptimal vendor? And you mentioned pricing, and yeah, pricing is always a good excuse. But what else, what are other pitfalls, or what other considerations would you highlight for our audience to be more aware of when they're going through these processes?

Pricing is definitely a factor, and one that you probably, if you got a lot of fraud folks in the room, they'd have a lot of opinions about. In terms of other areas to think about, I think oftentimes vendors come to fraud leaders, maybe, they're not, the fraud leader is not the initiator of the conversation with a particular vendor. Maybe that particular vendor has very aggressive salespeople who will reach out directly to your C-suite, and that filters its way down to you, and you find yourself as the fraud person having a conversation about something you're pretty sure right off the bat that you don't want. And I have to tell a story here that I think hopefully will do someone some good. A CEO of one of the companies that I've worked for, who is a super smart, fantastic leader, one of the best folks that I've ever worked with, passed along to me, And this was when I was relatively more senior, and I think I fielded it in a very different way than I would have once upon a time. Sent me an email and said, hey Holly, met with X, they're interested in talking to us about their fraud solutions. Do with this what you will. The fifteen-years-ago me would have freaked out and said, my gosh, the CEO definitely wants us to work with this person. This is probably their, I don't know, their best friend from college, their startup, or whatever the case may be. They definitely want it, I'm feeling this pressure. Now I want to have a good relationship with my leadership. I want to do what they want me to do. And I probably would have leaned more into working with that vendor, and in this instance, it was someone I don't have a particularly great opinion of. And come to find out afterwards, and, to his credit, he just said, I'm just passing this along, have the, you know, it's your decision to make, have the conversation, and I understand. I will never ever take that for granted, because I know a lot of leaders who've not been in that position, where they've just had their fate dictated to them, and then they had to make it work, going back to what we talked about before. What I found out later, which still cracks me up to this day, is this referral came because our CEO had stayed at an Airbnb that was owned by the cousin of someone who works at this vendor. He did not care at all. He literally, he meant it truly when he said, Holly, I don't care what you do with this, it's your decision. I'm just forwarding the email. But

Yeah.

Had I not said, I have some reservations about them, and really took him at face value in saying, like, I am not pressuring you in any way, and imagined pressure where there was none, I would never have thought, I could never have predicted that the way that referral came to be was because he happened to be standing outside near a koi pond on a Tuesday and struck up a conversation with someone and said, yeah, I can make an introduction. That's one thing I think also to think about, is how these conversations happen. Did you go out for a formal RFP? Did you go out for a more structured process to evaluate vendors? Are you kicking tires? Are you visiting booths at a conference? How did the conversation come to be, is also a part of how these things sometimes tend to happen, and can also go wrong. Pricing, I think, and the biggest one, going back to other sort of bullet points on an answer to your question, I would say the other big one besides cost, probably, is integration speed. And when you get other teams involved, those would be the big two that I hear the most stories from other merchants About. It's in terms of, they're not just pitching to you as the fraud leader when they come in and talk to you about their solutions. They're also talking to your engineering team, who have different priorities. They're talking to your product team, who have different priorities. Maybe they're talking to your finance team, who's thinking mostly about cost. Your engineering team is thinking mostly about how much work do we have to delay to do the work, or how do we slot this into a roadmap, so to speak. So it gets pretty nuanced pretty quickly, as far as the reasons why those decisions are made. And I think your job, if for those who are listening who are leaders in fraud, is to stick to your guns to some extent, really, and going back to the two-way street also, when you're in those, let's say you've made a vendor selection, you're now in integration, you've got the project management team from whichever vendor you've selected on a call with your technical team, as you're working through the integration, what data we're gonna send, how are we going to set this up. Oftentimes you're in the door already, right? You've closed the deal. The fraud team is rooting for you as the vendor to say, don't do that, that's dumb. Maybe not that way. You really don't want to make that trade-off. And I think sometimes not all solution providers will do that. They just go into

You know, yes man mode, and just say, okay, if that's the way that you want to integrate it, we'll work with that, we'll work with that. And between the fraud leadership and the people who work for the company who know fraud best, and the solution provider who's been selected who knows their product best, I think there's a real responsibility to speak up and say, you could do that, but I have to really strongly advise you against it, because it's gonna undercut the performance of the partnership.

Yeah. It's interesting, because my experience is that in a lot of organizations, the real buyer, meaning the function that in the end would approve the budget, is many times not the fraud team or the fraud leader themselves. And while the vendor believes the fraud team is the buyer, they are actually the champion. They are the, you definitely want to convince them so they would recommend you. But they are actually usually more a champion in the organization. Once they chose you, they really want you to succeed. And I think a lot of vendors don't understand that, and continue to try to prove their worth to the wrong party.

Agree. And that champion really, really wants an ally, which is, I think, at that point in the process, what you need to be if you're on the solution provider side. Understanding that there will still always be trade-offs, but to not just go, uh-huh, okay, okay, okay, and not push back on anything. Sometimes I think the very best working partnerships that I've had with vendors have been when there's some serious pushback, as there should be. I'm at the table because I understand the business that I work for, and the nuances, and what we need, and how our sales process, our checkout process works, how all of those different things. You, as the vendor, I think, are on the other side of the table because I can come to you and hopefully accurately depict to you what I need, and what I have available to give to you in terms of resources, to get the best use out of your product. You're at the table because you know the best way to use your product to get to the destination that I'm telling you that I need to get to. And I want you to tell me if I'm about to turn off on some dirt road full of potholes that's gonna leave me stranded.

That is so true. It is so true. I agree. And if there are any vendors listening, I don't know how much vendors realize that doing exactly what you just described, how much trust it instills into the buyer. And whenever, instead of trying to satisfy and please, you actually provide guidance, and you provide, let's say, not red flags but warning signs, this is something that would position you exactly as you kind of said, this would position you as an ally rather than, you know, a salesperson or kind of a sales function. Yeah, definitely. One hundred percent agree. Okay. So any more pitfalls, common pitfalls that we should know about?

I think, I'm still amazed at how many stories get back to me of folks not doing their due diligence, putting aside everything you and I just talked about, is the fraud leader truly the decision maker? Are they a decision influencer rather than a decision maker, which there's a big distinction between the two. There are still a lot of people who, there are vendors that I see who spend, and this is changing quite a bit with AI, but who spent a lot of money on SEO. And you're a newer leader who's just been sort of put in charge of the risk function, say at a company that you work for, and you've got a chargeback problem, for instance. And so the very first thing you do, you're sitting there alone in a room with no one else to talk to, you're gonna do some searching on the internet. You're gonna come across maybe a vendor who's put a lot of money into the impression that you have of them in that moment. And that's fine. Nobody ever started out on a journey towards vendor selection and didn't do some Googling. I'm not saying I'm against that. What I would say, where I've seen it go wrong, is where it stops there.

Mm-hmm.

And understanding, and I sometimes feel like I come across a bit too harshly to folks on the solution provider side, and I don't mean to. There are some I have a tremendous amount of respect for, and I think really know their stuff and really do their jobs exceptionally well. But these are people who are in sales positions, whose job is to tell you that they're gonna solve your problems, and it's on you to do more than go beyond glossy marketing. Or go beyond hearing things anecdotally from people. I would say it's important to talk to other merchants, but also to maybe ask some questions of the impressions that they're giving you. If they say, X is terrible, don't work with them, well why? Do you have direct experience with them? Did you hear about it third hand from someone's dog groomer's mailman's cousin who said they're terrible? So there's a weighting of the information that you get, and there's a journey, I think, that you need to go on and trying to get that information, both the specs and the technical information about what a potential partner can and cannot do, but also how they manage their relationships, how are they transparent?

Yeah.

At 11 o'clock Friday night. So I've definitely seen both the good and the bad of that. So leaning too much into, or not doing more digging, I would say, against those marketing claims, however you come across them, whether it's by word of mouth, whether it's by Google search, whether it's by someone schmoozing you at a conference and taking you out for a nice dinner or inviting you to a party. I still see a lot of people who say, I made the decision because I really liked that guy. And at the end of the day, if it goes badly, that's not an acceptable answer. And I think you should have some understanding of your responsibility as a leader, that you championed a particular selection. You own that if it goes really well, you also own it if it doesn't.

That is so true. Yeah, I think distinguishing between what is marketing fluff, as you put it earlier, and what claims can be backed up by real experience, real results, that is definitely an important part. So I was just about to say an important part of the process. So okay, there are all of these pitfalls. We heard all of these horror stories. By the way, I must say, I've seen, in my career, as a buyer, as a seller, and as a consultant, I've seen a lot of vendor procurement processes. And honestly, when I was thinking about it, before recording this episode, I honestly think that most of them went bad. Like I would say at least 50%, the buyer wasn't happy. And maybe it took two months, and maybe it took two years, but I think most deals, in my experience, don't, and they don't hold for many times. I see very little cases where teams are happy with their vendors three, four, five years in a row. So, as Claude says, the pain is real. So what is the right way to do that, and how can we mitigate the risk that we will choose the wrong vendor? What's your playbook? What's the process that you run when you go into such a process? And obviously, I know it's a difficult question, procuring a new IDV provider versus a new fraud platform, It's completely different processes. And there are a lot of different, whether these are the specific questions or the dimensions that you look at, but still, high level, how should we go about it?

It's a lot more challenging than it used to be, first of all. And this is, we've now arrived at the portion of the conversation where we're required to say AI every 15th word. And it was difficult before that. It's so much more difficult now, I think, with the speed with which things are going. So I use the term future-proofing quite a bit, and I think the best way to go about it, probably, is to pull your stakeholders together internally to have those conversations in advance. Prepare them for the moment where you're gonna arrive at an intersection where there's a trade-off, it'll take an extra sprint if we send the data at this point. And if we send these additional pieces of data, or if we choose one model over the next, if you're dealing with a vendor who's actually got kind of multiple options for what would be best for you, and you're considering both the quality of that solution, probably you're considering cost, you're considering a lot of things. Everyone is invested, maybe they have different priorities and different reasons entirely, sort of in-house on the merchant side. Have those conversations before you're in integration. Even before you've selected a vendor, if you've got really good partners within the business, if you don't, go out and make it happen that you get really good partners within the business, so that everyone sort of understands, we want to set this up for success as much as possible. Have as many of those conversations before something goes wrong. I used to have a team, and this is with a vendor that we had particularly really good experience with in a very strong partnership. One of the rarities that's different from what you're talking about, and we would get on recurring calls with them every once in a while. And sometimes there was nothing to talk about. And we just get on and say, hey, how was your kids' softball game, or how was your trip to Aruba? Or this is the concert that I went to last weekend, or, you know, I'm a football fanatic, so inevitably I feel like I end up talking

About football with anyone who was willing to talk football with me. So there'd be some talk about the horrible performance of the offensive line in last weekend's game, whatever the case may be. But there's nothing of substance, so to speak, to talk about on that call. And I had a member of my team ask, why don't we just cancel those? Rather than what we would often do is get on them and say, okay, say it's a 30 minute call, and we'd go, okay, we're giving you back 15 minutes. But I would always say, no, we don't cancel those, because it's important to have the relationship, so that when something goes wrong, I am not a net new entity to the person that I'm dealing with when I need to handle the challenge of what is suboptimal that I need to fix. And the same thing for them, going back towards me. We have an existing relationship. We're not strangers to each other.

Yeah. And something will go wrong, right? It's just a matter of time.

Yes. It will. And learn to feel comfortable saying that. I think that's another thing that I've learned over time, is so many people are just afraid that it reflects on them if they talk about that reality in the moment, and what you and I are saying is what the reality is. I think you position yourself better as a leader, both to your team, to your leadership within your company, in your vendor partnerships, just sort of holistically all around, if you're having pragmatic conversations about the reality, and not fantasy conversations about ideal things that are just never going to be the reality.

Absolutely. Okay, so first step in the process is, get your in-house stakeholders ready, aligned, prepared for this. Okay, so I've done that. What's next?

I think aligned and prepared then leads into really good, robust documentation, which can sound like not the sexiest, most exciting thing, but that is maybe a part of the first step that you were talking about, or the logical next step, is you need to have a structured process around this entirely. If you feel as if you're a passenger on a bus, so to speak, and you're not driving.

Yeah, I fully agree, and it's interesting because I think the details, almost, I think the details matter ten percent when you compare it to just having that structured process that lets you first of all own the decision, of course, and B, also make sure that the decision that you make is clean and isn't easily influenced by the cousin of the Airbnb owner where your CEO stayed. This is exactly the way, this is exactly not the way to solve, but at least to mitigate the risk of external influence. One hundred percent. So I got my ducks in a row, I got some sort of an RFI, RFP process that I'm ready to go about. And what's then? Do I just call the first five vendors that I find on Google? Or what else do I wanna make sure that I'm ticking before I make a decision?

There are considerations, and they're not considerations that everyone can make. Do you have the wherewithal, the ability to do a head-to-head POC? Not every merchant can. That's the reality of it. But I think that's the best possible scenario, if you've got the resources, both in terms of engineering resources, technical resources, and the time. You might have the resources in one respect, but you don't have the luxury of time to be able to do a POC. Sometimes that's also the reality of it, is there's an ideal way to do something, and then there's the moment where the ideal way to do it clashes with the reality of what the need is today. And you've got to make, if those trade-offs were easy, everybody in the world would be doing this job and wouldn't have as many horror stories to tell about it. Understanding sort of what your capabilities are. If you can do a head-to-head POC, wonderful. Do it if you can. If not, then the due diligence that I've talked about quite a bit throughout this conversation becomes that much more important, because you're rolling the dice to some extent, and you need to make that as much of an informed bet as you possibly can before you make that selection. And then I think the next step that follows, obviously, is your

Yeah.

Gonna, you're gonna go into contract negotiations. And if you've ever needed to be a stickler for details in your life, that is the moment when you need to find your inner extremely type A vibe and apply it with all your might to that contract process, because the devil is really in the details.

Double click on that, because I think this is interesting. I think it also goes to the point that you highlighted back at the beginning of the conversation, where many times we are obsessed with the technology, and this is kind of what we're testing in the POC stage. But what gets neglected many times is kind of the servicing wrapper. And I'm guessing that this is what you refer to when you're talking about contracts.

It is. And oftentimes, and this is I think a relatively new development in the industry, that a lot of vendors have a model where they make particular promises around performance, or around guarantees, around coverage if they get it wrong. The fine print on that, and that also speaks to stories that merchants tell each other at happy hours of conferences, is well, they say that if they approve an order that turns out to be fraud, they'll pay for it. But there's

Mm.

There's a whenever there's a "but," or a "however," or "in case of," that's where, you know, previously you've leaned into your technical relationships internally, this is where you really need to lean into hopefully your good relationship with legal. And the way that I would look at any of those promises, SLAs, uptime, what are our potential payments if that SLA is not met, or potential redress if that SLA is not met, those are all important things. And the way to approach it, I think, and this is an area actually where our generative AI friends can help us quite a bit, is how would you attack this? How would you, if there was a lot of fraud getting through, let's say you're dealing with a vendor who's making specific promises around a pegged reduction in fraud rate, and that fraud rate is not happening. What is the recourse? And you can run sort of hypothetical scenarios now with a considerable bit of speed. I would not recommend doing this solely with AI. This is an area where AI is a booster and an assist and an amplifier, but not the whole ball game, but can lead to maybe some better informed conversations with legal around, if this went wrong. We always say, hopefully it doesn't, but if it does, and you have to have that "if" conversation, I think that's your job in a nutshell. If you're on the merchant side and you're making a decision like this, how would they come back and say, actually, we don't have to honor that? Or, sure, it says that we'll do this if we don't do that, but it's dependent on you running 3DS on transactions, or it's dependent on you having protection upstream that maybe you don't have. So I think it's

Mm-hmm.

Incumbent on you, when you're in that part of the process, to go looking for the gotchas. And the best way that I found to do that is to kind of attack it almost like an incident planning scenario, so that you're prepared, and you maybe still go down the road, you don't walk away from that partnership with that vendor, but you probably come away with some red lines in that contract. And it will tell you a lot if they push back on them, or if they go, okay, yeah, that makes sense, we'll accept those and we'll incorporate them into the agreement. I would argue that the way a vendor responds to that will tell me a lot about what your future performance and your future relationship with that vendor is going to be like.

That is excellent. I really love it. It's so good. I must say that, for those listeners who don't have experience with these kinds of models, I will just highlight the fact that this can also be huge in terms of financial repercussions. These are not just nice to have. These are really, really good points around contract negotiations. Awesome. Cool. Holly, wow, we covered so much. So let me see if I can kind of summarize the takeaways that I got throughout our conversation. So let's start with, answering the question, is there a right fraud vendor? The answer is no. Enough said. I think also, before we deep dive into the why don't we have a right fraud vendor, and what's the process about finding out and deciding on the one that is right for you, we also talked about the fact that there are two different worlds that you want to consider, and also to consider in which world you are currently living. The world of zero to one, everything is currently burning, and anything, or at least most of what you are going to do, will probably be very, very helpful. Versus kind of business as usual optimization mode, where you're trying to go from the 95th percentile to the 99th percentile. And that is, as you say, much harder, requires much more effort, and also might impact how you think about things and how you think about your vendor decisions. Another point that we covered before we got into vendor selection, is also internal expectation management. And we talked about the MacGyver mode, or as you said, you need to MacGyver it sometimes, and with which I fully agree. And I think that fraud leaders, in my mind, I

For whatever reason, I thought a lot about it in the past few weeks. Fraud leaders are measured by their ability to make decisions without data, without the privilege of having data. We always talk about the fact that we want to make data-driven decisions. And yes, obviously we want to opt for that. And fraud leaders need to be very good at making data-driven decisions, how to get the data and so on. But also, fraud leaders are measured by their ability to make bold decisions when the shit hits the fan. And this MacGyver mode, both being able to decide as well as being able to execute with whatever you have on the table right now, that is so important. And I think this is truly, if there's one thing that I would impart on a fraud leader in terms of what would make you successful, it is that. Fully agree with that. Really like that call out. Vendor selection. So we talked, super high level, about the pitfalls. We talked about the fact that it's very easy to get attracted to shiny technology, but actually we also need to consider the servicing around it. And we just talked about it's both the SLAs, how fast would the vendor react when something goes wrong? How do they react when something goes wrong? Can they explain their decisions on an individual level? In general, the black boxiness of the thing. Tech isn't everything. It's many times what the vendor would put at the front, in their marketing and sales pitches. But it's definitely not everything, at least not in our space. We also talked about the fact that, I liked how you framed that success is a mutual effort. And it is up to the fraud leader and their organization to make sure that this is a success,

Or a successful kind of relationship, as much as it is on the vendor to do so. And I think you kind of really hit the nail on the head when you said the decision is actually the easy part. The hard part is what happens from the moment that you've made a decision, and you basically go into implementation mode. The fact of how you implement the decision is probably going to be as influential, if not more influential, than your vendor decision on your performance at the end.

Yeah, no worries. I continued. It should be completely fine.

Okay. I figured you were, so I was trying to keep my face neutral as if I was still listening to you talking. I was nodding, going mm-hmm.

Yeah, yeah. Good. Sorry about that. Let me see where I start. Integration. Okay. Yeah. You also mentioned the fact that in the end, there's a lot of homework to do, and especially nowadays when the space is very competitive and very crowded, it's

Hard part, I think, yeah, we should try to make the decision.

Sometimes it can take quite a lot of digging to bypass all the marketing and sales materials, claims, and promises that are being thrown around, and to actually get to real established results. That is not easy. And whether these results are relevant to you and to your case. And we also mentioned that one other factor that, of course, is always there when it comes to what sinks ships, what makes organizations pick the wrong fraud vendors, is pricing. Which is, again, as Claude said, an uncomfortable truth, but it is also something that is a bit, I think, hard to manage. And maybe my only tip here would be to fraud leaders, is also to try and see whether you can, especially if you have concerns about the budget of your first pick, also to have a second runner-up that is cheaper, and you don't need to position it like that, definitely not to your leadership, because then it's very clear what they would choose. But at least for you, It's very, very clear what happens if your budget is not approved, what is your fallback plan. And finally, we got to the playbook, how do you actually make sure that, as much as you can, you clean out all of the biases and mitigate all of the pitfalls that we just talked about? So you talked about four stages. The first one, which I love, is actually before you even start. Get your ducks in a row. And this can mean a lot of different things, and really depends contextually about who's the stakeholders that you need with you in order for this effort, for this integration, to succeed. And sometimes it can be that you would need to provide a very convincing business case for finance to approve. Sometimes it would be to talk in advance with engineering or with product,

And making sure that you have an alignment about how many resources you actually get for integration and testing and pre-testing and so on, and when you get these resources. So you can also plan your process, basically, so you don't get into the situation that you described, that sometimes you basically need to choose between how do I spend my engineering resources, do I do a POC or do I actually do the integration. And sometimes that's a very hard decision to make. And obviously, it's very clear what you would choose, and that puts you at risk, not having a POC. Second stage is basically to run through your RFI, RFP process. And I think this is also my experience, more important than which questions you ask and what boxes the vendors need to tick, It's just the fact that you have a solid grasp of what is the process, what are the different stages, and what do you check in each stage. And that helps with the cases of all sorts of vendors coming through the woodwork. And I've seen many cases where you have mutual investors, or the CEO is a friend of a board member, and as you said, it's sometimes very, very hard to avoid a lot of bias here in these processes. So having a structured, documented process is important. Testing, of course, make sure this is so important, that you are able to, let's say, validate some of the claims, some of the promises, through a POC before you commit. I hope that's a very clear point where I don't need to elaborate more than that. And finally, contract, and especially this is less about pricing and more about SLAs, and even more specifically about the SLAs when it comes to services where you get specific guarantees and specific kind of performance goals that are being promised.

And here I think I really loved your advice about, to an extent, war-gaming it, and kind of coming up with worst case scenarios, and trying to understand what those scenarios would mean to your business and how you can protect yourself, and also to raise these as questions to the vendor. And I fully agree. I've never done that, by the way, but I'm sure that this would probably reveal a lot about the experience that these vendors have and how they react to worst case scenarios. Better to find it out before you sign a contract. Awesome, Holly. I think it was a respectable discussion. I don't think it was a very hidden rant. I'm very proud of us too. I think when we had our couple of prep calls, I think I was concerned that it would be more heated than that, but we managed to keep it civil. But more than that, I thank you, because I think there are so many takeaways that our listeners can take with them, and hopefully make better decisions the next time that they would need to choose a new fraud vendor, which is unfortunately probably sooner than they are thinking at the moment. Holly, thank you very much. It was a pleasure having you on the show.

Thank you. It was fun. It wasn't, it was a lot of fun. I'm glad you're doing this.

Awesome. Glad to hear it. Thank you very much, folks, for joining us on this episode, and I will see you next Saturday. YouTube Description How to Pick the Right Fraud Vendor For You, with Holly Sandberg July 18, 2026 Episode #13 Guest: Holly Sandberg Choosing the right fraud vendor can be challenging without falling for the wrong promises. A few years ago, I watched a fraud vendor demo what was, honestly, too smooth. The dashboard was clean. The detection sounded instant. The integration was described as "lightweight," which is one of those words that should immediately make everyone in the room sit up a little straighter. Okay. Lightweight for who? Because fraud systems do not live in slide decks. They live inside messy customer journeys, half-documented data flows, payment edge cases, manual review queues, chargeback rules, executive pressure, and a backlog that engineering has already politely said is "full." So when a vendor says they can reduce fraud, improve approvals, lower chargebacks, protect revenue, and do it all quickly, you've got to ask yourself a very basic question. What are they not saying? Not in a conspiracy way. Just in a normal, practical, "someone is eventually going to have to explain this to leadership" kind of way. That is where this episode of the Saturday Fraud Strategist Podcast begins. Not with the shiny version of picking fraud vendors, but with the version fraud teams actually live through. The one with good intentions, unclear requirements, pressure from every direction, and a tool that may or may not behave the same way after the contract is signed. Picking fraud vendors is not just procurement. It is not just technology. It is not even just fraud strategy. It is an accountability decision. A bad choice can create false declines, missed fraud, operational cleanup, customer frustration, revenue loss, and a fraud team stuck explaining why the thing that was supposed to make life easier has somehow created a new category of meetings. Anyway. Very normal. Very fun. What you'll hear in this episode: ● A structured breakdown of picking fraud vendors, and why there is no single "right" vendor. ● A practical look at what is working, and what is still falling short, in today's crowded fraud technology market. ● A discussion of how institutions should think about internal alignment, stakeholder buy-in, and fraud vendor implementation realities. ● A closer look at ethical tensions around marketing claims, fraud vendor black box models, guarantees, and accountability. ● An examination of how poor vendor decisions affect fraud teams, customers, revenue, chargebacks, and business operations. ● Practical considerations for due diligence, fraud vendor POC planning, integrations, contracts, fraud vendor SLA terms, and escalation paths. ● A comparison of how teams should think about a chargeback vendor, fraud prevention platform, fraud detection platform, identity verification vendor, or doc verification vendor, depending on the actual problem they need to solve. ● A call for better fraud vendor relationship management between merchants, fraud leaders, vendors, engineering teams, finance, legal, product, and trust and safety stakeholders. Who should listen: ● Financial institution leaders and fraud professionals. ● Risk, compliance, trust and safety, and cybersecurity teams. ● Merchant-side fraud leaders evaluating vendors or dealing with inherited systems. ● Product, engineering, finance, and legal stakeholders involved in fraud technology decisions. ● Fraud vendors, solution providers, and customer success teams. ● Industry advocates focused on stronger fraud prevention outcomes. ● Anyone trying to understand how vendor selection impacts real people, real teams, and real institutions. Episode notes: (Accordion dropdown) The fraud vendor market is crowded, and that creates noise More tools should make fraud prevention easier. In theory. In reality, more vendors, more AI language, more dashboards, more "real-time decisioning" claims, and more black box fraud tools can make the buying process harder to trust. There is no universal "best" fraud vendor Okay, annoying but true. A vendor that works beautifully for one institution may be completely wrong for another. Fraud typologies, customer journeys, payment methods, internal resources, chargeback exposure, risk tolerance, and customer expectations all matter. A strong demo does not guarantee a strong implementation This is where things can get a little uncomfortable. A platform can look great in a sales process and still struggle if the data foundation is weak, the fraud vendor implementation is rushed, or internal teams are not aligned. A fraud vendor POC needs real planning A proof of concept should not just answer, "Did the model catch fraud?" It should also help teams understand data requirements, decision logic, workflow impact, false positive risk, operational lift, integration complexity, reporting quality, and what will happen when edge cases show up. Black-box fraud decisions create accountability problems If a system declines good customers, misses fraud, or creates unexplained friction, fraud teams need answers they can actually use. "The model is optimizing" is not an answer. Contract details matter more than people want to admit A fraud vendor SLA, guarantee, or chargeback promise can sound straightforward in a sales conversation, and become much less straightforward in the contract. Exclusions, response times, escalation paths, model transparency, support coverage, and implementation responsibilities all matter. Relationship management is part of the risk strategy Fraud vendor relationship management does not begin after something goes wrong. It starts before the contract is signed, when teams define ownership, communication expectations, success metrics, escalation procedures, and what accountability actually looks like in practice. Key takeaways: ● There is no universally right fraud vendor. There is only the vendor that fits a specific business, risk profile, customer journey, data environment, budget, and operational reality. ● Picking fraud vendors requires more than comparing features, dashboards, and promises. Teams need to understand how each tool will behave inside their actual fraud ecosystem. ● Awareness matters, but awareness alone is not a fraud strategy. ● Prevention requires both technology and operational intelligence. ● Vendor marketing claims should be validated through due diligence, customer references, fraud vendor POC testing, implementation planning, and contract review. ● Black-box fraud tools create risk when fraud leaders cannot explain decisions to customers, leadership, regulators, or partners. ● Implementation quality can matter as much as vendor selection. ● Fraud teams need internal alignment with engineering, product, finance, legal, compliance, and executive stakeholders. ● A chargeback vendor, fraud prevention platform, fraud detection platform, identity verification vendor, or doc verification vendor may solve different problems. Teams need to be honest about which problem they are actually buying for. ● Vendors should act as allies, not just sellers, especially during implementation, tuning, escalation, and crisis response. ● Contract details, SLAs, guarantees, and exclusions can create major financial and operational consequences. ● Ethical responsibility should guide how fraud technology is sold, selected, implemented, and managed. The bigger point is simple, even if the work is not. A fraud vendor becomes part of the operating system of the business. It shapes who gets approved, who gets blocked, how teams respond, how risk is explained, and how much control fraud leaders actually have when something breaks. And something will break. Maybe not dramatically. Maybe not publicly. Maybe it is just a slow pattern of false declines, confused support tickets, missed edge cases, chargeback leakage, or analysts quietly building workarounds because the tool is not quite doing what everyone hoped. Still counts. So the real question is not whether a vendor sounds credible during the demo. The better question is whether they still make sense when the real data, real customers, real fraud patterns, real SLAs, real implementation constraints, and real accountability questions show up. That is usually where the useful answers start. Or at least where the uncomfortable ones do. (Accordion dropdown close) Transcript > (Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Connect with Holly Sandberg | LinkedIn Leader in fraud operations, Americas Advisory Board at MRC Former Director of Trust and Safety at Reverb Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: False Positives Masterclass Part 3: Reducing false positives inside your system Episode Title: False Positives Masterclass Part 3: How to reduce FPs inside your system Meta Title: Reducing False Positives Without Opening Fraud Risk Meta Description: Reducing false positives starts with manual review, cleaner fraud rules, better exclusions, shadow testing, and fixing data quality issues first... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/reducing-false-positives Keywords Primary Keyword: reducing false positives Secondary Keywords false positive rate fraud rules fraud rule tuning manual review fraud detection rules fraud model precision fraud model recall false positives in fraud detection fraud prevention logic shadow mode testing data quality issues manual review fraud operations reduce false positives without increasing fraud

When people talk about reducing false positives, they usually jump straight to tactics: tuning rules, adjusting thresholds, adding exemptions, redirecting edge cases to manual review, and so on. All of that is important, but it shouldn't be where you start. In the previous parts of this masterclass, we've covered the two pieces of work that most teams skip: how to measure your false positives, and how to map them into buckets by actor, by solution, by flow, by data quality, and by what is or isn't under your control. That's the groundwork. Now you're ready to tackle the main course, fixing the parts of the system that are misbehaving. But this is where we should approach the work with discipline. Otherwise, you'll end up making changes that don't matter, overlook changes that would have mattered, or worst of all, open the door for additional fraud without realizing it. So let's get down to it.

One of the best ways to ground yourself before making any change is to go back to manual review. Pick a rule, a model, or an AI agent that creates a high number of false positives, and manually inspect a sample of the events it blocked. 50 to 100 should be enough in most cases. What you're trying to answer are often very basic questions. Is this rule actually as inaccurate as we think? Do we still want this rule to exist at all? If it exists, should it still make automated decisions, or should it fit into manual review instead? And if it should remain automated, what would it take to reduce its false positives without materially increasing our fraud losses? Each of these questions is business specific. There is no universal threshold for good enough. Sometimes your fraud ops team can take on the additional workload. Sometimes you have no manual reviews at all as part of your process. And sometimes a rule needs to remain automated, even if it's noisy, because you simply cannot afford to manually review that volume. But you need clarity before you change anything. The worst case is thinking a rule is fine because it looks logical, only to discover under review that it's blocking overwhelmingly legitimate traffic.

You can't reason your way through these situations, you have to look. But most importantly, this isn't about validating that you're fixing a real problem. It's about discovering how to do it.

Once you've reviewed the evidence, almost every misbehaving solution falls into one of three categories. The first is that the rule shouldn't exist at all. This happens more often than teams admit. Many times it would catch only a small population, would not block much fraud, but would generate a disproportionate amount of false positives. Usually it exists only because someone added it during a crisis, and no one ever bothered to revisit it since then. Removing such rules feels uncomfortable the first few times you do it, but if the fraud they catch is negligible and the false positives are significant, turning them off is one of the cleanest, safest ways to improve your system. So this category has your low-hanging fruits. The second category is of rules that should exist, but no longer make automated decisions. This is common with mid accuracy logic that still captures meaningful fraud, but not reliably enough to block without human review. If you have manual review capacity, and if this particular logic tends to produce events your analysts are comfortable judging, then flagging it to case management instead of auto decline can be the perfect compromise. You preserve fraud coverage, reduce false positives, and put a human in the loop that can observe pattern changes in real time. The downside, of course, is that it's resource intense. You have fewer false positives, but at higher operational costs. And to be honest, I try as much as possible not to resort to it. The third category is of rules that should exist, but they need to be improved first.

This is the most interesting scenario, and it's also the most common one. In this case, the logic captures fraud effectively, meaning the recall is high, but at the cost of impacting many good users, meaning the precision is low. So you need to refine it. The question is how?

The process for improving a false positive heavy rule is exactly the same as when you design a new fraud rule, just in reverse. Think about it. When you design a fraud rule, you review a set of confirmed fraud cases, identify the pattern they share, compare that fraud pattern to the general good population, and build logic that captures the fraud without catching too many good events. Now, to reduce false positives, you do exactly the same thing, but with the opposite goal in mind. You review a set of confirmed false positive cases, identify the repeating patterns, compare those patterns to the fraud cases to make sure you know how to separate them, and finally build exclusions that capture that separation without releasing too much fraud. And this last step is crucial. It's not enough to say a lot of false positives look like X. You need to prove that your fraud cases don't also look the same. Otherwise, your exclusion will undermine your fraud detection. But here's the thing: if you've worked through the steps I outlined without skipping any, you already did this manual review while validating the false positive metrics. Let's take an example. Suppose you have a very standard mismatch rule: decline if IP country mismatch account country. Obviously, given a simplified scenario, but keep in mind that in reality, many rules are not that much more sophisticated than that. Now imagine you manually review a sample of 100 events this rule blocked. After tagging them, you notice 30 were fraud, 70 were legitimate, and out of those 70 legitimate events, 20 involved US users whose IP addresses appeared in Canada. 20 out of 70 is a meaningful pattern. Maybe your business has a large US-Canadian commuting demographic. Maybe VPNs for Canadian endpoints are common. Maybe part of your user base works close to the border. For whatever reason, this specific mismatch, US account and Canadian IP, seems to produce a lot of false positives. If in your fraud cases you see little to no fraud coming from the same US-Canada pattern,

Then you have a clean separation. That means you can safely introduce an exclusion. Decline if IP country mismatch account country and not IP country equals Canada and account country

Equals US. Or better yet, broaden it slightly to include general neighboring country logic, depending on your business footprint and available features. The point is that the exclusion is derived from reviewing your false positive population, and is validated against your fraud population. This is how you refine rules without accidentally creating backdoors for fraudsters to exploit.

Anytime you change a fraud logic, even if it's just an exclusion on an existing rule, you're changing your posture. That's the same as releasing a new rule completely from scratch. So before you release the new version, test it. Shadow mode and challenger rules are your best friends here.

Keep the existing rule active, but run your refined rule in parallel. Measure the differences. How many additional events would it have allowed to go through? How many of those events matured into fraud? How many false positives would it have prevented? If the results hold over time, you can deploy with confidence. If not, adjust. How much time should you monitor these changes? Ideally, enough to gain a good measure of how fast fraud matures in the new version, and whether that's in line or not with the original version. And if you're really pressed for time, you may want to consider manually reviewing random samples.

As we explored in part two of the masterclass, sometimes the root cause of false positives is not faulty logic, but corrupted data. Best case, you've already identified it in your groundwork analysis. Worst case, you went through all of the motions just to figure it out on your detailed manual review. When this is the identified root cause, you have two options, depending on whether the data issue is fixable in the near term. Option one is to fix the data quality issue itself. If the corrupted field is under your control, be it your SDK, your internal integration, or your infrastructure, then the best long-term solution is to repair the data. That might mean fixing an API payload, adjusting an SDK version, or working with a product team to plug a hole. It may take a few days or a few weeks, depending on your organization. But once the data is fixed, the rule will often start behaving correctly again. Also, it's likely that you also solved a whole bunch of other issues that were plaguing other solutions at the same time. Your second option is to adjust the fraud logic for that flow. Not all data issues are internally fixable. Sometimes they originate from external parties that you just cannot control. Or, in the most frustrating cases, you do control these issues, but there's no fix in sight. If you cannot fix the underlying data soon, then you may need to adjust your fraud logic specifically for that flow. You have a couple of choices here. Either exclude the problematic flow from the rule altogether, lower the rule's weight or change its impact, or move those cases to manual review.

Whatever you do, just keep in mind that fraud prevention should be more pragmatic than elegant. And sometimes it's not about being right, it's about being smart. YouTube Description False Positives Masterclass Part 3: How to reduce FPs inside your system Release Date Episode #12 Guest: Okay, so here is the thing about reducing false positives. Most teams want to jump straight into tactics. Tune the rule. Adjust the threshold. Add an exemption. Move the weird edge cases into manual review. Fine. All of that might be useful. But honestly, if that is where you start, you are probably guessing. And guessing in fraud prevention is not exactly my favorite operating model. Not because it never works. Sometimes it does. Which is almost worse, because then everyone gets confident. Not a good look. In this episode, I continue the False Positives Masterclass by moving from measurement and bucketing into the part everyone actually wants to get to: fixing the parts of the system that are misbehaving. But the point is not just to reduce false positives. The point is to reduce false positives without creating a new fraud problem you only discover three weeks later when the losses mature and everyone starts quietly looking at the dashboard like it personally betrayed them. This episode is about discipline. It is about manual review, fraud rules, fraud model precision, fraud model recall, shadow mode testing, data quality issues, and the uncomfortable but necessary question every fraud team eventually has to ask: is this rule actually helping, or have we just been emotionally attached to it since that one fraud spike in 2022? What you'll hear in this episode: ● Why reducing false positives should start with manual review, not instinct ● How to decide whether a fraud rule should be removed, downgraded, or improved ● Why fraud model precision and fraud model recall matter when rules catch fraud but hurt good users ● How to build exclusions without accidentally creating a back door for fraudsters ● Why shadow mode testing and challenger rules are essential before release ● How data quality issues can make otherwise reasonable fraud prevention logic misbehave ● Why fraud operations teams need to be pragmatic, not elegant, when the data is broken You should listen to this episode if you: ● Own fraud rules, fraud detection rules, models, AI agents, or review flows ● Are trying to reduce false positives without increasing fraud losses ● Have a high false positive rate but are not sure which part of the system is causing it ● Need a more structured way to review manual review samples and top offenders ● Are dealing with corrupted data, noisy signals, or flows where fraud prevention logic keeps misfiring Episode notes & key takeaways: (Accordion dropdown) Reducing false positives starts with looking, not guessing The first uncomfortable point in this episode is also the simplest one: you cannot reason your way through a false positive problem from a dashboard alone. You have to look. Manually. At real events. I know, very glamorous. Everyone got into fraud operations so they could inspect 100 blocked cases and ask themselves why a rule exists. Honestly, though, this is where the useful work starts. If a rule, model, or AI agent is producing a high number of false positives, the first step is to manually inspect a sample of what it blocked. In most cases, 50 to 100 events is enough to get directional clarity. You are not trying to create a dissertation. You are trying to answer practical questions: ● Is this rule actually as inaccurate as we think? ● Does this rule still deserve to exist? ● Should it still make automated decisions? ● Could it move into manual review instead? ● What would reduce false positives without materially increasing fraud losses? The key is that every answer depends on your business. There is no universal "good enough" threshold. Some teams have manual review capacity. Some do not have manual review at all. Some rules are noisy but still necessary because the volume is too large to send to analysts. Annoying? Yes. But also reality. Three paths for every misbehaving fraud rule Once you review the evidence, most misbehaving fraud rules or models fall into one of three categories. This is where the work gets cleaner. The first category is rules that should not exist at all. These are the low-hanging fruit. Usually they were created during a crisis, caught something once, and then stayed in production forever because nobody wanted to touch them. Very human. Very common. Also, not great. If the fraud coverage is negligible and the false positives are significant, removing the rule may be the safest and cleanest fix. The second category is rules that should exist, but should not make automated decisions anymore. These are mid-accuracy rules. They catch meaningful fraud, but they do not do it reliably enough to justify auto-decline. If your fraud operations team has capacity, moving that logic into case management can preserve fraud coverage while reducing false positives. You put a human in the loop, and that human can spot pattern changes in real time. The tradeoff is cost. Manual review is useful, but it is not free. You reduce false positives, but you increase operational workload. So you've got to ask yourself whether the compromise is actually worth it. The third category is the most common and most interesting: rules that should exist, but need to be improved. These rules often have high recall and low precision. In plain English, they catch fraud, but they also hit too many good users. So the answer is not to delete them. The answer is to refine them. False positive reduction is fraud detection in reverse One of the more useful ways to think about reducing false positives is that it is basically fraud detection in reverse. When you build a fraud rule, you usually start with confirmed fraud. You review the fraud cases, identify the pattern they share, compare that pattern against the good population, and then build logic that catches fraud without catching too many legitimate events. To reduce false positives, you do the same thing with the opposite goal. You start with confirmed false positives, identify the patterns they share, compare those patterns against the fraud cases, and then build exclusions that separate good users from fraud without releasing too much risk. This is where teams sometimes get into trouble. It is not enough to say, "A lot of false positives look like X." Okay, useful. But do fraud cases also look like X? If yes, your exclusion is not an exclusion. It is a fraud invitation with better formatting. That last step matters. A lot. How to build exclusions without creating fraud back doors The example in this episode is intentionally simple: decline if IP country does not match account country. It is a basic mismatch rule, and yes, real-world rules are often more complex. But not always as much as we like to pretend. Suppose you manually review 100 events blocked by this rule. Thirty are fraud. Seventy are legitimate. Of those 70 legitimate events, 20 involve U.S. users whose IP addresses appear in Canada. That is a meaningful false positive pattern. Now the next question is the important one: do your fraud cases also show that U.S.-account, Canadian-IP pattern? If not, you may have a clean separation. Maybe you have commuters. Maybe VPNs route through Canadian endpoints. Maybe your user base lives near the border. Whatever the cause, the false positive pattern is real, and the fraud overlap is low. That gives you the basis for a safer exclusion: Decline if IP country does not match account country, unless IP country is Canada and account country is the U.S. Or, depending on your business footprint and available features, maybe you broaden that into neighboring-country logic. The point is not the exact rule. The point is the method. The exclusion comes from the false positive population and is validated against the fraud population. That is how you reduce false positives without accidentally creating a new entry point for fraudsters. Am I being too optimistic that everyone does this step? Probably. Test every fraud logic change before release Any time you change fraud prevention logic, even if it is "just" an exclusion, you are changing your risk posture. Treat it like a new rule release. Shadow mode testing and challenger rules are your best friends here. Keep the current rule active, but run the refined version in parallel. Then measure the difference. You want to know: ● How many additional events would the new version allow? ● How many of those events eventually mature into fraud? ● How many false positives would the change prevent? ● Does the new logic behave consistently over time? If the results hold, you can deploy with more confidence. If they do not, adjust. It is not glamorous. It is not a big dramatic launch moment. It is just better fraud operations. The monitoring window should depend on how quickly fraud matures in your environment. If you have time, wait long enough to compare fraud outcomes in the challenger version against the original. If you are under pressure, manually review random samples. Not perfect, but better than flying blind. Data quality issues can make good logic look bad Sometimes the problem is not the rule. It is the data feeding the rule. This is frustrating because everyone wants the fix to be in the fraud logic. Change the rule, lower the score, add the exception, ship it. But if the underlying field is corrupted, missing, inconsistent, or broken in a specific flow, the rule may be behaving exactly as designed. The input is just bad. Okay. Annoying, but useful to know. If the data issue is under your control, the best long-term answer is to fix the data quality issue itself. That might mean repairing an API payload, updating an SDK version, fixing an internal integration, or working with product and engineering to close the gap. It may take days. It may take weeks. But once the data is fixed, the fraud rule may start behaving correctly again. Also, you may fix other problems at the same time. Data issues rarely ruin just one thing. They usually wander around the system quietly making several things worse. Very efficient, in the worst possible way. When the data cannot be fixed quickly, adjust the logic Not every data quality issue is fixable in the near term. Sometimes it comes from an external partner. Sometimes it comes from a platform you do not control. Sometimes it comes from a system you technically do control, but the fix is sitting somewhere behind 14 roadmap priorities and one team that keeps saying "next quarter." Not a good look, but familiar. If you cannot fix the data soon, adjust the fraud logic for that flow. That might mean excluding the problematic flow from the rule, lowering the rule's weight, changing the rule's impact, or moving those cases into manual review. This is where fraud prevention needs to be pragmatic rather than elegant. A clean system is nice. A working system is better. Sometimes the right answer is not the most theoretically satisfying one. Sometimes it is the one that keeps good users moving while still containing fraud risk. Final takeaway: Reducing false positives is not about making your system softer. It is about making it more precise. That means reviewing the actual events, separating rules that should be removed from rules that should be downgraded or improved, building exclusions based on evidence, validating those exclusions against fraud cases, testing changes in shadow mode, and fixing data quality issues when the logic is not really the problem. Anyway, the slightly uncomfortable takeaway is this: if you are reducing false positives without looking at the underlying cases, you are not tuning. You are guessing. And maybe you get lucky. Maybe. But in fraud prevention, luck is not really a control. It is more of a temporary condition. (Accordion dropdown close) Transcript > ( Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. This is part of a larger series, so if you landed here first, you may want to go back and listen to the previous episodes. We've already covered quite a lot that will make this one much easier to follow, so get caught up here: False Positives Masterclass: How to measure FPs in systems that hide them False Positives Masterclass Pt. 2, Turning data into intelligence Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: Ad Tech in Financial Fraud: Lessons from AI-Powered Fraud Fighting Episode Title: What AdTech Taught Me About Financial Fraud, with Gilit Saporta Meta Title: Ad Tech in Financial Fraud: What Teams Can Learn Meta Description: Ad tech in financial fraud exposes new lessons for fraud teams using AI, bot detection, device hijacking insights, and stronger cross-industry response... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/ad-tech-in-financial-fraud Keywords Primary Keyword: ad tech in financial fraud Secondary Keywords: Financial fraud prevention AI fraud prevention Bot detection Advertising fraud AI-powered fraud fighting Fraud detection Fraud prevention strategies AI fraud detection Fraud analytics Fraud investigation Automated fraud detection Fraud risk management Ad fraud detection Fraud prevention technology What AdTech Taught Me About Financial Fraud, with Gilit Saporta July 4, 2026 Episode #11 Guest: Gilit Saporta In this episode of The Saturday Fraud Strategist, I talk with Gilit Saporta about ad tech in financial fraud, which sounds very niche until you realize it touches malware, fake traffic, bot detection, consumer abuse, advertising fraud, and a lot of systems quietly pretending they have this handled. Ad tech fraud is not just "someone clicked a fake ad." That would almost be simple. What we're really talking about is a whole ecosystem where fraudsters monetize traffic, hijack devices, manipulate advertising spend, and sometimes pull regular people into the mess without them even knowing it happened. Gilit brings nearly two decades of fraud-fighting experience across financial services, crypto, e-commerce, and digital advertising, so the conversation gets practical pretty quickly. We look at what makes ad tech in financial fraud different from traditional fraud, where fraud detection is improving, and where systems still fall apart because the context is missing. And honestly, that's the part that keeps coming up. You can have AI fraud prevention. You can have automated fraud detection. You can have dashboards that look very impressive in a board meeting. But if the system can't tell the difference between a weird but legitimate pattern and an actual fraud signal, now you've got a problem. Maybe a very expensive problem. What you'll hear in this episode: ● A step-by-step breakdown of how ad tech fraud works and why it does not behave like traditional financial fraud ● A practical look at what modern fraud detection is getting right and where it still struggles ● Why institutions, platforms, and technology teams need to stop treating fraud risk management as someone else's cleanup job ● A conversation about AI-powered fraud fighting, automation, accountability, and the uncomfortable gap between speed and judgment ● How consumers get pulled into device hijacking, malware, scams, fake apps, and fraudulent advertising ecosystems ● Why better financial fraud prevention depends on context, not just bigger models or faster alerts ● A discussion about collaboration between fraud teams, trust and safety, researchers, regulators, and the organizations sitting on all the useful data Who should listen: ● Fraud professionals and financial institution leaders ● Risk, compliance, and cybersecurity teams ● AI, machine learning, and fraud analytics practitioners ● Trust and safety teams ● Ad tech, e-commerce, and digital platform leaders ● Regulators, policy advisors, and industry advocates ● Anyone trying to understand why fraud keeps finding the cracks between systems This episode is for people who care about fraud prevention strategies beyond the press release version. Detection matters. Compliance matters. But you've got to ask yourself, are we actually preventing harm, or are we just getting better at labeling it after the fact? Episode notes: (Accordion dropdown) Ad tech in financial fraud has become a very convenient playground for fraudsters. You have money moving through complex advertising systems, traffic that can be faked or manipulated, devices that can be hijacked, and consumers who often have no idea they were part of the operation. That's already messy. Now add AI, automation, bots, fake apps, malware, and advertising networks that were not exactly built for perfect transparency, and the whole thing starts to look less like a simple fraud problem and more like an ecosystem problem. Gilit and I talk through how organizations are using AI fraud detection, bot detection, advanced fraud analytics, and intelligence sharing to improve prevention. And to be fair, some of this is real progress. I am not here just to complain into the microphone. Mostly. But there are still major gaps. AI systems can move fast, but they can also jump to conclusions. They can detect patterns, but they do not always understand why those patterns matter. They can flag anomalies, but without strong fraud investigation workflows and human judgment, you may just end up with very confident confusion. We also get into the difference between malicious automation, legitimate automation, and wasteful automation. That distinction matters more than people think. If every bot is treated the same, or every strange behavior is treated as fraud, teams create noise. If they ignore the automation problem completely, fraudsters get a very nice invitation. Not a good look. The consumer impact is a major part of this conversation. Device hijacking, malware, fraudulent applications, scam-driven advertising, and deceptive ecosystems often hit people who never expected to become victims. And when they do, shame and embarrassment can keep them from reporting it. Behind the bot traffic, fake installs, automated clicks, and suspicious patterns, there is usually a person somewhere who got exploited, manipulated, or quietly harmed by a system that did not catch the problem early enough. We also talk about the upcoming Fraud Fighters AI Playbook and what responsible AI adoption should actually look like for fraud teams. Not the shiny version. The operational version. The version where teams have to think about governance, data quality, accountability, fraud prevention technology, investigation workflows, and whether their tools are helping people make better decisions or just giving them more alerts to triage. Key takeaways: ● Awareness helps, but awareness is not prevention. ● Fraud detection needs technology, context, and human judgment working together. ● AI-powered fraud fighting is only useful when it is supported by strong investigative frameworks. ● Ad tech in financial fraud creates risk across advertising fraud, identity abuse, malware, consumer protection, and financial services. ● Organizations need better ways to separate malicious automation from legitimate or simply wasteful automation. ● Ad fraud detection cannot rely only on faster alerts. It needs better context, better data, and better operational judgment. ● Fraud teams, trust and safety teams, researchers, regulators, and platforms need to collaborate more directly. ● The human cost of fraud should not be treated as a side note. The bigger question is whether organizations are building systems that actually reduce harm, or just systems that explain the harm after it has already happened. The alternative is accepting fragmented systems, weak accountability, and fraud ecosystems that keep scaling faster than the defenses around them. That's not a great plan. (Accordion dropdown close) Transcript > (Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. To go deeper, get your copy of The Fraud Fighter's AI Playbook by Gilit Saporta, Chen Zamir, and Shoshana Maraney on O'Reilly: https://www.oreilly.com/library/view/the-fraud-fighters/9798341660359/ Connect with Gilit Saporta | LinkedIn VP, Product Management - Fraud Fighting at DoubleVerify Author at O'Reilly Publishing Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook"

All right. Welcome back, everybody, for another episode of The Saturday Fraud Strategist. And with me today, a very special guest, one that has so much experience in the field of fraud, fraud prevention, and one of the most experienced fraud fighters that I've got to meet throughout my career. I also had the privilege in the last year to be her co-author, writing our upcoming book. We'll talk about it a bit later. Gilit Saporta, it's such a pleasure to have you on the show.

It's such a pleasure to be here, Chen. Thank you so much for inviting me. We just realized that you and I go way back, to the point that you still have me saved with my maiden name on your contact list. That's so cool.

Yeah, I met Gilit on my first day doing fraud prevention. Well, likely the first day. So that means that we know each other for nearly 17 years now. It's a bit scary to say that, but yeah. Gilit, you've been around, and I'm sure that a lot of the listeners who are listening right now know you either personally, through a conference, or through a podcast. Can you share a bit about who you are and tell us a bit about your career so far, just so we're all on equal ground?

So yeah, I've been around, and I was one of the lucky ones to be part of the early days of e-commerce and e-commerce financial crime. Our first opportunity to work together, you and I, was with the reincarnation of my first student job, doing some manual review of credit card payments, mostly for people who were paying for VPN services. Even the ones who were doing so without a stolen credit card back then were probably buying the services to do something nefarious or naughty later on.

Not to watch the newest Netflix release.

Who knows, who knows? No, Netflix was still a service for shipping DVDs to your home back then. That's how old we are. Do you realize that? So yeah, it was a fun ride and an amazing opportunity to learn about how diverse and creative the human mind can be when it comes to both fraudulent manipulations and just bizarre behaviors that are not fraudulent online. I think that, in a nutshell, characterizes what I do and what I love doing throughout my career. I moved from financial fraud, where you and I met, with PayPal. After nearly a decade with Fraud Sciences, which got acquired by PayPal, I moved into e-commerce with Forter, where I got to be part of the fairly early days of that amazing company. Then I moved into cryptocurrency fraud with Simplex, which got acquired by Nuvei. During the first COVID lockdown in 2020, I moved into DoubleVerify, where I work today as VP of Product for our wonderful Fraud Lab. That gave me the chance to experience both the most sophisticated, targeted version of crypto fraud, where you're targeting one victim in a very spear-phishing kind of way, all the way to bot attacks with DoubleVerify, where we identify massive volumes of events with anomalies in order to help uncover fraud attacks.

Yeah, that's amazing. The reason why I was so keen to have you on the show is because you've had such a fascinating journey. When I look at it, I see three things. First, you've spent almost two decades on cutting-edge teams, building cutting-edge technologies. And it's not just one team. You've seen several problems from several different angles over a very extended period of time, always on the forefront, dealing with frontier problems and frontier solutions. Second, you've transitioned from financial fraud and e-commerce fraud into ad tech fraud, which is a space I don't know much about. And third, and this is probably the main reason why I wanted to have you on the show, you've had roles that involved training. That positioned you in a place where you had to think about abstractions, methodologies, and the theory behind fraud and fraud prevention. I thought, who better to talk to about fraud in general, and the differences between financial fraud prevention and ad tech fraud, than Gilit? So I'm super excited to dive into it. Let me start with the first thing that came to mind. I'm not even sure I know what ad tech fraud is. I can imagine, but I can also imagine there are several major typologies. Maybe I know some of them, maybe I don't. Can we start by discussing that and how it's different from financial fraud?

Yeah, for sure. I think you and I both share this passion for abstraction, methodologies, and training. It's a great way to help teams and the industry think about the next potential attack and threat we should be looking out for. In ad tech, we define fraud as any kind of manipulation that's out there to monetize and drain the advertising budgets of brands. The largest brands in the world are our clients today, which means I get the fun and very stressful experience of having to explain fraud attacks to brands like Disney and many others. There's a huge sensitivity not just to the financial impact, but also to brand reputation, brand suitability, and brand safety. What does it mean for a large organization to be perceived as monetizing fraud? Or supporting fraud? Or allowing children to be exposed to content they shouldn't be exposed to because some manipulation in the advertising pipeline led them there? We see that all the time. The classic example I provide when I explain what I do is how advertising fuels our digital lives. If you look at the mobile device we all have next to us, there is a lot of technology running both in ways we see and under the hood. Once I became more aware of it through our Fraud Lab, I realized that almost every device has some app running code that may be generating ad calls, whether you see them or not. My dad experienced this all the time. No matter how often I told him not to click suspicious links, he would install all sorts of junk apps. Flashlight apps, for example, even though the phone already has a flashlight. A few days later he'd notice the device heating up, the battery draining quickly, or his data plan disappearing. It was malware producing invisible ads that brands were paying for. That's one example of an ad tech fraud typology. A device hijacking attack. Someone is tricked into installing malware. Advertisers pay substantial amounts to run ads on that device around the clock, and nobody is aware of it. These are the kinds of operations I feel proud about uncovering at scale and helping the industry address, especially when there's a human victim involved. Part of my passion for education, and I think yours as well, comes from the fact that we've seen so many fraud attacks that are technically sophisticated but could have been prevented through public awareness. People shouldn't feel ashamed of being victims. My dad often noticed something was wrong, but he felt embarrassed. We see that in romance scams, ransomware, and countless other fraud schemes. We need to remove the shame element so we can all be safer.

It's interesting because I expected you to talk about affiliate fraud and things like that. What I didn't expect was how much ad tech fraud resembles scams. There are elements of social engineering, cybersecurity, malware, and classical fraud prevention, because ultimately someone needs to monetize and move the money. That's super interesting. Going back to 2020 and your move into DoubleVerify, I'd assume you came in with a tremendous amount of experience compared to many team members, even if not in the ad tech domain itself. What parts of your experience translated directly into your new role? What frameworks or processes were you able to lift from financial fraud and apply on day one?

That's such a great question. First, I was completely overwhelmed by the volume of data and expertise I needed to absorb. Even six years later, I'm still amazed by the researchers here. Their understanding, creativity, and mindset are incredible. It felt like walking into Hogwarts, and I definitely wasn't the wizard in that scenario. The first thing we heavily invested in was collaboration, both within the Fraud Lab and across the broader organization. Fraud fighters naturally gravitate toward each other. That's one reason fraud meetups are so enjoyable. We love sharing analytical riddles and discussing attacks. But if you're talking to sales, commercial teams, or even engineering teams, you can quickly lose people. It's important to connect fraud prevention to human outcomes, like my dad's hijacked phone, but also to business outcomes. What's the financial impact? What does this do to the bottom line? Should you accept more risk to gain more scale? That need for cross-organizational collaboration became a major focus. In fact, Shoshana and I, who is also our co-author on the upcoming book, created a video series on FraudLab.com about translating fraud fighter language into business language that executives can understand.

And you think that's easier in financial services because the connection to the bottom line is more obvious?

Yes. It's easier when you're in a risk organization that directly measures chargebacks, clawbacks, and financial losses. It's harder when you're in product organizations or ecosystems where scale and bot attacks create more indirect impacts. The second thing that transferred directly was the good story, bad story approach. Whenever we see an anomaly, we ask, what's the most plausible legitimate explanation? And what's the most plausible fraudulent explanation? The ability to ask why is something we still practice every day. In fact, it's one of the biggest challenges for AI agents. Our AI systems tend to jump to conclusions. If they see a traffic spike at 3:00 a.m., they immediately say, "Bot attack." A human analyst would ask, "What if Taylor Swift tickets just went on sale and everyone hit the site at once?" The challenge is teaching AI to consistently generate reasonable human explanations before concluding fraud. You need to hold both possibilities simultaneously.

I completely agree. LLMs are incredibly quick to find conspiratorial explanations. I want to double-click on the good story, bad story approach. You're describing it in the context of individual cases, but you work in a Fraud Lab. You're not investigating one case at a time. How does that framework translate into scalable, analytical work?

Great question. Our Fraud Lab is somewhat unique, because we've automated a lot of the operational work. That allows us to focus on edge case analytics. You still need automation to process huge volumes of traffic. Whether that's rule engines, machine learning, or AI systems, something has to filter the majority of activity. But I don't believe in one model to rule them all. The attacks targeting senior citizens are different from those targeting gamers, which are different from attacks targeting sports audiences. The patterns vary dramatically. That's why I love the good story, bad story approach. Take a TV generating ads 24 hours a day. One story is that it's a bot attack. Another story is that it's a legitimate display at an airport showing sports programming around the clock. Both explanations fit the data. You need systems capable of distinguishing between them. You can't just say, "24 hour traffic equals fraud." That won't work.

How do you actually do that at scale?

That's exactly why we need excellent people building these systems. We're like kids in a candy store right now. The challenge isn't access to AI. It's orchestration. We've found success with specialized skills. For example, we have an investigate skill trained to tell good and bad stories. That skill feeds into a detect skill that looks for anomalies and compares them against the outputs of the investigate skill. It's like building with LEGO pieces. You create smaller components and connect them together. One of the motivations behind our book was to show fraud fighters that they can now mix and match capabilities much more easily than before. Even if you're moving from fintech into ad tech, AI can help accelerate the learning process.

That's fascinating. We spent years talking about these theories, and suddenly we have tools that can operationalize them. Going back to your first weeks at DoubleVerify, what surprised you? What felt truly novel?

I was surprised by the murkiness of the industry. Ad tech is a B2B ecosystem where everyone knows everyone. That was very different from fintech, where many adversaries were anonymous criminals or organized crime groups. In ad tech, you sometimes have companies operating in gray areas. They're not necessarily criminals, but they're serving ads in ways that provide little or no value to advertisers. The industry relies heavily on trust and transparency. That led me to appreciate collaboration in ways I hadn't expected. I never imagined I'd spend part of my job working directly with other fraud labs, such as Roku's, to bring down device hijacking schemes together. That level of cooperation surprised me. It also required learning about legal considerations, business sensitivities, and collaboration across organizations.

I don't think I've ever heard of two private companies collaborating to bring down a fraud ring in financial services. How does that work?

The regulator plays a major role. The Media Rating Council and organizations like TAG, the Trustworthy Accountability Group, invest heavily in collaboration and education. The goal is to maintain trust in the ecosystem and support a free internet supported by advertising. These groups provide resources, events, and frameworks that help the industry work together. Ad tech is younger than fintech, but in terms of collaboration, I think it's actually more mature.

That's amazing. If you returned to financial services tomorrow, what lesson would you bring from ad tech?

One major lesson is that bot does not automatically mean fraud. Bots can be shopping assistants, agents, and other forms of automation working on behalf of humans. The challenge is distinguishing malicious bots from legitimate bots. That's something ad tech has spent years refining. We've tried to share many of those ideas in the book.

Can you give an example?

It becomes almost philosophical. Intent is difficult to measure. We often focus on impact instead. For example, websites today are constantly scraped by bots. Sometimes those bots belong to major AI companies collecting data to train models. Sometimes they're projects built by students. The intent isn't necessarily malicious. But the impact can still be costly. I know of a large e-commerce website that has been scraped continuously for years. It may have started as a school project someone forgot to turn off. But it's still consuming resources and generating costs. That's where I think we all need to become more responsible. Not all bots are malicious, but some are wasteful. And AI has dramatically increased the scale of that problem.

That's really interesting. It sounds like we now have a new category that's neither good nor bad, just careless or wasteful.

Exactly. And the challenge is scale. One example we discuss in the book involves AI-generated content farms. Some are promoting dropshipping schemes. Others are more dangerous and lead users toward ransomware or other scams. The sophistication isn't always impressive. What's impressive is the volume. That's the challenge we need to solve as an industry.

I'm glad you brought up the book. We started writing it about a year ago, and everything has changed since then. The technology, the attacks, the defenses, our understanding of the space. If we started writing it again today, it would be a different book. What made you want to take on such a challenging project?

Terrible FOMO. That's the honest answer. Fortunately, O'Reilly has allowed us to keep updating and refining the content until the very last minute. It's daunting to write about a topic moving this quickly. What helped was that you constantly brought us back to frameworks and methodology. I kept showing up with new attacks every week saying, "Put this in the book." You kept saying, "Let's generalize." That balance helped create something more durable. My hope is that readers walk away feeling more confident discussing AI and fraud, regardless of their starting point.

I'm curious to see how quickly it ages. One question I ask all guests is this: leaders know they need to adopt AI. They're being pushed to do it. Where should they start?

If your organization is encouraging AI adoption, start by understanding what technology is already integrated into your enterprise environment. What systems already have access to your data? What tools are approved? Our team experimented with everything early on. That was fun, but we also built a lot of things that weren't scalable. Eventually, we rebuilt many of them into workflows where AI could participate across the entire process. From documenting findings in Jira, to accessing data sources, to routing insights to the right business stakeholders. That level of automation requires maturity. A year ago, most organizations didn't have that infrastructure. Today, many do. The key is starting with your existing ecosystem.

That's such a good point. Many organizations still think AI is magic. They assume it's plug and play. But like any data-driven technology, it depends on the quality and accessibility of your data.

And it will lie to your face shamelessly. It will confidently generate information that isn't there. You have to be very careful.

Exactly. Gilit, this has been so much fun. Let me summarize some of the key takeaways. Fraud teams often struggle to connect their work directly to business outcomes, even though that connection is critical. We discussed the good story, bad story approach, and how AI finally gives us tools to scale that way of thinking. We talked about the importance of granularity rather than trying to build one model that does everything. We discussed how bot detection is no longer about identifying all bots as bad, but rather distinguishing between good, bad, and sometimes simply wasteful automation. And finally, we talked about AI adoption and the importance of getting your data in order before expecting meaningful results.

Yeah, it's an inherent challenge. If you're doing a great job in fraud prevention, you're often invisible. It's a bit like parenting teenagers. If everything is working, nobody notices.

That's a great description of the catch-22. Gilit, thank you so much for joining me. The Fraud Fighters AI Playbook is either out now or coming very soon. You may still be able to get a free copy from O'Reilly. The link is below. Thank you again for joining me today.

Thank you.

All the best, and I'll see you all next Saturday. YouTube Description What AdTech Taught Me About Financial Fraud w/ Gilit Saporta July 4, 2026 Episode #11 Guest: Gilit Saporta In this episode of The Saturday Fraud Strategist, I talk with Gilit Saporta about ad tech in financial fraud, which sounds very niche until you realize it touches malware, fake traffic, bot detection, consumer abuse, advertising fraud, and a lot of systems quietly pretending they have this handled. Ad tech fraud is not just "someone clicked a fake ad." That would almost be simple. What we're really talking about is a whole ecosystem where fraudsters monetize traffic, hijack devices, manipulate advertising spend, and sometimes pull regular people into the mess without them even knowing it happened. Gilit brings nearly two decades of fraud-fighting experience across financial services, crypto, e-commerce, and digital advertising, so the conversation gets practical pretty quickly. We look at what makes ad tech in financial fraud different from traditional fraud, where fraud detection is improving, and where systems still fall apart because the context is missing. And honestly, that's the part that keeps coming up. You can have AI fraud prevention. You can have automated fraud detection. You can have dashboards that look very impressive in a board meeting. But if the system can't tell the difference between a weird but legitimate pattern and an actual fraud signal, now you've got a problem. Maybe a very expensive problem. What you'll hear in this episode: ● A step-by-step breakdown of how ad tech fraud works and why it does not behave like traditional financial fraud ● A practical look at what modern fraud detection is getting right and where it still struggles ● Why institutions, platforms, and technology teams need to stop treating fraud risk management as someone else's cleanup job ● A conversation about AI-powered fraud fighting, automation, accountability, and the uncomfortable gap between speed and judgment ● How consumers get pulled into device hijacking, malware, scams, fake apps, and fraudulent advertising ecosystems ● Why better financial fraud prevention depends on context, not just bigger models or faster alerts ● A discussion about collaboration between fraud teams, trust and safety, researchers, regulators, and the organizations sitting on all the useful data Who should listen: ● Fraud professionals and financial institution leaders ● Risk, compliance, and cybersecurity teams ● AI, machine learning, and fraud analytics practitioners ● Trust and safety teams ● Ad tech, e-commerce, and digital platform leaders ● Regulators, policy advisors, and industry advocates ● Anyone trying to understand why fraud keeps finding the cracks between systems This episode is for people who care about fraud prevention strategies beyond the press release version. Detection matters. Compliance matters. But you've got to ask yourself, are we actually preventing harm, or are we just getting better at labeling it after the fact? Episode notes: (Accordion dropdown) Ad tech in financial fraud has become a very convenient playground for fraudsters. You have money moving through complex advertising systems, traffic that can be faked or manipulated, devices that can be hijacked, and consumers who often have no idea they were part of the operation. That's already messy. Now add AI, automation, bots, fake apps, malware, and advertising networks that were not exactly built for perfect transparency, and the whole thing starts to look less like a simple fraud problem and more like an ecosystem problem. Gilit and I talk through how organizations are using AI fraud detection, bot detection, advanced fraud analytics, and intelligence sharing to improve prevention. And to be fair, some of this is real progress. I am not here just to complain into the microphone. Mostly. But there are still major gaps. AI systems can move fast, but they can also jump to conclusions. They can detect patterns, but they do not always understand why those patterns matter. They can flag anomalies, but without strong fraud investigation workflows and human judgment, you may just end up with very confident confusion. We also get into the difference between malicious automation, legitimate automation, and wasteful automation. That distinction matters more than people think. If every bot is treated the same, or every strange behavior is treated as fraud, teams create noise. If they ignore the automation problem completely, fraudsters get a very nice invitation. Not a good look. The consumer impact is a major part of this conversation. Device hijacking, malware, fraudulent applications, scam-driven advertising, and deceptive ecosystems often hit people who never expected to become victims. And when they do, shame and embarrassment can keep them from reporting it. Behind the bot traffic, fake installs, automated clicks, and suspicious patterns, there is usually a person somewhere who got exploited, manipulated, or quietly harmed by a system that did not catch the problem early enough. We also talk about the upcoming Fraud Fighters AI Playbook and what responsible AI adoption should actually look like for fraud teams. Not the shiny version. The operational version. The version where teams have to think about governance, data quality, accountability, fraud prevention technology, investigation workflows, and whether their tools are helping people make better decisions or just giving them more alerts to triage. Key takeaways: ● Awareness helps, but awareness is not prevention. ● Fraud detection needs technology, context, and human judgment working together. ● AI-powered fraud fighting is only useful when it is supported by strong investigative frameworks. ● Ad tech in financial fraud creates risk across advertising fraud, identity abuse, malware, consumer protection, and financial services. ● Organizations need better ways to separate malicious automation from legitimate or simply wasteful automation. ● Ad fraud detection cannot rely only on faster alerts. It needs better context, better data, and better operational judgment. ● Fraud teams, trust and safety teams, researchers, regulators, and platforms need to collaborate more directly. ● The human cost of fraud should not be treated as a side note. The bigger question is whether organizations are building systems that actually reduce harm, or just systems that explain the harm after it has already happened. The alternative is accepting fragmented systems, weak accountability, and fraud ecosystems that keep scaling faster than the defenses around them. That's not a great plan. (Accordion dropdown close) Transcript > (Accordion dropdown) To go deeper, get your copy of The Fraud Fighter's AI Playbook by Gilit Saporta, Chen Zamir, and Shoshana Maraney on O'Reilly: https://www.oreilly.com/library/view/the-fraud-fighters/9798341660359/ Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. To go deeper, get your copy of The Fraud Fighter's AI Playbook by Gilit Saporta, Chen Zamir, and Shoshana Maraney on O'Reilly: https://www.oreilly.com/library/view/the-fraud-fighters/9798341660359/ Connect with Gilit Saporta | LinkedIn VP, Product Management - Fraud Fighting at DoubleVerify Author at O'Reilly Publishing Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: False Positive Fraud Detection Models, Masterclass Pt. 2 Episode Title: False Positives Masterclass Pt. 2, Turning data into intelligence Meta Title: False Positive Fraud Detection Models Meta Description: How to reduce false positives in fraud detection models by finding root causes, fixing data quality issues, and improving fraud decisioning... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/false-positive-fraud-detection-models Keywords Primary Keyword: false positive fraud detection models Secondary Keywords false positive reduction fraud detection models reducing false positives fraud model thresholds fraud decisioning fraud risk scoring fraud rules fraud system optimization fraud prevention systems manual review fraud analysts data quality issues corrupted fraud signals model drift root cause analysis payment fraud detection fraud operations False Positives Masterclass Pt. 2 Release Date Episode #10 One of the most common mistakes I see fraud teams make is attacking false positives head on. A customer complains. The CEO says the model is blocking too much. Someone opens a dashboard, adjusts a fraud model threshold, maybe tweaks a few fraud rules, and suddenly everyone feels like progress is happening. Honestly, not a good look. Not because false positive reduction is the wrong goal. It is absolutely the right goal. The problem is that most teams go tactical immediately. And if you are tactical about the way you reduce false positives, you should probably only expect tactical gains. In this episode, we get into the second part of the false positives masterclass: how to break down false positive fraud detection models into buckets you can actually prioritize and fix. We look at where false positives come from, which ones are driven by fraud detection models, fraud rules, manual review, upstream partners, fraud analysts, data quality issues, corrupted fraud signals, and payment fraud detection workflows. Quantifying false positives is useful. But it is not a plan. What you'll hear in this episode: ● Why reducing false positives requires root cause analysis, not just model tuning ● How to identify who actually declined the event: a rule, fraud model threshold, AI agent, fraud analyst, manual review team, issuer, acquirer, or fraud vendor ● Why upstream payment partners can create false positives your own fraud prevention systems cannot directly fix ● How fraud decisioning breaks down across payment fraud detection, fraud risk scoring, and operational workflows ● Why fraud system optimization starts with identifying the worst offenders ● How data quality issues, corrupted fraud signals, and model drift create false positives that look like fraud risk ● How fraud operations teams can prioritize the buckets that are large enough, fixable enough, and valuable enough to address first Who should listen: ● Fraud operations leaders trying to improve fraud detection accuracy ● Fraud analysts working through manual review queues ● Risk teams managing fraud rules, fraud model thresholds, and fraud risk scoring ● Data science teams responsible for fraud detection models and model drift ● Payment fraud detection teams dealing with issuer declines and upstream partner decisions ● Fraud prevention teams trying to reduce false positives without increasing losses ● Anyone who has ever stared at a false positive dashboard and thought, "Okay, now what?" Episode notes: (Accordion dropdown) A frustrating but familiar pattern Fraud teams often respond to false positives only after a customer complaint, executive escalation, or sudden concern that "the model is blocking too much." Okay, fair. But now you've got to ask yourself: are you solving the root cause, or just adjusting the part of the system that is easiest to see? That distinction matters. Modern fraud prevention systems give teams a lot of tools Fraud detection models, fraud rules, fraud risk scoring, manual review, AI agents, payment fraud detection systems, and fraud analysts all play a role in stopping suspicious activity. The same system that protects you can also block good users if you do not understand where each decision is actually coming from. The big gap is visibility A false positive might come from your own fraud rules. It might come from a conservative fraud model threshold. It might come from manual review. It might come from an issuer, acquirer, processor, fraud vendor, or another upstream partner. If you do not know who said "no," you cannot know what to fix. False positives are not just model errors Good customers get blocked. Fraud analysts waste time reviewing preventable cases. Manual review queues grow. Payment fraud detection teams chase the wrong problem. Fraud operations leaders struggle to explain why customer friction is rising. And somewhere in the middle of all that, a perfectly legitimate user is wondering why your system decided they looked suspicious. Not great. Data quality issues Sometimes the fraud logic is fine. The problem is that the system is operating on corrupted fraud signals. Maybe the IP field is wrong. Maybe device IDs are missing. Maybe payment metadata never passed through. Maybe a mobile SDK bug is making iOS traffic look strange. Suddenly, your fraud detection models drift. Your fraud rules misfire. Your fraud risk scoring looks worse than it is. And now the team is "optimizing the model" when the real problem is a broken input. Classic. The path forward The better path is to bucket false positives by actor, partner, user journey, product flow, platform, payment method, geography, and data issue. Prioritize the problems that are: ● Large enough to matter ● Actually under your control ● Not caused by data issues you cannot fix ● Connected to high-impact fraud rules, model thresholds, or manual review policies That is how fraud system optimization becomes strategic instead of reactive. Key takeaways: Reducing false positives is not just about fraud model thresholds, fraud rules, or manual review. It is about understanding the full fraud decisioning system, including partners, platforms, workflows, analysts, corrupted fraud signals, and data quality issues. Once you know where the problem comes from, you can finally decide what to fix first. And honestly, that is a much better place to be. (Accordion dropdown close) Transcript > (Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. This is part of a larger series, so if you landed here first, you may want to go back and listen to the previous episode. We've already covered quite a lot that will make this one much easier to follow, so get caught up here: False Positives Masterclass: How to measure FPs in systems that hide them Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook"

One of the common mistakes I see fraud teams make is attacking false positives head on. They'll get a customer complaint, and they'll now trace back why this false positive happened and how to make sure it doesn't happen again. Or the CEO complains that the model is blocking too much, and now they try to optimize it. The problem, of course, isn't the fact that they're trying to minimize their system's false positives. It's the fact that they're being tactical about it. And if you're tactical about how you do things, you can only expect tactical gains at best. In part one of the false positives masterclass, and if you missed that one, the link is down below, we accepted the reality that your system isn't built to report its own mistakes. We discussed several methods that can help you work around that and generate a usable picture of your false positives. Then comes the next problem. Quantifying false positives is merely an observation. It is not a plan. To reduce false positives in a meaningful way, you have to go one level deeper and ask a different set of questions. Where do they come from? Which are driven by my own system, and which by my partners? Which are actually data quality problems masquerading as fraud risk? And most importantly, where should I start? That's what the second part of the series is about, breaking down your false positives into buckets that you can prioritize and act on. How do you do that? Here's my seven-step process. The first and most important step is to attach every false positive to the actor that made a decision. When a transaction is declined or an onboarding attempt is blocked, someone or something said no. That someone might be a rule in your system, a machine learning model threshold, an AI agent, a human analyst, a manual reviewer, even a third-party partner, an issuer, acquirer, or fraud vendor. Without this categorization, you will default to optimizing the things you can see, usually your rules and your models. That's how teams spend months fine-tuning rules, only to later learn that most of their declines were coming from an issuer they never spoke to. So your first task is to map every decline to the actor that made that decision. Sometimes it will be a single rule. Sometimes it will be a decision based on a certain score threshold. Sometimes it will be a human decision. Sometimes it will be a response coming back from a payment partner. The point is that you don't have to do this perfectly on day one. Even a rough breakdown makes an enormous difference, because it will help you get a sense for where the most value lies. Once you have that basic map, you can ask the following question: how much of this is even under my control? This is particularly important in card payments, where a single card transaction can pass through half a dozen hands before the issuer finally says yes or no. Between the cardholder and the issuing bank, you often have the merchant or platform itself, a payment service provider, an acquirer, an acquiring processor, a card network, an issuer processor, and finally the issuing bank. And that's even without counting the third-party fraud vendors that some of these actors plug into their own stack. Each of these actors can decline a transaction. Each has its own risk logic, and each contributes its own false positives. Now, if 60% of your false positives are driven by upstream partners, then your maximum sphere of influence is capped at 40%. That doesn't mean you should give up and walk away, but it does mean you should rethink your goals, how you set expectations internally, and where you spend your energy. And I think that this is an important point that I see a lot of fraud leaders miss. You cannot tune rules you don't own. You can, however, quantify the impact, make it visible, make sure everyone in the organization understands where the limits of your influence actually are. And trust me on that, this saves a lot of frustration later. And also, strategically, if you're unhappy with your partner's performance and believe it's substandard, you can always work to replace them. Once you have your decisions sorted into high-level buckets, it's time to go one level deeper.

For example, you take a rule engine bucket that is responsible, say, for 25% of all false positives, and you break those 25% down to individual rules. And you do that as best you can for each one of your buckets. Now, when I say as best you can, what I mean is that realistically this can easily become time-consuming and low-value work. But let's try to break it down into the top five offenders in each category, and remember that even if a rule is relatively accurate, high volume usually means it is a primary source of false positives. High-volume decision is a good rule of thumb in terms of where to start. The point is that you want to identify the specific actors which are responsible for the most amount of false positives in absolute terms. These are your quick wins. At the same time, your gut might tell you that some offenders are flying below radar, legacy rules, outdated policies, or solutions that were never properly validated. Don't ignore that gut instinct, even if these actors have low volumes, or at least don't ignore it before you collect data on it. So we now know what is blocking your users, but we still need to know where those users are coming from, even within the part of the stack you control. False positives are rarely evenly distributed. They tend to cluster around specific flows, such as mobile versus web, iOS versus Android, maybe different products or payment methods, or maybe even new versus established users. If you only look at the actor that declined the event, you will see a rule or a model misbehaving. But if you look at which flow the user went through, you might find a deeper issue, a specific flow that produces corrupted or missing data, causing your entire fraud stack to misfire. Here's an example. Imagine you have an integration bug in your mobile SDK that sends incorrect IP data for iOS signups. You don't see that bug at first. What you see is that several geo-based rules suddenly have higher false positive rates on that platform, or a model that uses IP-based features also seems to drift, or maybe analysts complain that events coming from mobile look weird. If you only look at the rule level, you waste weeks recalibrating rule sets. But breaking it down by flow, you quickly realize the logic is fine on web, and the issue is isolated to iOS. Now, keep in mind that this isn't always a data bug. It might also be that a specific flow concentrates many good users who behave differently than your general population, which on its own can drive false positives up. But the point is, once you've bucketed your false positives by actor, do the same by user journey. Which product, which platform, which payment method, which geography, which specific funnel? I guarantee you'll see patterns emerge very quickly. The moment you start seeing patterns by flow, you will almost always run into the same culprit: data quality. Sometimes the underlying fraud logic is actually fine, and yet on a particular slice of traffic, your performance tanks. That's often because the system is operating on corrupted or missing inputs, default IP addresses where the real IP failed to capture, placeholder emails or garbage values, device IDs that reset to null on some OS versions, payment metadata that is never passed through for a certain method, timeout, or integration errors with third-party intelligence sources. You want to locate the exact data fields that are affected in those population segments you've identified. A reliable method I use all the time is to simply group data points by their value and look for values that have a suspiciously high count. And if you think about it, how many times have you done exactly that to uncover fraud patterns just to stumble upon a bug? Once you identify data quality issues, you have some detective work to do. First, we need to remember that often corrupted data points have cascading effects. A corrupted email field would, of course, corrupt email velocity checks that are based on it. So your first task is to identify all the impacted data points and link those to your misbehaving rules or models. For most organizations, this exercise can prove incredibly hard, but you have a shortcut: your list of top offender solutions from step three.

All you need to do is cross-reference your corrupted data points with the inputs used by your worst-performing rules. This will save you days of analyzing data skills by associating data issues with solutions. You will be able to complete the last link in the chain, attributing a value to each of these issues, just as you've done with offenders. You have a dollar amount, which you can put on that email bug, at least in terms of false positives. And now it's time to bring it all together. Now you have a complete map, not only of how many false positives you have, but also where and why they are generated. With that, prioritization becomes much more straightforward. You start with buckets that are large enough to matter, under your control, and not caused by data quality issues that you cannot fix. In my experience, in many organizations, these are likely to be a small number of high-volume, high-impact rules, one or two model thresholds that were set conservatively, maybe a specific case management policy that encourages overblocking, a couple flows where your system produces data issues. These are exactly the areas we'll focus on in part three, where we'll get into the mechanics of fixing your decision logic to be less trigger-happy. The work would only be effective because you've done the root cause analysis first and know how to invest smartly. For now, if you've gone from "we have false positives" to "we know where most of them come from, and which parts we can actually fix," you are already well ahead of most teams, and that's a good place to be at.

YouTube Description False Positives Masterclass, Part 2[2] Release Date Episode #10 One of the most common mistakes I see fraud teams make is attacking false positives head on. A customer complains. The CEO says the model is blocking too much. Someone opens a dashboard, adjusts a fraud model threshold, maybe tweaks a few fraud rules, and suddenly everyone feels like progress is happening. Honestly, not a good look. Not because false positive reduction is the wrong goal. It is absolutely the right goal. The problem is that most teams go tactical immediately. And if you are tactical about the way you reduce false positives, you should probably only expect tactical gains. In this episode, we get into the second part of the false positives masterclass: how to break down false positive fraud detection models into buckets you can actually prioritize and fix. We look at where false positives come from, which ones are driven by fraud detection models, fraud rules, manual review, upstream partners, fraud analysts, data quality issues, corrupted fraud signals, and payment fraud detection workflows. Quantifying false positives is useful. But it is not a plan. What you'll hear in this episode: ● Why reducing false positives requires root cause analysis, not just model tuning ● How to identify who actually declined the event: a rule, fraud model threshold, AI agent, fraud analyst, manual review team, issuer, acquirer, or fraud vendor ● Why upstream payment partners can create false positives your own fraud prevention systems cannot directly fix ● How fraud decisioning breaks down across payment fraud detection, fraud risk scoring, and operational workflows ● Why fraud system optimization starts with identifying the worst offenders ● How data quality issues, corrupted fraud signals, and model drift create false positives that look like fraud risk ● How fraud operations teams can prioritize the buckets that are large enough, fixable enough, and valuable enough to address first Who should listen: ● Fraud operations leaders trying to improve fraud detection accuracy ● Fraud analysts working through manual review queues ● Risk teams managing fraud rules, fraud model thresholds, and fraud risk scoring ● Data science teams responsible for fraud detection models and model drift ● Payment fraud detection teams dealing with issuer declines and upstream partner decisions ● Fraud prevention teams trying to reduce false positives without increasing losses ● Anyone who has ever stared at a false positive dashboard and thought, "Okay, now what?" Episode notes: (Accordion dropdown) A frustrating but familiar pattern Fraud teams often respond to false positives only after a customer complaint, executive escalation, or sudden concern that "the model is blocking too much." Okay, fair. But now you've got to ask yourself: are you solving the root cause, or just adjusting the part of the system that is easiest to see? That distinction matters. Modern fraud prevention systems give teams a lot of tools Fraud detection models, fraud rules, fraud risk scoring, manual review, AI agents, payment fraud detection systems, and fraud analysts all play a role in stopping suspicious activity. The same system that protects you can also block good users if you do not understand where each decision is actually coming from. The big gap is visibility A false positive might come from your own fraud rules. It might come from a conservative fraud model threshold. It might come from manual review. It might come from an issuer, acquirer, processor, fraud vendor, or another upstream partner. If you do not know who said "no," you cannot know what to fix. False positives are not just model errors Good customers get blocked. Fraud analysts waste time reviewing preventable cases. Manual review queues grow. Payment fraud detection teams chase the wrong problem. Fraud operations leaders struggle to explain why customer friction is rising. And somewhere in the middle of all that, a perfectly legitimate user is wondering why your system decided they looked suspicious. Not great. Data quality issues Sometimes the fraud logic is fine. The problem is that the system is operating on corrupted fraud signals. Maybe the IP field is wrong. Maybe device IDs are missing. Maybe payment metadata never passed through. Maybe a mobile SDK bug is making iOS traffic look strange. Suddenly, your fraud detection models drift. Your fraud rules misfire. Your fraud risk scoring looks worse than it is. And now the team is "optimizing the model" when the real problem is a broken input. Classic. The path forward The better path is to bucket false positives by actor, partner, user journey, product flow, platform, payment method, geography, and data issue. Prioritize the problems that are: ● Large enough to matter ● Actually under your control ● Not caused by data issues you cannot fix ● Connected to high-impact fraud rules, model thresholds, or manual review policies That is how fraud system optimization becomes strategic instead of reactive. Key takeaways: Reducing false positives is not just about fraud model thresholds, fraud rules, or manual review. It is about understanding the full fraud decisioning system, including partners, platforms, workflows, analysts, corrupted fraud signals, and data quality issues. Once you know where the problem comes from, you can finally decide what to fix first. And honestly, that is a much better place to be. (Accordion dropdown close) Transcript > (Accordion dropdown) Resources & links: Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. This is part of a larger series, so if you landed here first, you may want to go back and listen to the previous episode. We've already covered quite a lot that will make this one much easier to follow, so get caught up here: False Positives Masterclass: How to measure FPs in systems that hide them Connect with Chen Zamir | LinkedIn Host of The Saturday Fraud Strategist Helping fintechs build smarter fraud defenses Co-author of "The Fraud Fighter's AI Playbook" Optimized Episode Title: Why PSPs Struggle With Fraud Prevention Technology Episode Title: Why PSPs Are Getting Fraud Services Wrong Meta Title: Fraud Prevention Technology for PSPs and Merchants Meta Description: Why PSPs struggle with fraud prevention technology, fraud scoring, payment fraud prevention, and the operational support merchants actually need... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/fraud-prevention-technology Keywords Primary Keyword: fraud prevention technology Secondary Keywords: payment fraud prevention fraud prevention software fraud detection technology fraud detection software fraud management software AI fraud detection machine learning fraud detection fraud scoring fraud risk management merchant fraud prevention fraud prevention for payment service providers chargeback prevention false positive reduction payment risk management fraud operations fraud investigation tools fraud detection models Why PSPs Are Getting Fraud Services Wrong Release Date Episode #9 So over the years I've had a lot of conversations with Payment Service Providers that wanted to build fraud prevention technology. Not necessarily for their own internal risk controls. That would almost be too obvious. What they really wanted was to offer fraud prevention software as a value-added service to their merchants. Honestly, I get it. Payment fraud prevention can help PSPs differentiate, win deals, increase stickiness, and create new revenue. Pretty good on paper. But then you get to the uncomfortable part. A lot of teams assume fraud detection technology is basically an API, payment data, and a machine learning fraud detection model that returns a fraud score. Not a good look. I break down why fraud prevention technology is much harder to build, maintain, and operationalize than it looks, why fraud scoring alone does not solve merchant fraud prevention, and why payment service providers need to think seriously about fraud operations, chargeback prevention, false positive reduction, payment risk management, and the support merchants actually need. What you will hear in this episode: ● Why PSPs want to offer fraud prevention technology as a value-added service ● Where fraud prevention software becomes more complicated than expected ● Why AI fraud detection and machine learning fraud detection are not enough on their own ● How fraud scoring can create confusion if merchants do not know how to act on it ● Why merchant fraud prevention requires operational support, not just fraud detection software ● How chargeback prevention, false positive reduction, and payment risk management affect real business outcomes ● Why fraud prevention for payment service providers needs to include strategy, support, and fraud investigation tools Who should listen: ● Payment service providers considering fraud prevention technology ● PSP product leaders building payment fraud prevention services ● Fraud operations and payment risk management teams ● Merchants evaluating fraud prevention software or fraud management software ● Risk leaders responsible for chargeback prevention and false positive reduction ● Teams working with fraud detection models, fraud scoring, or fraud investigation tools ● Anyone trying to understand why fraud prevention technology is not just a model returning a score This episode is for people who want to reduce fraud without pretending fraud risk management is magically solved because someone added "AI" to the roadmap. Episode notes: (Accordion dropdown) The broader context More PSPs want to offer fraud prevention technology to their clients. Okay, fair. Merchants already trust their PSPs, the integration path looks easier, and bundling payment fraud prevention into an existing payments relationship sounds clean. But you've got to ask yourself: which merchants are actually buying fraud prevention services from their PSP instead of a dedicated vendor, and what do they really need? What's working There is real value here. PSPs already sit close to payment data, transaction flows, chargebacks, and merchant relationships. Modern fraud detection technology, AI fraud detection, machine learning fraud detection, and fraud scoring can help identify suspicious payments faster. Strong fraud prevention software can support chargeback prevention, reduce manual review, and improve payment risk management. Where gaps remain Here's the problem. Fraud prevention technology is not finished when the model returns a score. Now what? Does the merchant know which cutoff to use? Can they review their fraud patterns? Can they investigate suspicious payments? Can they tune rules, reduce false positives, review chargebacks, or block repeat abuse? The human impact A bad fraud decision is not abstract. A false positive can block a good customer. A missed fraud attack can create chargebacks. Weak fraud scoring can leave a merchant exposed. Fragmented fraud management software can make teams slower, less confident, and more reactive. And for smaller merchants, especially the ones most likely to buy fraud prevention through a PSP, there may not be a fraud team at all. So when the system says "high risk," they may be thinking, "Great. What am I supposed to do with that?" Honestly, fair question. Ethical responsibility It is easy to market AI fraud detection as if it solves everything. It does not. If PSPs sell fraud prevention technology as a simple add-on but do not support the operational reality behind it, merchants inherit the confusion. That is not fraud risk management. That is outsourcing anxiety with a nicer UI. The path forward The better path is not necessarily "never build." Some PSPs can absolutely build strong fraud prevention software. But the real question is whether they can afford the long-term investment required to keep fraud detection models competitive, support merchants, manage fraud operations, and adapt as fraud patterns change. Because the real product is not just the score. The real product is helping merchants make better fraud decisions. Key takeaways: The question is not just whether PSPs should offer payment fraud prevention. Some should. Some probably should not. The better question is whether they understand what they are signing up for. Fraud prevention software is not just something you launch. It has to be maintained, tuned, explained, supported, and adapted as fraud patterns change. So if the plan is "we'll train a model and merchants will figure it out," then honestly, you've got to ask yourself: Is that a product, or just a future support queue? Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. (Accordion dropdown close) Transcript > (Accordion dropdown)

Over the years, I had many calls with PSPs wondering if I can help them build fraud prevention technology. Usually, though, they don't want to build it for their own internal risk controls. Instead, what they want is to offer it as a value added service, or VAS, to their clients. As someone who has built and ran a fraud vendor that partnered with PSPs, I believe that it makes strategic sense. Offering VAS products is one of the main ways PSPs can differentiate in today's highly commoditized market.

It helps win deals, it increases product stickiness, and it creates new revenue channels. What does not make sense is how most PSPs are thinking about it. So let's talk about it. The most common mistake I see PSPs make is trying to build fraud tech on their own. Now, I get why they do it. On the surface, it looks like a straightforward task. All we need is an API, check, payment data, check, and what's left is to train a model for fraud detection, right? But nothing can be further from the truth.

The first reason is that fraud detection technology may seem simplistic from the outside, but in reality, the simple task of training a machine learning model becomes a black hole for resources in the effort to showcase competitive performance. It's not just about configuring the algorithm. It's about acquiring third-party data sources, developing velocity checks, dealing with false positives and dirty fraud labels, managing different client segments and regions, and the list goes on. The effort you need to put into launching it is just a tiny piece of the overall investment that is required to keep it alive and competitive. After all, fraudsters adapt, and so do customer behavioral patterns. And the cost of maintaining fraud detection models over time is bigger than what many of the uninitiated might suspect. But here's the thing, let's assume for a second you've developed a fantastic fraud detection model. You now are able to return fraud scores in your payments API. So what? Do clients now know how to use it? Which score cutoff to choose? Can they better analyze their fraud patterns, review their chargebacks, investigate suspicious payments, block that pesky address that keeps requesting refunds? The answer to all of the above is still no. Let's take a step back for a moment. If we're developing a product, we need to consider, what do clients want? But in order to answer this question, we also need to consider who these clients are, and specifically, which clients would buy fraud prevention services from their PSP rather than from a vendor. So let's review the pros and cons for buying fraud services directly from your PSPs. The pros: I already know and trust my PSP. Minimal engineering resources required, if at all. And having only one vendor to manage both payments and fraud. The cons, on the other hand: PSPs offer less customization. PSPs cover only payment flow, so no onboarding, login, etc. PSPs likely have linear APIs that skip many relevant data points. PSPs have fewer dedicated resources to fraud, meaning the performance is likely unoptimized. And finally, if I'm working with multiple PSPs, my fraud management would be fragmented as well.

Now, what can we learn from this? Mainly that there are several distinct groups that would rather buy fraud services from their PSP than directly from a vendor. These are small merchants that can't afford the resources or cannot deal with the operational overhead, low fraud merchants that do not require best in class technology and will deprioritize fraud prevention investments in general.

And funnily enough, merchants of high risk verticals might also be a target. Often they choose to work in parallel with both direct vendors and their PSPs, as they are less margin sensitive and put importance on performance optimization. What does this teach us? It teaches us that the type of clients that choose to work with their PSP on fraud prevention usually does not have it as a top priority, and that they probably won't spend a lot on it. They don't have a large fraud team, if at all. So they would probably love it if someone else could take care of it for them, someone just like you. And here lies the crux. More often than not, clients who work with their PSPs on fraud prevention need professional services as much, if not more, as they need the tools and technology. Instead of building amazing UI and UX features, PSPs are often asked to support their clients with their specific fraud management needs. This can be as simple as recommending the right model cutoff score, or as comprehensive as analyzing emergent fraud patterns and recommending remediation steps. The reason I believe most PSPs miss this crucial point is that these needs are not necessarily clear to the merchants themselves. Often they encounter fraud for the first time, think AI is the solution, like it is for all humanity's problems, and expect that by just receiving a score, fraud would be resolved. So, in essence, we have two non-expert parties that believe they are aligned on expectations when, in reality, they are both wrong. Does that mean that PSPs should never build their own fraud VAS products? No. Evidently, both Stripe and Adyen are doing really well on that front. The question is whether you can afford to invest the same. YouTube Description Why PSPs Are Getting Fraud Services Wrong Release Date Episode #9 So over the years I've had a lot of conversations with Payment Service Providers that wanted to build fraud prevention technology. Not necessarily for their own internal risk controls. That would almost be too obvious. What they really wanted was to offer fraud prevention software as a value-added service to their merchants. Honestly, I get it. Payment fraud prevention can help PSPs differentiate, win deals, increase stickiness, and create new revenue. Pretty good on paper. But then you get to the uncomfortable part. A lot of teams assume fraud detection technology is basically an API, payment data, and a machine learning fraud detection model that returns a fraud score. Not a good look. I break down why fraud prevention technology is much harder to build, maintain, and operationalize than it looks, why fraud scoring alone does not solve merchant fraud prevention, and why payment service providers need to think seriously about fraud operations, chargeback prevention, false positive reduction, payment risk management, and the support merchants actually need. What you will hear in this episode: ● Why PSPs want to offer fraud prevention technology as a value-added service ● Where fraud prevention software becomes more complicated than expected ● Why AI fraud detection and machine learning fraud detection are not enough on their own ● How fraud scoring can create confusion if merchants do not know how to act on it ● Why merchant fraud prevention requires operational support, not just fraud detection software ● How chargeback prevention, false positive reduction, and payment risk management affect real business outcomes ● Why fraud prevention for payment service providers needs to include strategy, support, and fraud investigation tools Who should listen: ● Payment service providers considering fraud prevention technology ● PSP product leaders building payment fraud prevention services ● Fraud operations and payment risk management teams ● Merchants evaluating fraud prevention software or fraud management software ● Risk leaders responsible for chargeback prevention and false positive reduction ● Teams working with fraud detection models, fraud scoring, or fraud investigation tools ● Anyone trying to understand why fraud prevention technology is not just a model returning a score This episode is for people who want to reduce fraud without pretending fraud risk management is magically solved because someone added "AI" to the roadmap. Episode notes: (Accordion dropdown) The broader context More PSPs want to offer fraud prevention technology to their clients. Okay, fair. Merchants already trust their PSPs, the integration path looks easier, and bundling payment fraud prevention into an existing payments relationship sounds clean. But you've got to ask yourself: which merchants are actually buying fraud prevention services from their PSP instead of a dedicated vendor, and what do they really need? What's working There is real value here. PSPs already sit close to payment data, transaction flows, chargebacks, and merchant relationships. Modern fraud detection technology, AI fraud detection, machine learning fraud detection, and fraud scoring can help identify suspicious payments faster. Strong fraud prevention software can support chargeback prevention, reduce manual review, and improve payment risk management. Where gaps remain Here's the problem. Fraud prevention technology is not finished when the model returns a score. Now what? Does the merchant know which cutoff to use? Can they review their fraud patterns? Can they investigate suspicious payments? Can they tune rules, reduce false positives, review chargebacks, or block repeat abuse? The human impact A bad fraud decision is not abstract. A false positive can block a good customer. A missed fraud attack can create chargebacks. Weak fraud scoring can leave a merchant exposed. Fragmented fraud management software can make teams slower, less confident, and more reactive. And for smaller merchants, especially the ones most likely to buy fraud prevention through a PSP, there may not be a fraud team at all. So when the system says "high risk," they may be thinking, "Great. What am I supposed to do with that?" Honestly, fair question. Ethical responsibility It is easy to market AI fraud detection as if it solves everything. It does not. If PSPs sell fraud prevention technology as a simple add-on but do not support the operational reality behind it, merchants inherit the confusion. That is not fraud risk management. That is outsourcing anxiety with a nicer UI. The path forward The better path is not necessarily "never build." Some PSPs can absolutely build strong fraud prevention software. But the real question is whether they can afford the long-term investment required to keep fraud detection models competitive, support merchants, manage fraud operations, and adapt as fraud patterns change. Because the real product is not just the score. The real product is helping merchants make better fraud decisions. Key takeaways: The question is not just whether PSPs should offer payment fraud prevention. Some should. Some probably should not. The better question is whether they understand what they are signing up for. Fraud prevention software is not just something you launch. It has to be maintained, tuned, explained, supported, and adapted as fraud patterns change. So if the plan is "we'll train a model and merchants will figure it out," then honestly, you've got to ask yourself: Is that a product, or just a future support queue? Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. (Accordion dropdown close) Transcript > (Accordion dropdown) Optimized Episode Title: AI Phishing Detection: Should Fraud and Cybersecurity Converge? Episode Title: Should Fraud and Cybersecurity Teams Converge? Meta Title: AI Phishing Detection Meets Fraud Strategy Meta Description: AI phishing detection now depends on shared fraud telemetry, email security signals, upstream detection, and AI model testing to stop AI-powered fraud... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/ai-phishing-detection Keywords Primary Keyword: AI phishing detection Secondary Keywords: AI email security invoice fraud ACH fraud accounts payable fraud credential stuffing account takeover security telemetry fraud telemetry upstream fraud detection AI model testing fraudster ROI AI-powered fraud Should Fraud and Cybersecurity Teams Converge? June 13, 2026 Episode #8 Guest: Cy Khormaee Every few years our industry rediscovers the same debate: should fraud and cybersecurity teams actually sit together? And honestly, usually both sides hate the idea immediately. Not because they dislike each other. Mostly because both teams are already overwhelmed and nobody wants another meeting. But over the last couple of years, something changed. The signals started converging. Credential stuffing became account takeover. Account takeover became fraud. Fraud became phishing. Phishing became invoice fraud and ACH fraud. And suddenly the same security telemetry that detects compromised infrastructure also helps identify fraudulent users before they ever reach checkout. That is where things start getting weird. In this episode, I sat down with Cy Khormaee, who helped build Recaptcha at Google and now runs Aegis AI, to talk about why AI phishing detection is forcing fraud and cybersecurity teams closer together whether they like it or not. And honestly, once you realize the same behavioral signals can stop both account takeover and payment fraud detection, the organizational separation starts feeling a little artificial. We get into AI email security, AI-powered fraud, fraudster ROI, upstream fraud detection, and why modern attackers are moving faster than most enterprise security stacks were designed for. Also, I learned that Google literally tracked the market price of breaking CAPTCHA systems like a stock ticker. Which honestly feels extremely fraud-brained. What you'll hear in this episode: ● A practical look at why fraud and cybersecurity teams are starting to share the same signals ● How credential stuffing and account takeover pushed security tools into fraud prevention use cases ● Why AI phishing detection depends on more than static email rules or reputation checks ● How AI email security is changing as attackers use AI to generate more targeted phishing attacks ● Where invoice fraud, ACH fraud, and accounts payable fraud sit between security and fraud operations ● Why security telemetry and fraud telemetry become more useful when teams connect the full user journey ● How Recaptcha evolved from image puzzles into behavioral detection and fraud prevention infrastructure ● Why "good people leave tracks" still applies across both fraud and security signals ● How upstream fraud detection helps stop problems before money leaves the platform ● Why fraudster ROI is one of the most useful ways to think about modern defense ● What teams should ask vendors before buying AI-powered fraud or AI security tools Expect a conversation about tools, signals, attacker economics, and the awkward reality that fraud and security may already be converging, whether the org chart admits it or not. Who should listen: ● Fraud leaders and fraud analysts ● Cybersecurity professionals ● Trust and safety teams ● FinTech fraud prevention teams ● Email security teams ● Accounts payable and payment risk teams ● Teams evaluating AI phishing detection or AI email security vendors ● Anyone working on credential stuffing, account takeover, invoice fraud, ACH fraud, or upstream fraud detection Basically, if your fraud team and cybersecurity team only meet during incident review, this one may be worth playing in both rooms. Episode notes: (Accordion dropdown) This conversation starts with ransomware in a classroom, which is funny for about three seconds and then immediately becomes the point. Honestly, for a cybersecurity professor, that is either terrible timing or very strong teaching material. Maybe both. It also happens to be a clean way into the actual topic: what happens when security stops being theoretical and fraud stops staying neatly inside the fraud team? Cy and I talk about the overlap from a practical angle. Not "should the org chart change?" in some consulting-deck way. More like: are both teams already looking at the same attacker behavior, identity patterns, and telemetry? A lot of the answer is yes. We get into Recaptcha, account takeover, phishing, invoice fraud, ACH fraud, and why the line between security and fraud keeps getting blurrier. Cy also explains why AI-generated attacks are making the old rule-based playbook harder to trust, especially when adversaries can adapt faster than most enterprise systems. The part I think fraud teams should pay attention to is the operational one: how far upstream can you detect the problem before it turns into money movement, cleanup, and a very uncomfortable incident review? We also talk about what to ask vendors claiming they have AI, including how their models work, what happens to your data, and whether AI model testing is actually happening or just being implied in a slide deck. Because "we use AI" is not an answer. Not anymore. Key takeaway: Fraud and cybersecurity do not need to become the same team tomorrow. But they do need to stop pretending their signals live in separate universes. AI phishing detection, AI email security, fraud telemetry, security telemetry, account takeover detection, and upstream fraud detection are all starting to point at the same thing: the attack path is connected, so the defense probably has to be connected too. The goal is not perfect security. That does not exist. The goal is to make the attacker's business model worse. Less glamorous than "stop all fraud." Probably more realistic. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. (Accordion dropdown close) Transcript > (Accordion dropdown)

Welcome to another episode of the Saturday Fraud Strategist. And with us today, joining us, is Cy Khormaee. Cy is a seasoned cybersecurity expert. He's a professor for AI at UCLA. And he's also the founder and CEO at Aegis AI. Cy, welcome to the show.

Well, Chen, thank you so much for having me. It's really a pleasure to be here. And it's funny, you know, there is never any break from security. I was just teaching yesterday and we saw our learning platform got ransomware, live in the classroom. So I was reminded there are no days off.

Well, that's at least good material for the students.

Yes, absolutely. Gave me a really good intro. So this is the first class. They didn't even know who I was. I was like, hey, so good news and bad news, there's an incident. The good news is this is actually what I do for a living. So let me give you like two or three tips that we're gonna do right now before we proceed.

I'm sure it looked completely scripted to the students.

Yeah, it did. Except it actually impacted, you know, it's a large LMS platform that was impacted. I think it impacted like a large number of schools across the United States. So I swear it wasn't actually me.

Well, I don't know. But anyway, Cy, for those listeners who don't know you, maybe you can tell us a bit more about yourself.

Yeah, so just quick background, you know, I grew up doing large scale ETL at Microsoft, so built some of the early systems behind AdCenter Analytics. Back when ETL meant you were writing, you know, C data pipelines, you know, people now have it so easy. I then spent about a decade building companies, all kind of in the AI and data space. So one applying it to commercial real estate, one applying it to sales technology, how to kind of send great, relevant emails to people, and then, you know.

Yeah.

Spent some time at Google, I'm sure we'll dive deeper into that, where I built, you know, some of the earliest, you know, versions of reCAPTCHA, Safe Browsing, a lot of the core platforms that are behind Google's phishing, malware, and fraud systems. And, you know, more recently I'm running this company with some friends called Aegis AI, still fighting the good fight against fraudsters in the security space, trying to keep email safe.

That's awesome. So speaking of which, I think one of the reasons why I thought it would be super interesting to speak to you is because over the past few years, I would say five years or so, there's this, let's call it, debate in our industry, whether, you know, fraud prevention and cybersecurity shouldn't actually be within the same organization, like fraud being rolled up to the CISO, or similarly. And I wonder, like, obviously I have my thoughts about it, but I wonder where do you stand on this controversial topic.

Yeah, it's a really interesting topic. And what's really funny about it too is, look, I have so many friends in so many different areas, because, by the way, like, no matter what, fraud and security are very close partners and they do very similar jobs. The difference is the goals are really different. If you think about most fraud professionals, especially in, say, the credit card space, it's like, okay, take that fraud rate from 3, 4, 5% to 1%. Like, that's the goal. In security, it's don't get compromised. And they're like slightly different goals, but all the behaviors are similar. And what's really funny is I see leaders get asked to combine the roles all the time, and both leaders are like, no, we don't want to, we're already busy enough. So maybe one of the biggest limiting factors is both jobs are so hard, it's almost impossible to combine them in the same person. But I do think the activities are very similar, and there's a lot of overlap where they're working more and more together, and it does make a lot of sense to put them together, despite, you know, sometimes the resistance, because I think both sides are just way too busy all the time.

Yeah, that's super interesting. I think that, you know, when I think about it, I see that, as you kind of mentioned, the processes and sometimes kind of the talent in terms of profile might look quite similar. But I think also, to an extent, let's say fraud is much more of an everyday occurrence when you compare it to cybersecurity, right? At least in most organizations. And so how you approach it, whether this is a theoretical threat against you where you need to kind of just safeguard yourself and mostly pen test yourself, or is it a daily grind that you need to kind of manage and you know that you'll get on the short end of the stick on a daily basis. I agree. It's like, even though the organizations look like they work similarly, the way that they kind of structure their KPIs, their goals, is quite different. Interesting. I must say, though, and maybe we'll get to that in the end, in the past couple of years we see a lot of changes, right? In cybersecurity and fraud. So I don't know, like, if it made you kind of rethink where you stand on it.

Totally. I think if you asked me this question six years ago, I would say, hey, they're very different orgs, different technology and stacks, like it's too hard to integrate them, kinda don't bother. I think that we've observed that the signals are converging rapidly, and then with AI, there's more and more opportunity to share data across the kill chain. Even, you know, Matt Vega on your team is someone we know well. You know, we first overlapped when he was at Instacart, because I was the product leader for reCAPTCHA at the time. Instacart had been using reCAPTCHA to stop kind of credential stuffing logins, like people just trying a bunch of different passwords to see if they can get through. And at some point they had a big issue with carding. And, you know, the team came and said, hey, like, can we use reCAPTCHA to stop this? Like, can you use the security tool to solve a fraud problem? Like, I don't know. But let's try. Let's dive in together. Let's give it a shot. Turns out the answer was yes, you absolutely can. So the same tool, the same telemetry, just more broadly applied, was really effective. And the other thing we learned is actually watching the behavior from login to on-site behavior to takeout behavior was very helpful. So it's not only do I give you three similar signals with the same observability, but you can connect those three experiences into a fraud journey. It really tells a story about the adversary or the benign user, to have kind of higher accuracy. And so that was maybe one of the first wake-up calls where the tools themselves were just hyper-converged, right? And it kind of led us down this journey of kind of looking at fraud as a capability of reCAPTCHA, which had traditionally been a security tool. And we should also maybe tell that story at some point too. On the other side, you're seeing data become more and more accessible, where you can have everything in a data lake and start to put together that journey all at once. So I guess if fraud professionals are kind of listening and thinking about that, it's like, what toys does security have on the tool side that you want? And then what data does the security team have that you want? And I would bet that your security counterpart wants the same toys from you, and the same data from you. And that sharing is gonna create a really positive match.

And for those listeners that are not, I mean, I think everybody's familiar with reCAPTCHA, but not necessarily understanding or matching the experience and the name of the product. Can you maybe just explain quickly what reCAPTCHA is and how it looks like?

Yeah, and also how it looks like over time, right? So you think about when CAPTCHA really came out, something like fifteen years ago. It was like that little squiggly bit of text that you were trying to read to understand what's happening. You know, Google acquired the technology 'cause it was a great way to train some of the Google text to, sorry, kind of image-to-text AI, the very, very early versions of OCR Google had.

Then it evolved to do kind of image recognition. So I think we've all struggled to figure out, you know, what is a stoplight, what is a bicycle. It's this very existential question that you find. And, you know, again, that was used for a lot of like the early maps training data. At some point in that period, we found two things, maybe three things. One is that everyone's pretty bad at this, especially regular users are pretty bad at this recognition. And adversaries are really good at it. And that creates a couple problems. The first problem is it becomes an ineffective gating tool when there are warehouses of people manually solving it who are much better than anyone in real life. So that creates an asymmetric to the downside detection technique. The second is the data you're getting out of it is getting pretty dirty and makes it difficult to use on AI. So that's kind of at the point I took over the product, at the point where there were these images that would kind of figure out what the future of this product was. At the same time, we talked a little bit about credential stuffing. For those of us who were in the community in 2016, 2017, there's a bunch of data breaches that happened. So essentially it went from like a password and a username being somewhat infrequent and somewhat hard to come by, to it being ubiquitous. I think everyone's passwords were leaked on a, you know, maybe two to three times a year basis, right? That's like very problematic, because it means that you can test a very small number of these passwords to bypass any login. Which meant that these login breaches and these ATOs were happening at an exploit increasing rate. And we realized, hey, we can apply reCAPTCHA on top of that and stop it. And we don't stop it because of the images. We actually stop it by looking at the behavior of the client on the page. So if you look at, you know, your behavior on the internet, it's well observed over a long period of time. You're gonna show up at a bunch of places. I'm gonna see your cookies and your clients everywhere. From that I can infer you're a pretty good actor. You know, there's not a lot of bad reputation out there.

Who knows, but probably not, right? On the flip side, if I've never seen you before, can you imagine this scenario of, like, you walk into a bank versus I walk into a bank with a ski mask on? Well, that's immediately gonna raise a bunch of questions, right? And it's a very easy way to filter out kind of benign from malicious. And we've done that across a bunch of different areas. If anyone's been on a Zoom call, a Google Hangout, you know, basically 80% of probably the internet usage of this audience is all gated with reCAPTCHA, that's looking at kind of these signals. I'll show you kind of a little chart of, like, what that looks like over time. You know, I think I took over this product in, you know, 2017. We had about 55 of the top 1,000 sites. And, you know, over the five years or six years there, we brought it up to 320 sites, which, by the way, it's probably the larger of the top 1,000. So it really represents a propensity of that traffic. And we're able to do that because we're able to actually get better detection. Because you're able to do a detection that the adversaries don't have a way to game, or not an easy way to game. They can't see the answer like they can with a CAPTCHA. And it doesn't annoy users. And so we found this really happy medium that really enabled this project to take off. And then the third thing that happened that was really beneficial is we got to a data density where, like, 99.999% of users were going to be known to us, and known to us across multiple different concurrent platforms. So I can say with very high certainty.

Hey, again, Chen is like a very legitimate user, because I can see him in all the other sites. So there's a lot of cross-telemetry that can happen. And so you can actually say, hey, look, if I just haven't seen you before, that becomes a stronger and stronger negative signal versus trying to look for the negative signals in sparse data. And I think that, if you think about most detection platforms, is such a powerful place to be, where we go from understanding the majority of the market to understanding the minority of the market. So you can really look and say, look, I already see most of the world, and you look really strange compared to most of the world. That's a really great way to detect, 'cause also, by the way, adversaries don't have that data, so it's great. And it's asymmetrically advantageous to the defenders, which is again a unique place to be.

That's super interesting, because I think all of these three signals that you just described are, you know, we use exactly the same methods in fraud prevention. They just look slightly similar. So, for example, you mentioned that bad actors would be really good at solving the bicycle challenges. And it's the same kind of idea, where if I would see a customer's name spelled with a spelling mistake.

I would know it's not a fraudster, because a fraudster will, if they do not copy paste it, they would check it really, really well before they click the button. So that's one. Second, we have a saying that in fraud, good people leave tracks. So if you see someone new that you don't have their history, that on its own is quite suspicious, especially if you have data at scale.

And thirdly, of course, anomaly detection is a hallmark. So we go back to the fact that it's interesting that, you know, both of these are quite different teams, quite different, let's say, domains, but in the end the behavior of the bad actor, and how we are trying to detect it, is so similar.

Yeah. And by the way, if you look at the signal gathering itself, it's identical. So if you look at, like, how we think, what we're looking for, as your point, we actually converged this, and we started a fraud business out of reCAPTCHA. Like, it's actually the same tool. We just give it to a different team for a different purpose, but it's actually the same tool. And, actually, in a lot of places we deployed, it would start just like it did at Instacart, from that front door, and then just kind of percolate backwards towards the fraud team. So maybe the big difference is, like, the security team is always at the front door with authentication, identity. And then we just, like, propagated that signal backwards until it reached the takeout, which is kind of where the fraud team really lived and owned. And somewhere in the middle, with the negotiated difference between the fraud and security, it actually helped both sides. Because in security and ATO, I may not be totally sure when you first log in, but I'm gonna get increasing certainty as I go down this funnel. By the time I reach takeout, we have high certainty. And that actually plays on both the ATO as well as the takeout scenario. So again, it benefits both teams closely. So, to your point, this, I mean, you kind of started this conversation by asking, like, should the teams converge? This is an example where the tool basically converged, and we had these security teams and the fraud teams working closer together on the exact same data, with the exact same install point, with the exact same telemetry, and having really positive conversations. And even by starting to blend their metrics, we're like, hey, actually, if you block a little more on the ATO side, specifically this type that's going after, like, a high-ticket user that looks like a compromised user, we can reduce fraud rates on the back end. And so the numbers started to influence each other, which is maybe where you're getting to, where it's like there's starting to be, like, true organizational overlap when, because the data is shared, and each team can affect each other's KPIs, which is always true, they just couldn't see it. They're actually maybe able to make smarter trade-offs at the front door now, which is really interesting, or even delay things. Like, if we think about, you know, when is the decision made to delay a checkout or delay a delivery? You go, we think about delivering a phone. That's a big miss. If you deliver a phone, I'm gonna make this up, but it's a 30% margin product. So if I deliver one phone to a fraudster, I need to sell three phones to make up for that, or five phones to make up for that. So it's a big, big miss. So, say, look, at the front door, I see all this data that's making me more and more nervous. And so let me gate the delivery of that product until we do the proper review. Let me gate even maybe the running of that card. So a lot of times we'll do, like, the prepay or the issue of the payment. Let's do an additional 30-second fraud check. Let's do the deep dive, even have an analyst look at it before.

We even touch the credit card payment network that might affect our standing there. So I think that's where we're seeing, like, extremely positive overlap that I'm pretty excited about. And, again, it's just making life harder for the fraudsters. So, like, my whole business is we want to make their lives harder. Even at reCAPTCHA, you know, by the way, two funny things about reCAPTCHA, the way we measure it as a fraud teammate. One is that we measured how much it cost to break a CAPTCHA. So there's a market price for this, and we just track it, you know, just like the stock price. And my job is to make that number really high.

Interesting.

So it's unaffordable to do this, because it's never impossible. All fraud people know it's never impossible. It's just a matter of cost. And your adversary is running a business. And my job is to make the business not profitable, so they go do something else with their lives. On the flip side, every once in a while, there's an article that says, like, AI can break reCAPTCHA, which is totally true. It absolutely can. The difference is, can you break it at cost? And what's funny is right now, in most cases, it's much cheaper to use a human to break a CAPTCHA than it is to actually have an adversary do it. And even, actually, in some of our later work I'll show you, like, we found ways to do this automatically with LLM-based browsers. It's all possible. It's just more expensive.

Yeah, it's just about ROI, right? It's not about whether it's possible, but whether it's worth it for the bad actor to actually do that, or to go somewhere else.

Exactly. 'Cause I think the only person that says fraud or security breaches are impossible does not work in fraud or security. Nothing's impossible. It's like, it's all possible. In fact, it's all somewhat, unfortunately, somewhat likely. It's just, you know, to your point, a matter of cost.

Absolutely. Interesting.

Okay, so you described the way how you kind of penetrated companies through the cybersecurity team and slowly made your way to the fraud team. And I wonder if you thought of basically packaging the reCAPTCHA product for fraud teams. Was that a product you were thinking of launching? Did you launch something like that?

Yeah, we actually did. So it's a great point. That's actually what we did. And the reason is the value increases. So we talked a little bit about the metrics difference between security and fraud. Security, like, always has a fuzzy number on what an ATO cost is. You're like, I don't know, if I take over some account, how disruptive is it? How likely is a takeout to happen? How likely is it to disrupt the user experience? It's all a little bit fuzzy. And, to your point you made earlier, it's kind of risk-based, right? We're trying to, like, squint at a number and figure it out. Whereas fraud's like, no, no, no, there's like actually a number. And what's even worse is if you're running like a 20, 30% margin business, maybe a retail business, you're running 10%. That's five, six, seven, maybe ten percent of your net income, because you're losing just straight net income off the top. So it's really, really serious. And I'll tell you, the reason I got excited about this as a product leader is every CEO I talked to, including like Fortune 10 retailers, knew their fraud number. They mention it in the earnings call. No one's really talking about their ATO numbers. It actually is a great motion for us, and I think for the world, because we were basically moving from an important, but, you know, from the organization's perspective, lower value use case, to a higher value use case. We're like, hey, if I stop one percent of your fraud, that's like, you know, 10% of your net income. That's like, every dollar I saved you is real dollars back in your pocket that, like

I can ask for some portion of, and it's super clear. Like, there's literally a number, and you can run live A/B tests. So if you want, you can run a 10% sample forever, and constantly measure me against that, and measure the spikes. The one thing I'll say that threw off that sample, which is kind of funny, is a lot of adversaries became so afraid of interacting with reCAPTCHA, they'd actually have a separate sensor network to identify sites with reCAPTCHA and avoid them. Because if you got onto a site with reCAPTCHA, we'd look at your infra and we'd burn it.

And so people did not want to get anywhere near the reCAPTCHA sites over time, because they viewed the risk was really not worth it. And, by the way, one of the most costly things I can do to an adversary is understand their command and control, understand their IPs, understand the infrastructure, especially compromised infrastructure they've acquired. Burn it, meaning tell everyone that it's bad. Maybe even, by the way, tell the owners of that infrastructure that they've been compromised, help them fix it.

Yes.

And now the adversary has a two, three, four, five month rebuild process, which is really, really difficult to deal with, which again really drives the cost up, which again is my goal. So I make their lives a little bit difficult. I'm not their most... I mean, there's probably a dartboard with my face on it somewhere in, you know, a non-extradition country. Or a few dartboards.

You hope. Yeah, that's super interesting. I mean, I would hazard a guess, like, I don't know how you see it, but I would hazard a guess that today it is actually easier for malicious actors to rebuild their infrastructure, and even to kind of mutate it in real time to kind of avoid this burn, right? Because that's a very common tactic of both fraud prevention teams as well as cybersecurity teams.

Yeah, it's a really good point. You know, let me actually dig up some data here. It's kind of interesting to see. I'll show you some data here really quickly. And what we're gonna show is that actually the major providers, well, it's become harder, and this is some data presented at M3AAWG, so this is presented at an academic conference. But, you know, what we found is, like, the major providers are getting better at gating random IPs, random domains. So that stuff doesn't work very well anymore. So you actually need to use compromised domains, which means that the diversity of domains used in attacks is actually falling. Because you're right, it is trivially easy to create new domains and new IPs. The problem is those don't pass the basic security controls anymore from, you know, Google or anyone else. And so, to commit fraud successfully, not that this is a guide, but it's probably very well known, is, like, you need to go find someone with a good reputation, ATO them, take over their infrastructure, and use that.

To do the takeouts. Like, that's what's effective. And so in this graph, what you're seeing is, you know, a lot of these attacks are coming from a smaller number of domains, and specifically attacks that are, you know, AI-powered. So if you have the smaller set of domains that are more precious, I need to have a much higher conversion rate attack. I can't afford to really spray and pray, and just see what happens, that I could with this infinite creation of IPs and domains that's, to your point, fully automated. If I compromise the domain, I spend some money and time, it's a unique thing, an asset I need to use.

I better have a hit rate that's ten, twenty, thirty percent, not point zero, point zero zero one percent.

Yeah, that's interesting, because I think it goes back to the principle of good people leave tracks, and the fact that it's, like, it's not only about the identity that is kind of carding, or the user that is trying to access an account, but it's also the infrastructure itself, right?

Yeah. And SharePoint are actually both examples of infrastructure, right? I think, again, Matt on your team does a phenomenal job of showing some really terrifying, like, fake ID, fake KYC passing stuff that is terrifying. And that's, nothing where, like, people will take real identities, or sell real identities, or sell real facial identification, or real devices, like, human identity, device identity, aged Google accounts, these are all assets. Okay, essentially everything a fraud fighter would look at are, like, all these trails, to your point, are all things people realize, and they're buying and trying to assemble these trails. And then the harder we make that, the more the trail we look at, the more difficult it is. So if all I need is to not look like a headless Chrome browser, that's easy. If I need to have an aged GAIA account, the correct username and password, a CAPTCHA bypass, egress, an aged credit card or banking account, like, now you made my job really difficult. I wanna go do something else in my life, right? Like that, just as we expand, that chain becomes exponentially more expensive, which is good.

Yeah, exactly. It makes it mainly more expensive, and then my ROI goes down, and I go elsewhere. So I wonder, you described kind of a lot of examples of how, especially, you know, through the reCAPTCHA lens, how fraud teams adopted cybersecurity tools. I wonder if you've seen the opposite, or if you had that experience in reCAPTCHA, where you kind of baked into the product.

Features or capabilities that you've seen in the fraud prevention world, and you thought that this actually can be also very useful for cybersecurity.

Yeah, absolutely. And, you know, we're even doing this today. So at reCAPTCHA, one thing that we did was we started looking at the transaction signals. So, like, okay, well, we're working our way back to the takeout. You know what we should do is we should go look at the transaction, right? Because that's a classic case where, like, I get a lot of data about not just if this is bad or not, which is kind of where security typically lives, but, like, how much is it? How often is it stolen? What's going on here? Like, if you think about the Instacart example, how often are groceries actually stolen? Like, I assume pretty rare. It was actually a different kind of abuse, right? Versus a phone is a really nice commodity to sell. It's compact, it's high value. Like, that's a great one. So, versus, I don't know, like a hair tie. Like, I don't know what you're gonna do with that, but good luck reselling that. It's not worth it, right? So you can make these decisions in a more nuanced way. And, actually, even brings you kind of what we're working on today, and look, working with Sardine on, which we're really excited about, is starting to look at some things in the invoice, right? This is actually, again, a

I think a perfect example of the marriage between these two areas. Let me see this. I mean, here's an example where, like, it's actually one of the hardest problems. You know, right now we're an email security company. So at Aegis, what we basically do is we apply all of the latest LLM technology to spot all of these AI-generated attacks. Like, that's kind of fundamentally our business. One of the big blind spots, one of the reasons, you know, our companies are working together, is the invoice, because I can make this invoice perfect, to your point. I can assemble the right vendor, the right company, the right names. All of this can be perfect. What I have on my side that's unique is I understand the email origin, provenance, assembly. I've got a lot of sophistication on how I look at emails. And then Sardine has a lot of sophistication on how they look at the transaction and the account number and things like that. So that's why we're literally, again, just putting those two pieces together to fight this new wave of fraud, because otherwise, I think you can slip through the cracks, again, by putting, you know.

I require both parts of that trail to be assembled perfectly to attack me. If I only require one or the other, it becomes like five, six, seven times easier, 'cause I only gotta fool one side of the house. Versus now you gotta fool Aegis and Sardine to get past us. I think that's a really powerful combination that, again, that gift just keeps on giving, and that porous barrier between security and fraud keeps on happening. By the way, in this use case too, by the way, it's both a fraud and a security use case, because email security is traditionally owned by security. When that gets breached.

It lands on the desk of an ACH fraud team or accounts payable fraud team. And so basically they're just downstream of security. So they're, again, completely married, and should be using each other's tools and data to influence each other, especially because, you know, and we talked about a lot of our friends at CrowdStrike and SentinelOne in this space. By the time it gets to the EDR, it's already a mess. Like, it's already on the device. A bad thing has happened. By the time there's a fraud control alert, like there is a mess to be cleaned up. You're right - I only gave you through (27:52). Here's the rest of the transcript, continuing from (27:55) to the end.

Yeah.

Yeah.

If I can take telemetry from fraud and push it forward in the funnel and act faster, I can keep that mess off your desk, and you can just look at that and be like, that would have been bad, it's good we learned from it, and then move on with life, which is really what we want. Because, again, it started with both fraud teams and security teams are perpetually too busy. I've never met a head of fraud or head of security who's like, I don't have enough stuff to do. They all have way too much stuff to do, way too much risk to deal with, and they don't have comfort with their posture, because there's so much out there. If we can take one more thing off their plate.

Yeah.

Take an incident and make it an alert, I'm delighted to do that.

Absolutely. I agree with this notion, or this strategy, so much. It's just so much more efficient and clean to deal with these threats before they actually manifest, and especially before they monetize and exit the funds from your platform. And then, you know, yeah, I mean, you can start chasing the money, and you can even get it back, but it will cost you quite a lot. So, yeah.

Yeah, yeah. It's very tough. You know, we talk a lot with our friends at the local FBI office. We talk about that all the time. They're like, look, we get hundreds of thousands of reports, I can chase one percent of the really big ones. And even then, they're gonna help, and they're great, but they're gonna have one or two agents chase them through an ACH network and try and get it back. You know, it's like one percent of one percent of one percent is kind of your level of getting that back once that transaction's happened.

Yeah.

Yeah, I mean, if you're at the point where you need the FBI, you're not in a good spot. So tell me, you started speaking about Aegis, but I want to kind of go back one step. Tell us about, you've done reCAPTCHA, what made you think about Aegis, the idea? Because, as we kind of talked about a bit at the earlier part of the conversation, things have radically changed in the last couple of years. And both how fraudsters, and let's say malicious cyber criminals, behave has really changed, really shifted. So what's the idea behind Aegis? How do you see that?

Yeah, I mean, really, you know, all of us here spent about a decade at Google looking at these problems. We observed a couple things. And, again, very similar to the fraud world. One is that the attacks are accelerating, and they're becoming unique, and they're using more and more sophisticated tactics. And it used to be, in fraud and security, you see an attack, you write a rule, you configure it in a big rules engine. That's just traditionally how it worked for a long, long time. But as the attacks become more unique, you can't keep up by writing rules. You just can't physically do it. And then at some point you get this rules engine that has like a hundred thousand rules. At Google, we had a 10,000 rule-based engine. It's too big, it's too unmanageable. And look, if there's a false positive, good luck figuring out why it happened, right? You become the navigator of the rule maze, which is a losing battle. And so I said, look, this is where it's going, this is the future. How do we get in front of it? And we really get in front of it by saying, let's replace rules with LLMs. Which, again, very similar. I think me and Soups have a very similar premise of how we think about our similar worlds, is like, great, attacks are more diverse, more sophisticated, they're using more pieces. What's really good at that? LLMs and unsupervised AI, right? So let's apply that to our problem in our respective places. And that's, I think, worked really well. You know, we started this company, we kind of had, I'll show a couple of slides here. We had, or a couple pieces of data, more importantly. We had a trend where we saw the attacks were kind of increasing at a rate of about 4,000% since ChatGPT came out. Just the volume was going up. And that was before we started. More recently, we presented at, you know, a conference as well. It's like the success rates of bypassing controls are really, really scary. So we've seen kind of the AI spear phishing attacks, these N of one attacks generated by AI, be almost twice as effective as traditional attacks at bypassing

Say.

Existing detection systems. In this case, it's existing Google, Microsoft, point solution email security tools. Because they will flat out recognize and know the rule you wrote, and evade it. They just kind of quickly sidestep it. And it's easy. And they don't even have to do that intelligently. They can also make 50 attempts and see which one gets through, right? So they kind of understand that fuzzing attack. The second thing we've seen that's been a big evolution over the last eight months.

Yeah.

Is the percentage of these AI-powered attacks has gone from 2% to 15% over a very short period. And that kind of tells us that adversaries are coming up the S curve and making more and more dangerous content at higher and higher volume. So I expect actually both the percentage bypass rate as well as the volume of these attacks to rapidly escalate, essentially as adversaries adopt AI. And, you know, we're all trying to adopt AI too. We talked at one of the last fraud conferences, with the Sardine team and an audience, about how difficult it was, in some cases, for fraud teams to adopt AI tools internally, because there's compliance, there's procurement, there's a whole process they have to go through. Adversaries don't have any of that. Man, they wanna drop the latest DeepSeek model, OpenAI, or Claude, and give it a shot. Like, they don't care about your data. They're just gonna give it a try and see what happens. And so they're moving at this rapidly accelerated rate. And so one of the biggest reasons, you know, we had half of our team leave Google to join us, and a bunch of folks

No.

From Facebook and Amazon as well. Like, so much care around this mission is we really believe that the adversaries are well ahead of the defenders here, maybe one, two, three years ahead. And we have to come together as a team, because we care so much about this mission, to act quickly to create an effective defensive layer before this thing really cracks wide open. And we're already seeing the cracks accelerate right now. I'm hearing from customers every day that, like, the email attacks are getting out of control, their existing tools are not stopping it, they don't know what to do about it. And so we're like very, very dedicated to work as hard as we can, as fast as we can, to offer them that layer of control that doesn't really exist in the industry yet.

So you're a founder. That means you're an optimist by nature. So, what's your optimistic view on how to actually put a stop to it? Because the picture that you've painted so far, which, by the way, I totally agree with, is pretty grim.

Yep. Yeah, I think, and look, I am an optimist, right? A little bit crazy. It's always a hard thing to be like, hey, I'm gonna rebuild this thing over time. But look, one thing is I think someone's gotta solve it. So one thing that makes me really happy about this mission is like someone's gotta do it. So it's not a solved problem. Our premise here, though, is really to investigate every attack. So a lot of time all the other controls, and this is also true for historical fraud controls, they kind of look at like a statistical average. They kind of try to blend a bunch of things and look for a pattern, and look for abnormalities, right? It doesn't really work moving forward, again, in these new generative attacks. And so you actually have to investigate every single attack. Like, that's really the core answer here. And so that's really the platform that we've built. And so I can even show you kind of what looks live in the product, not to get too salesy, but, like, it's just cool to kind of see. If you look at this email, traditionally you'd be like, I don't know, we look at the headers, we see, like, what's odd and what's not. But really, you pop open kind of our AI analysis, and what it's really gonna do is it's gonna call a series of agents. And the agent can call a friend. So it investigates the header, and tries to figure out what's going on there. Then the header agent calls reputation, URL inspector, content analyzer, and they'll dig into their respective areas to find risk. Then it goes to understand the context, like, why are they asking for the sensitive data? And eventually can classify intent. Are you using urgency tactics? What are you using to kind of get the user to act? And by sequentially asking these questions, just like any fraud analyst team or any security analyst team would, you can get to the bottom and find, you know, and secure what we call the TTP, or, like, the real tactic that is malicious, the real illegal activity or fraud that's happening, versus just saying, hey, this thing kind of looks odd. And I think that's really the future, where you can go head to head and really understand kind of bypass a lot of these threats. And I'll show you one other thing that's kind of really relevant as an example. A classic rule, and this is from an attack we just saw pretty recently, is that if you are showing something here that's like a classic DocuSign document, is what we're showing on the screen.

Yeah.

You must link to DocuSign. And we're showing, on the other side here, the DOM that does have a DocuSign link as part of the original source tag. The problem is that tag is actually totally dead, not used for anything by the actual rendering DOM. The rendering DOM is going to use this href that's actually malicious, and linking to a malicious URL. And so, in this case, this adversary has completely understood this rule, has satisfied the rule successfully, and delivered a malicious payload past a classic gateway. And this is just one of many examples of how these bypasses are happening. Maybe another kind of notable one, that's one of the scarier ones, is we were the zero-day reporters of the Salesforce breach, where Salesforce was compromised and sending malicious emails out on behalf of their users in the fraud space. You know, Robinhood recently had the exact same issue. And so I don't want to pick on Salesforce and Robinhood, it's happened across 10, 20, 30 large companies that we've seen, and countless smaller companies. So just expect this to happen. Expect the reputation to be high, expect a bunch of the data to look good, you really have to dig deeper and deeper to really find the abnormalities, in order to really understand, you know, what's at risk and what's not. Even to the extent you need to have specialized, you know, agentic crawlers. And I'll kind of show maybe one other quick example here of what the future of this detection can look like, as we kind of race to keep up with the adversaries. I'm gonna switch screens here to something else.

Yeah.

Is we developed what we call Vanguard, which is an agentic browser that is LLM powered. And as you go through, you can launch this thing, it'll go through and kind of open a page. And one of the biggest forms of attacks that we see every day is something that requires human interaction. Going back to kind of the CAPTCHA story, right? Where it's like, if I have a human doing interaction, I can get around most crawlers, which is which is true. 99% of the time it's actually true. And so, in this case, an adversary has put a password in an email and said, hey, Trent, here's the, you know, here's a patch in the email, please open the email, put it in the document, open the document, right? That's the classic pattern. Now our browser is able to read that email, understand that, it's able to input that password, open the document, find the risky, in this case, QR code, scan the QR code, and find a risk. It can also do things, if I skip ahead, like going back to all the CAPTCHA days, it can read a CAPTCHA, it can solve and pass a CAPTCHA to recognize, and this is a Google phishing page. So there's all kinds of things now that are possible, where we're using AI to combat AI. And I think you have to use these AI-native methods, in order to spot and combat this future risk that is already in the market. And I think it's just accelerating at a rate where it's a 10%, 20% problem now. But guess what, adversaries, once they discover what works, they're just gonna turn that crank up really quickly. And so I think within a year, it's gonna be an 80, 90% problem.

Yeah.

Yeah.

And by that time you're gonna see a bunch of compromises before the defensive layers put in, and that I want to avoid for as many people as we can, before we get and help the people who've been compromised.

Yeah.

Yeah, I'll say, on a side note, that I'm very, like, what's, like, what I'm very much curious to see is how the fraudsters would react to that point in time where AI tokens would not be subsidized anymore, and that would mess around with their ROI. So that would be interesting. But putting that aside, I want to say, I mean, look, I fully agree with the fact that AI, and specifically agentic AI, is probably the most exciting technology that has entered, I mean, speaking from the fraud prevention kind of perspective, our space, for the past probably like 15 years or so. I think, by the way, it is also interesting, and you mentioned that, you said fight AI with AI. I think that's the first time that both sides, both us the defenders, and the adversaries, are literally using the same tech. I think it's the first time that it ever happened. Very interesting. But, you know, in the end, we know that, you know, AI has a lot of limitations, and has a lot of quirks, because all of us are very new to them, and none of us are like very, very kind of, you know, like seasoned experts in how to kind of roll out an AI-based product, it's, you know, it takes time, and it takes a lot of ironing all the kinks out. And I'm wondering, like, what have you seen? What are the challenges and the limitations that, you know, I would ask it in this way. I would assume that a lot of the listeners right now are either thinking about building something with AI, or acquiring, like buying some vendor that promises AI-based capabilities. And, like, what would be the warning signs, or the pitfalls, or the questions that you would highlight, that, like, based on your experience, teams should ask.

Yeah, it's a really good question, and a hard one. So let me start with buy, and then we'll go to build. I think with buy, I'll just start by saying, like, first of all, pierce through the noise. There are so many, like, every single vendor says they know and are doing AI. Almost all of them have to. I think their board will force them to, right? The percentage that are actually, that know anything about AI at depth is actually sub-one percent. I think that's actually one reason I mentioned we hire from Google and Amazon, larger companies, is like, first of all, I've been doing this stuff, NLP, for 20 years. Like, I've been in the trenches, in the Stone Age, banging rocks together to get this done for a long, long time, with a lot of struggle. So, you know, I've kind of lived these wars. And my team at Google's been doing this since 2017. So they all have about a decade of experience fighting fraud with transformer models. Like, they've got the reps. I would bet on teams, again, I'm biased, but, like, we've seen this work incredibly well, that have the experience, and they are out there. You just gotta pierce through all the marketing noise, and get to those teams. And they may not be as good at marketing, because they're gonna be really technical, deep AI researchers, threat experts. Like, go to the people with real content.

Give me one question. How do I pierce through the noise?

Yeah, I mean, a really good question is, like, how do your models work? And I think a really bad answer is, like, I don't know, we call OpenAI, or we call frontier model, not to pick it up, just like, we call frontier model. That's gonna be undifferentiated. Tell them that we don't really know what we're doing. So what I showed you in that demo is, like, hey, look, like, we built all of our own AI, and custom SLMs and LLMs. Here's how they reason together. Here's how we apply it to the problem. And there should be no limited depth if you get on a call with that technical team. On how deep they can go to kind of talk about that. So that's kind of one. Two is, in that depth, and part of your TPRM and security review process, they're gonna ask, what do you do with my data? If you're using a frontier model, they're like, well, I guarantee all these things, except I'm gonna ship all your data off to a third party, and then all bets are off. We've fully separated the inference, the training layer. So I can tell you everything at the top level inference read level is read-only. We don't do anything with that. Everything that we train on is stored in a separate, segregated data store that you have read-write access to. So you've got full transparency all the time. And you should expect from all vendors that basic, and we do more, but that basic level of transparency and separation, it shouldn't be, we're just gonna train on whatever you send us. That's not an acceptable answer. So that all again goes to depth. And then third is, look for efficacy. You know, there's two classic things you look for, it's false positives and false negatives. Roughly speaking, false negatives, how many things you miss, false positives, or how many good things that you caught. And, again, fraud, it's like, how many legitimate transactions are you blocking? How many illegitimate transactions are you missing? Look for that data, do that A/B test. But I think you need all three of those things together. You look to make sure they actually understand what is happening in their models, and are not just, like, tossing it off on a third party. You need to understand they know how to respect your data, and give you transparency on what is happening, and control over your data.

Yeah.

And then third is, are very clear and very scientific about how they A/B test things. And I'll say something that, for us, like, we are constantly A/B testing our models every single day, and competing with our own models to make them better. That's just how we think about the world, because we know adversaries are doing this, we know models are getting better, we know this is the motion we have to do. And if a team is not crisp about how they're doing that, they're not gonna be able to evolve, and their product probably doesn't even work in the first place. Because I don't know how you got there without a good scientific approach of doing that testing. It's not magical, right? It's gotta have boundaries. It's gotta evolve like a tool. And so I look for those three things as you buy. Maybe moving to build, which is even harder, that's a hard one. I would say one thing that I would really encourage teams to do more, particularly in the fraud world, is do on-prem deployments, especially for custom tooling. I think a lot of times they get blocked by trying to figure out if a third party tool is safe, and then how to integrate into their environment.

Yeah.

I think there's a lot of cases where you can deploy your own model in a totally closed environment, and see what happens in a very safe way. 'Cause you're just kind of running analysis out of that. I think that would actually unblock a lot. So imagine you set up your own server, you deploy Llama or Gemma to it, and you just roll code against that. I think most data analysts are gonna be successful at doing that. And it unlocks a lot of the TPRM fear you have of bringing vendors, which is all justified, and allows you to iterate rapidly internally. I think that's one. I think two is, think about convergence of the data lake and the kill chain. I'll flash a slide. I'm gonna have to share it, you know, maybe as a takeaway for this as well, but I think it's really important. What did I do here? Clicking the wrong buttons. Is to look at the hyperconvergence of the data layer. So I think right now, even within fraud, there's a lot of fraud data that lives in separate vertical silos. Try and break all of that down, and put it in one shared data lake. And, to your point, let's look at that between security and fraud. Let's put it all in the same data lake.

Always.

And you can operate with an application, or MCP server, whatever you want, across that data. So break open kind of that data within your infrastructure. I think those are probably two. I'll skip over, maybe, of course, maybe my last one, that's like very trivial, is like, use AI tools whenever you can, like, even if it's on your phone, even if it's, like, picking what to eat at a restaurant. Like, sometimes a lot of the fraud audience has not yet gotten into AI, it feels really scary, and don't have familiarity with it. Like, the two things I love to recommend if you've not gotten comfortable with AI, is like, the next time you're at a restaurant trying to figure out what to eat, or what bottle of wine to order, take a picture of that menu, upload it to your favorite chat app, GPT, Gemini, Claude, whatever it is, and ask it to make a selection for you, and see how magical that process is. Something like everyone can do. And I find it's a great way for people to kind of open the aperture to, like, really understand what AI is, and then they can go use it on their computer, and then they can use Claude Code and Skills, and go deeper. But oftentimes that first experience to kind of, you know, get your hands in it and see the value, is the most important. So maybe that's the third one. If you're not yet comfortable with AI, this is a great way to do something really small and simple, even in your personal life, where there's no risk, and just, like, apply it. And the only risk is like the glass of wine you drink, or the meal you eat. That's your risk here.

I, on one hand, I completely agree, and I think that's a prerequisite to life, basically, from now on. On the other hand, I like, I don't know what you think about it, but I feel that the fact that a lot of folks are now exposed to technology in a very kind of, you know, like direct way, means that, I mean, my experience is that folks tend to attribute completely magical powers and capabilities to AI, without really understanding how to not test it, but how to kind of, you know, like think critically about what is the input that they've put in, and what was the output that they've gotten out. And I think that actually, you know, we hear a lot of talk about none of us having jobs in two years, or five years, or whatever time, and I think a lot of that actually is being driven by the fact that a lot of people are exposed to this, but they don't see the limitations. They don't see the hiccups.

Yep. No, it's true. I always say it's, you know, it's not a magic wand, it's a tool. And, to your point, you want to understand the tool's constraints, its benefits, what is it good at, what is it not good at, in order to apply the tool to the right problem set. It's funny, actually, this is the number one thing I try and have my MBA students walk away from the class with, is like a baseline understanding of, like, what's the size and shape of this tool? Where can I use the tool? Where should I not use the tool, based on its current capabilities? And then this hands-on feeling, to your point, is gonna give you a hands-on view of what that is. And I encourage you to do this frequently, because, guess what, this is a tool, but it's a tool that's evolving its capabilities on a monthly basis. So you actually need to be hands-on with that tool, to feel it evolve month by month, because that framework in your head is gonna evolve with it.

Yeah, love it. I love it. Had the same conversation with Matt Vega as well, so, yeah, I totally agree. Awesome.

Me and Matt love talking about this. We're gonna do like a regular podcast together, just to, like, shoot the shit.

Who doesn't love talking about AI nowadays, right? Except for my mother.

And, by the way, even my mother, you, I do it for my mother, because my mother's, you know, eyesight's gotten a little worse over time. I have her on the voice assistant now. So, rather than trying to read her phone, so the phone text bigger and bigger and bigger over time, she can just talk to the agent. So I even think, like, that's been a huge, a huge value, where you can, like, raise accessibility. It's doing so many cool things there. Even for the, don't think of themselves as technologists. Like, I think my mom has never, like, truly adopted the iPhone. But with the conversation, it's like, she's an amazing communicator verbally, so just do that.

Yeah.

Yeah, I wish I could say the same about my parents, but they'll get there. They'll get there. I think that we can take a lot from this conversation that we just had over the last hour or so. I just want to repeat some of the things that, you know, I wrote down while you were speaking. So we started with, you know, talking about fraud prevention and cybersecurity, and one of the

It'll take time.

Main takeaways that I take is that fraud teams should, you kind of you dubbed it as toys, should look at the toys that the cybersecurity team in their company plays with, and basically see if they can, you know, gain some value out of these toys. Fully agree with that. We talked about three signals that help separate the bad actors from the legitimate users. We talked about the fact that bad actors would actually usually

We talked about the fact that bad actors would usually interact very cleanly with your product. And if you see an unclean interaction, that actually might, or might not, it probably means that this is a good user. We talked about the second signal, where we said good people leave tracks. We repeated it a couple of times throughout the conversation. The lack of

Mm-hmm.

I love that thought, by the way. It's a great, it's a great metaphor.

It's old fraud sciences, PayPal, yeah, proverb. The lack, the lack of data, the lack of history, basically means that there's already something sus here. And we talked about, obviously, you know, given that you have enough data, anomaly detection, and the fact that, you know, you have a very small subset of the population that just looks differently, that is already also

Mm-hmm.

A signal that you want to look at. There was one other thing that you mentioned, which I really liked, basically tracking how much it costs to break your product, or to kind of steal from your platform. That is super interesting. I think having a dashboard with such a KPI, and tracking that, and maybe even that being a goal for your team, can be.

Yeah.

A very interesting way of managing fraudsters' ROI.

Yep. By the way, that, we literally had that dashboard, and that literally was a goal for our team. So we'd be like, it's, you know, we track that like a stock price. Yep. So you're right on. Yeah.

Awesome. Okay. Awesome. Awesome. Awesome. I love it. Love it. We talked about trying to, as much as you can, upstream your detection. So when you stop something bad from happening, it is still, or at least the funds are still in your system. Not only that, you know, it's less expensive, but also it's much more efficient, because trying to, you know, like reverse time, or, you know, like basically once milk has been spilled, there's so much you can do about it, right? So trying to go upstream as much as you can. And then we talked about, you know, like adopting and implementing AI technology, and how to go about it. So we talked about some helpful tips for teams who buy, and we talked about a couple of things that you want to look at. You want to obviously make sure that the data stays your data, and is not shipped to China or something like that. We, you want to make sure that you are able to do A/B testing or a POC. That's kind of okay, that's a no-brainer for fraud prevention teams. And thirdly, I thought found it very interesting, you want to basically make sure that you are not being sold a wrapper, where the technology is basically owned by Anthropic or OpenAI, because, I mean, what are you actually buying at that point?

Totally. You might as well just buy OpenAI or Anthropic directly. Just go direct if that's all it is.

Yeah, yeah, and good luck with that as well.

Then we talked about teams who build. We talked about deploying locally, fully agree, much easier, especially in, you know, regulated environments like banks and fintechs, much much easier. We talked about, obviously, data lake access, that's a given one. And, lastly, and I think this is kind of, you know, a general

Yes.

Tip, or a general takeaway, for everybody, including our mothers, just, you know, just get more comfortable with AI. That's going to be what we're all going to use for work, life, every day. Yeah. Cy, super, super interesting. I want to thank you. I'm sure that a lot of our listeners got a lot of very interesting insights out of this conversation.

Chen, I really appreciate it. It was always a pleasure to talk with you, and always pleasure to partner with Sardine too. So we're excited to do more together, excited to be joined together as fellow fraud fighters, and anything I can do to help folks in the audience fight fraud, I'm always down.

Right on, man. I love it. Awesome. Folks, thank you very much for joining us today, and I see you next Saturday. YouTube Description Should Fraud and Cybersecurity Teams Converge? June 13, 2026 Episode #8 Guest: Cy Khormaee Every few years our industry rediscovers the same debate: should fraud and cybersecurity teams actually sit together? And honestly, usually both sides hate the idea immediately. Not because they dislike each other. Mostly because both teams are already overwhelmed and nobody wants another meeting. But over the last couple of years, something changed. The signals started converging. Credential stuffing became account takeover. Account takeover became fraud. Fraud became phishing. Phishing became invoice fraud and ACH fraud. And suddenly the same security telemetry that detects compromised infrastructure also helps identify fraudulent users before they ever reach checkout. That is where things start getting weird. In this episode, I sat down with Cy Khormaee, who helped build Recaptcha at Google and now runs Aegis AI, to talk about why AI phishing detection is forcing fraud and cybersecurity teams closer together whether they like it or not. And honestly, once you realize the same behavioral signals can stop both account takeover and payment fraud detection, the organizational separation starts feeling a little artificial. We get into AI email security, AI-powered fraud, fraudster ROI, upstream fraud detection, and why modern attackers are moving faster than most enterprise security stacks were designed for. Also, I learned that Google literally tracked the market price of breaking CAPTCHA systems like a stock ticker. Which honestly feels extremely fraud-brained. What you'll hear in this episode: ● A practical look at why fraud and cybersecurity teams are starting to share the same signals ● How credential stuffing and account takeover pushed security tools into fraud prevention use cases ● Why AI phishing detection depends on more than static email rules or reputation checks ● How AI email security is changing as attackers use AI to generate more targeted phishing attacks ● Where invoice fraud, ACH fraud, and accounts payable fraud sit between security and fraud operations ● Why security telemetry and fraud telemetry become more useful when teams connect the full user journey ● How Recaptcha evolved from image puzzles into behavioral detection and fraud prevention infrastructure ● Why "good people leave tracks" still applies across both fraud and security signals ● How upstream fraud detection helps stop problems before money leaves the platform ● Why fraudster ROI is one of the most useful ways to think about modern defense ● What teams should ask vendors before buying AI-powered fraud or AI security tools Expect a conversation about tools, signals, attacker economics, and the awkward reality that fraud and security may already be converging, whether the org chart admits it or not. Who should listen: ● Fraud leaders and fraud analysts ● Cybersecurity professionals ● Trust and safety teams ● FinTech fraud prevention teams ● Email security teams ● Accounts payable and payment risk teams ● Teams evaluating AI phishing detection or AI email security vendors ● Anyone working on credential stuffing, account takeover, invoice fraud, ACH fraud, or upstream fraud detection Basically, if your fraud team and cybersecurity team only meet during incident review, this one may be worth playing in both rooms. Episode notes: (Accordion dropdown) This conversation starts with ransomware in a classroom, which is funny for about three seconds and then immediately becomes the point. Honestly, for a cybersecurity professor, that is either terrible timing or very strong teaching material. Maybe both. It also happens to be a clean way into the actual topic: what happens when security stops being theoretical and fraud stops staying neatly inside the fraud team? Cy and I talk about the overlap from a practical angle. Not "should the org chart change?" in some consulting-deck way. More like: are both teams already looking at the same attacker behavior, identity patterns, and telemetry? A lot of the answer is yes. We get into Recaptcha, account takeover, phishing, invoice fraud, ACH fraud, and why the line between security and fraud keeps getting blurrier. Cy also explains why AI-generated attacks are making the old rule-based playbook harder to trust, especially when adversaries can adapt faster than most enterprise systems. The part I think fraud teams should pay attention to is the operational one: how far upstream can you detect the problem before it turns into money movement, cleanup, and a very uncomfortable incident review? We also talk about what to ask vendors claiming they have AI, including how their models work, what happens to your data, and whether AI model testing is actually happening or just being implied in a slide deck. Because "we use AI" is not an answer. Not anymore. Key takeaway: Fraud and cybersecurity do not need to become the same team tomorrow. But they do need to stop pretending their signals live in separate universes. AI phishing detection, AI email security, fraud telemetry, security telemetry, account takeover detection, and upstream fraud detection are all starting to point at the same thing: the attack path is connected, so the defense probably has to be connected too. The goal is not perfect security. That does not exist. The goal is to make the attacker's business model worse. Less glamorous than "stop all fraud." Probably more realistic. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. (Accordion dropdown close) Transcript > (Accordion dropdown) Optimized Episode Title: Fraud False Positives: Measuring The Mistakes Fraud Systems Hide Episode Title: False Positives Masterclass: How to measure FPs in systems that hide them Meta Title: False Positive Fraud Detection And Risk Signals Meta Description: False positive fraud detection hides good customers in declined traffic. Leaving fraud teams to find the real loss with incomplete data and weak labels... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/false-positive-fraud-detection Keywords Primary Keyword: false positive fraud detection Secondary Keywords: false positive rate false positive reduction fraud rules simulation fraud decisioning fraud prevention strategy fraud risk management declined transactions manual review control groups user feedback entity resolution transaction monitoring payment fraud detection chargeback data fraud analytics fraud prevention tools False Positives Masterclass: How To Measure FPs In Systems That Hide Them Release Date Episode #7 Honestly, most fraud teams have no idea how many good users they are actually blocking. Ask someone for their chargeback data and you'll usually get a very precise answer. Ask how many legitimate customers were declined by mistake and suddenly things get a lot less scientific. Usually somewhere between a shrug and "probably not many." Not a great sign. False positive fraud detection is fundamentally difficult, not because fraud teams do not care, but because fraud systems are often designed in ways that make false positives invisible by default. If you approve a transaction, the system gets feedback. Fraud turns into chargebacks. Legitimate users come back and transact again. But when you block someone, the signal disappears. The complaint gets buried in a support queue. The customer never retries. The event never becomes a label. And suddenly your fraud analytics pipeline has no idea the mistake even happened. That is really the core problem this episode explores. More specifically, how fraud teams can start measuring false positive rates using imperfect but practical approaches like fraud rules simulation, manual review, entity resolution, control groups, transaction monitoring, and user feedback. Before you can reduce false positives, you first need to prove they exist. What you'll hear in this episode: ● Why false positive fraud detection is difficult in systems built around incomplete feedback loops ● How declined transactions disappear from fraud analytics and model training data ● Why chargeback data is easier to measure than blocked legitimate users ● A breakdown of fraud rules simulation and where simulation fails operationally ● How manual review helps identify hidden false positives inside payment fraud detection systems ● Why entity resolution becomes one of the strongest tools for linking blocked users to later legitimate behavior ● How control groups expose hidden weaknesses in fraud decisioning systems ● Where user feedback loops can help, and where they become dangerous ● Why fraud prevention strategy depends on understanding false positive reduction at the operational level ● How fraud risk management changes once teams understand where false positives actually come from A conversation about fraud systems, hidden mistakes, operational blind spots, and why measuring false positives is mostly an exercise in triangulation rather than certainty. Who should listen: ● Fraud leaders and fraud analysts ● Risk and compliance teams ● Fraud operations managers ● FinTech fraud prevention teams ● Payment fraud detection professionals ● Teams managing fraud decisioning systems ● Data science and fraud analytics teams ● Anyone responsible for transaction monitoring, fraud prevention tools, or false positive reduction Basically, if you have ever looked at your fraud system and wondered whether you are blocking more good users than you realize, this episode is for you. Honestly, the answer is probably yes. Episode notes: (Accordion dropdown) Visibility Fraud systems are very good at measuring confirmed fraud. They are much worse at measuring legitimate customers who disappear after getting blocked. And that creates a strange problem operationally: fraud teams often optimize around what they can see while ignoring the losses hidden inside declined transactions. So I walk through several practical methods fraud teams use to estimate false positive rate, including fraud rules simulation, manual review, control groups, user feedback, and entity resolution. None of them are perfect. That is kind of the point. Triangulation You combine several incomplete signals until the picture becomes good enough for decision-making. The conversation also gets into fraud decisioning layers, model thresholds, transaction monitoring, operational friction, and why fraud prevention tools often create blind spots the teams themselves cannot fully measure. And honestly, once you realize how many false positives never become structured labels, you start understanding why so many fraud teams underestimate the problem in the first place. Key takeaway: Zero false positives is a myth. The real goal is understanding where false positives come from, how much of the problem is actually under your control, and which mistakes are worth fixing operationally. Once fraud teams can finally measure false positive fraud detection properly, they can stop guessing and start making tradeoffs intentionally. Less magical. Probably much more useful. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter.

Everyone agrees false positives are bad, but most teams don't know how to quantify them. Ask a fraud leader for last month's chargeback count, and you will get a precise number. Ask how many good customers were blocked by mistake, and you'll get a shrug, or worse, none. Why? Because the way we build fraud systems makes false positives invisible by design. Here's what I mean. If you approve a transaction, you get feedback. If it was fraud, you'll hear about it. If it was good, you will see the customer come back, spend more, log in again, do normal customer things. But if you block something, you get nothing. Best case, a customer complains, but even then, that complaint will rarely make its way back into your data. It usually dies in a silo, buried in a support ticket, stuck in an inbox, or fading into the memory of a support rep. It almost never becomes a label attached to a specific event that fits into your model training pipelines or powers your dashboards. So before we talk about reducing false positives, we have to talk about labeling them. So how do you measure an error that the system is designed to hide? Let me start with a disappointing truth. There is no silver bullet. Measuring false positives is always an exercise in triangulation. You need to combine several imperfect methods to get a good enough picture for decision making. Let's get down to it. Teams naturally gravitate toward simulation because it's easy. You simply backtest your logic against historical data. Take a rule, or a rule set, or a workflow, and run it against the last six months of traffic. Look at all the events it would have declined, then cross reference them against their final status, legit or fraud. Don't forget to account for fraud maturation, since chargebacks take time to materialize. Recent data is effectively unlabeled. A 30-day buffer is the minimum, and a 90-day window ensures your clean users are actually clean. Any transaction labeled clean that your rule would have blocked is a theoretical false positive. The appeal here is accessibility. You can run this in SQL or your fraud platform without deploying any new infrastructure. But simulation comes with important limitations. You are testing one rule in isolation, but in production that rule lives inside a complex decision stack with different rules, model thresholds, AI agents, manual reviews, third-party decisions, and so on. You also cannot backtest operational friction. Simulation assumes perfect code execution, but misses bugs, integration issues, or the behavior of upstream actors like issuers or processors. All of those can create false positives, and none of them are captured in a backtest. So yes, simulate. It's a good start. Just be aware of its limitations. The second method is older, slower, and much more powerful. You manually review decline traffic. Instead of looking at the logic, you look directly at the impact. You take a sample of blocked events, either across the board or for a specific solution, and you ask an experienced fraud investigator to label them manually. This is essentially the same process you use to design your rules or hunt for new fraud patterns, just applied in reverse. Now, the limitations are obvious. It doesn't scale, it's time consuming, and it's expensive operationally. You also need experienced analysts, and even then, a percentage of events will remain as inconclusive gray cases. On the flip side, because this is an offline audit rather than a live decision, case-specific accuracy matters less. You also don't need to label everything. All you need is a sense of where you stand and where the worst offenders are. After all, it doesn't really matter if you conclude that the rule has an accuracy of 45% or 48%.

The third method is the one I consider the most underrated.

Linking. The idea is simple. In many cases, good users show up multiple times on your platform before or even after they get blocked. They may try again with a different card. They may reattempt onboarding with a different email address. They may transact again from the same device with an IP address you don't find risky. If you can link these events together and see the full sequence, you suddenly get an extremely strong signal that the original decline was a false positive. This linking can be straightforward: same user ID, different card. Same device, different email. Same IP and name, different device. Or it can be fuzzier, like a blocked user followed by a successful registration of their spouse. Once you have the basic infrastructure in place, the method scales beautifully. You simply run over your block population periodically and try to associate them with good events that happen before or after. The catch is that you need that infrastructure. You need to research a heuristic that would perform well. You need some kind of entity resolution solution. You need to be able to pull sequences of events over time. That is non-trivial work, but the ROI is massive. This really enables you to generate a continuous stream of high-precision false positive labels. It won't catch every false positive, but the ones it does catch will be very reliable. The fourth method is one that some might feel uncomfortable with: control groups. You take a randomized sample of traffic, typically around 1%, and whitelist it completely. No rules. No model-based declines. No manual reviews. Nothing. The population sails straight through to completion. Then you simply watch what happens. If your overall system performs well, you will see a meaningful amount of fraud in that control group. If, on the other hand, you notice that the control group's false positive rate is much higher than your expectations, then you know that you need to re-examine your setup. Control groups are powerful because they answer a very direct question: what would have happened if we removed the system entirely? The answer usually isn't pretty to look at, but it is an honest one. This approach, however, is only practical at scale. You need enough traffic that 1% or 2% still gives you statistically meaningful numbers in a timeframe that makes sense. You also need the budget to absorb the loss. It may not seem like much, but if your incoming fraud pressure is 5% and you approve 1% of that, you just created five basis points of loss before even considering chargeback fees. So this is a great tool to have, but it's likely only practical in your situation if your organization is mature and profitable enough. The fifth method, requesting direct user feedback, is a more opportunistic tactic, only available in select contexts. If you're looking at declined transactions on accounts you already know and trust, you can trigger a notification: "We blocked a transaction ending in 1234. Was this you?" But this isn't something you can do across the board. It's unusable at onboarding, when you've never seen the user before. Asking a stranger if they are a fraudster doesn't really make much sense. Additionally, and I cannot stress this enough, do not give fraudsters a way to mark their own attempts as good. If a "yes, it was me" button automatically lifts the block, you haven't built a feedback loop. You've built a back door for fraudsters to find and exploit. But as an additional source of ground truth in very specific situations, such as failed login attempts or contact detail changes, this can be surprisingly helpful. If your conclusion is "none of these methods are perfect," you are correct. That is the nature of the beast. But remember, you're not looking for a single magic metric that tells you the exact number of false positives in your system. You're looking to combine several independent indicators to build both accuracy and scalability. In practice, most teams end up with some combination of simulations for quick rule-by-rule sense checks, manual reviews for high-value flows and top offending solutions, linking as the main automation workflow once the engineering work is done, control groups for large populations where you can tolerate some control losses, and finally, user feedback in narrow, carefully chosen contexts. And if you mix these properly, you will end up with a workable estimate of your false positives, broken down by at least a few important dimensions: product flow, traffic type, decision layer, and so on. And once you can measure the problem, you can finally start asking the right questions.

It's one thing to know that you have a false positive problem. It's another thing to know where it comes from. Is it mostly from rules? Your model threshold? Your human reviewers? A specific integration that corrupts IP addresses on mobile? A third-party payment processor that's hyper-aggressive on a specific region? In the second part of this series, we'll take the false positives you've managed to detect and bucket them by root cause. Wrapping up, let me just say that zero false positives is a myth. But you can still get to a point where every remaining false positive is either consciously accepted or outside of your sphere of influence. And that clarity alone is worth a lot. YouTube Description False Positives Masterclass: How To Measure FPs In Systems That Hide Them Release Date Episode #7 Honestly, most fraud teams have no idea how many good users they are actually blocking. Ask someone for their chargeback data and you'll usually get a very precise answer. Ask how many legitimate customers were declined by mistake and suddenly things get a lot less scientific. Usually somewhere between a shrug and "probably not many." Not a great sign. False positive fraud detection is fundamentally difficult, not because fraud teams do not care, but because fraud systems are often designed in ways that make false positives invisible by default. If you approve a transaction, the system gets feedback. Fraud turns into chargebacks. Legitimate users come back and transact again. But when you block someone, the signal disappears. The complaint gets buried in a support queue. The customer never retries. The event never becomes a label. And suddenly your fraud analytics pipeline has no idea the mistake even happened. That is really the core problem this episode explores. More specifically, how fraud teams can start measuring false positive rates using imperfect but practical approaches like fraud rules simulation, manual review, entity resolution, control groups, transaction monitoring, and user feedback. Before you can reduce false positives, you first need to prove they exist. What you'll hear in this episode: ● Why false positive fraud detection is difficult in systems built around incomplete feedback loops ● How declined transactions disappear from fraud analytics and model training data ● Why chargeback data is easier to measure than blocked legitimate users ● A breakdown of fraud rules simulation and where simulation fails operationally ● How manual review helps identify hidden false positives inside payment fraud detection systems ● Why entity resolution becomes one of the strongest tools for linking blocked users to later legitimate behavior ● How control groups expose hidden weaknesses in fraud decisioning systems ● Where user feedback loops can help, and where they become dangerous ● Why fraud prevention strategy depends on understanding false positive reduction at the operational level ● How fraud risk management changes once teams understand where false positives actually come from A conversation about fraud systems, hidden mistakes, operational blind spots, and why measuring false positives is mostly an exercise in triangulation rather than certainty. Who should listen: ● Fraud leaders and fraud analysts ● Risk and compliance teams ● Fraud operations managers ● FinTech fraud prevention teams ● Payment fraud detection professionals ● Teams managing fraud decisioning systems ● Data science and fraud analytics teams ● Anyone responsible for transaction monitoring, fraud prevention tools, or false positive reduction Basically, if you have ever looked at your fraud system and wondered whether you are blocking more good users than you realize, this episode is for you. Honestly, the answer is probably yes. Episode notes: (Accordion dropdown) Visibility Fraud systems are very good at measuring confirmed fraud. They are much worse at measuring legitimate customers who disappear after getting blocked. And that creates a strange problem operationally: fraud teams often optimize around what they can see while ignoring the losses hidden inside declined transactions. So I walk through several practical methods fraud teams use to estimate false positive rate, including fraud rules simulation, manual review, control groups, user feedback, and entity resolution. None of them are perfect. That is kind of the point. Triangulation You combine several incomplete signals until the picture becomes good enough for decision-making. The conversation also gets into fraud decisioning layers, model thresholds, transaction monitoring, operational friction, and why fraud prevention tools often create blind spots the teams themselves cannot fully measure. And honestly, once you realize how many false positives never become structured labels, you start understanding why so many fraud teams underestimate the problem in the first place. Key takeaway: Zero false positives is a myth. The real goal is understanding where false positives come from, how much of the problem is actually under your control, and which mistakes are worth fixing operationally. Once fraud teams can finally measure false positive fraud detection properly, they can stop guessing and start making tradeoffs intentionally. Less magical. Probably much more useful. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Optimized Episode Title: Social Media Investigation: When Fraud Teams Stalked Facebook Episode Title: I Used to Stalk People on Facebook Meta Title: Social Media Investigation And Fraud Prevention Meta Description: Social media investigation once helped fraud teams follow digital breadcrumbs, but online privacy and AI OSINT tools changed how scams are fought... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/social-media-investigation Keywords Primary Keyword: social media investigation Secondary Keywords: social media OSINT online privacy data privacy public data publicly available information identity intelligence AI OSINT tools OSINT for fraud prevention scam prevention social engineering scams fintech fraud prevention customer privacy digital investigation open source intelligence techniques I Used to Stalk People on Facebook Release Date Episode #6 Back in 2009, when I started working in fraud prevention at PayPal, we had this saying: "Good people leave tracks." And honestly, that was kind of the whole job. Fraudsters tried to erase themselves. Fake identities, disposable emails, wiped browser cookies, brand-new accounts. Legitimate users, meanwhile, usually left digital breadcrumbs everywhere because nobody really thought much about online privacy back then. So yes, part of the job was basically social media investigation. And honestly, I got weirdly good at it. In this episode, I tell the story of how a random Facebook profile picture, a colonial-looking building, and an old backpacking trip through Vietnam helped us approve a transaction that initially looked like obvious fraud. Now, if listening to that story makes you cringe a little, good. It should. The bigger conversation here is not really about Facebook stalking. It is about how fraud prevention changed once online privacy, customer privacy, and data privacy became much more serious priorities across the internet. And now we have this strange tradeoff. As private citizens, most of us are probably happy that publicly available information is harder to access than it was 15 years ago. But as fraud professionals, we also lost a huge amount of visibility that once helped us understand identity intelligence, behavior patterns, and fraud risk. Not a simple problem. What you'll hear in this episode: ● How social media investigation worked inside fraud teams in the early days of fintech fraud prevention ● Why fraud analysts relied heavily on publicly available information and digital breadcrumbs ● A real fraud investigation story involving Facebook, geolocation mismatch, and identity verification ● How online privacy and data privacy reshaped fraud prevention workflows ● Why social media OSINT became harder as platforms tightened customer privacy controls ● How open source intelligence techniques evolved from manual investigation into AI OSINT tools ● Why identity intelligence became more difficult once social networks reduced public visibility ● A practical discussion about OSINT for fraud prevention and its limits today ● How scammers and social engineering scams changed the privacy conversation entirely ● Why fraud fighters may need to rethink their relationship with privacy regulations A conversation that starts with an old-school fraud investigation story that turns into a broader discussion about whether losing access to personal data may have actually protected us in the long run. Who should listen: ● Fraud leaders and fraud investigators ● Trust and safety professionals ● FinTech fraud prevention teams ● Risk and compliance professionals ● OSINT and digital investigation practitioners ● Cybersecurity and identity teams Anyone interested in social media OSINT, online privacy, identity intelligence, or open source intelligence techniques. Basically, if you ever used Facebook like an investigative database, this episode is probably going to make you a little uncomfortable. Episode notes: (Accordion dropdown) This episode starts with a fraud case from 2009 that, looking back now, feels slightly insane. At the time, social media investigation was one of the most useful fraud prevention tools we had. Fraud analysts relied heavily on publicly available information, Facebook profiles, geotags, interests, photos, friend networks, and digital breadcrumbs users left behind online. And honestly, it worked. The bigger point, though, is how dramatically online privacy changed the fraud landscape. Today, customer privacy settings, platform restrictions, and data privacy expectations make that same style of investigation much harder. Even with modern AI OSINT tools and better open source intelligence techniques, fraud teams still lost access to huge amounts of behavioral context that once helped explain suspicious activity. But then the conversation flips. Once you think about modern scam prevention and social engineering scams, you start realizing something uncomfortable: if fraudsters still had the same access to public data they had 15 years ago, things could be much worse. So maybe privacy regulations are not just making fraud prevention harder. Maybe they are also giving us a fighting chance. Key takeaway: Fraud prevention used to depend heavily on visibility. The more digital breadcrumbs users left behind, the easier social media investigation became. But as online privacy and customer privacy evolved, fraud teams lost access to many of the signals that once helped validate trust and identity. At the same time, scams evolved too. And that leaves fraud professionals in a strange position: frustrated by privacy limitations while also quietly grateful those same protections exist. Less convenient for investigators. Probably safer for everyone else. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. YouTube Description I Used to Stalk People on Facebook Release Date Episode #6 Back in 2009, when I started working in fraud prevention at PayPal, we had this saying: "Good people leave tracks." And honestly, that was kind of the whole job. Fraudsters tried to erase themselves. Fake identities, disposable emails, wiped browser cookies, brand-new accounts. Legitimate users, meanwhile, usually left digital breadcrumbs everywhere because nobody really thought much about online privacy back then. So yes, part of the job was basically social media investigation. And honestly, I got weirdly good at it. In this episode, I tell the story of how a random Facebook profile picture, a colonial-looking building, and an old backpacking trip through Vietnam helped us approve a transaction that initially looked like obvious fraud. Now, if listening to that story makes you cringe a little, good. It should. The bigger conversation here is not really about Facebook stalking. It is about how fraud prevention changed once online privacy, customer privacy, and data privacy became much more serious priorities across the internet. And now we have this strange tradeoff. As private citizens, most of us are probably happy that publicly available information is harder to access than it was 15 years ago. But as fraud professionals, we also lost a huge amount of visibility that once helped us understand identity intelligence, behavior patterns, and fraud risk. Not a simple problem. What you'll hear in this episode: ● How social media investigation worked inside fraud teams in the early days of fintech fraud prevention ● Why fraud analysts relied heavily on publicly available information and digital breadcrumbs ● A real fraud investigation story involving Facebook, geolocation mismatch, and identity verification ● How online privacy and data privacy reshaped fraud prevention workflows ● Why social media OSINT became harder as platforms tightened customer privacy controls ● How open source intelligence techniques evolved from manual investigation into AI OSINT tools ● Why identity intelligence became more difficult once social networks reduced public visibility ● A practical discussion about OSINT for fraud prevention and its limits today ● How scammers and social engineering scams changed the privacy conversation entirely ● Why fraud fighters may need to rethink their relationship with privacy regulations A conversation that starts with an old-school fraud investigation story that turns into a broader discussion about whether losing access to personal data may have actually protected us in the long run. Who should listen: ● Fraud leaders and fraud investigators ● Trust and safety professionals ● FinTech fraud prevention teams ● Risk and compliance professionals ● OSINT and digital investigation practitioners ● Cybersecurity and identity teams Anyone interested in social media OSINT, online privacy, identity intelligence, or open source intelligence techniques. Basically, if you ever used Facebook like an investigative database, this episode is probably going to make you a little uncomfortable. Episode notes: (Accordion dropdown) This episode starts with a fraud case from 2009 that, looking back now, feels slightly insane. At the time, social media investigation was one of the most useful fraud prevention tools we had. Fraud analysts relied heavily on publicly available information, Facebook profiles, geotags, interests, photos, friend networks, and digital breadcrumbs users left behind online. And honestly, it worked. The bigger point, though, is how dramatically online privacy changed the fraud landscape. Today, customer privacy settings, platform restrictions, and data privacy expectations make that same style of investigation much harder. Even with modern AI OSINT tools and better open source intelligence techniques, fraud teams still lost access to huge amounts of behavioral context that once helped explain suspicious activity. But then the conversation flips. Once you think about modern scam prevention and social engineering scams, you start realizing something uncomfortable: if fraudsters still had the same access to public data they had 15 years ago, things could be much worse. So maybe privacy regulations are not just making fraud prevention harder. Maybe they are also giving us a fighting chance. Key takeaway: Fraud prevention used to depend heavily on visibility. The more digital breadcrumbs users left behind, the easier social media investigation became. But as online privacy and customer privacy evolved, fraud teams lost access to many of the signals that once helped validate trust and identity. At the same time, scams evolved too. And that leaves fraud professionals in a strange position: frustrated by privacy limitations while also quietly grateful those same protections exist. Less convenient for investigators. Probably safer for everyone else. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter.

In 2009, I started my fraud prevention journey when I joined PayPal as a fraud analyst. Back then, we had this saying: good people leave tracks. The idea was simple. Fraudsters were trying to cover their own tracks. They used fake identities, signed up with disposable emails, deleted browser cookies. Either way, the identities they used would actually appear as brand new. Legitimate customers, however, never bothered with cleaning the breadcrumbs they left behind. Why would they? Back in 2009, when we reviewed fraud cases, this mainly meant that we stalked people on Facebook. And boy, was I good at it. One day, a colleague of mine was reviewing a case she couldn't decide on. The account and card were American. The IP came from Vietnam, which was a known fraud hub. And the Facebook page linked to the account's email had nothing on it. No geotags on recent pictures. No status updates about traveling there. No liked pages that referred to the item purchased. Basically, nothing that could connect the identity to that return. But there was one thing that caught my eye. The profile picture showed the person standing in front of a colonial-looking villa, surrounded by what looked to me like tropical vegetation. Now, funnily, I happened to backpack through Vietnam just a few years prior to that. I got this strong gut feeling that the picture might have been taken there. So I searched for "colonial building Vietnam" in Google Images. That was before you could do reverse image search.

And lo and behold, the very same building popped up immediately.

That's how we knew the account holder was indeed traveling in Vietnam, and we decided to approve the purchase. Now, why am I telling you this story? First, to brag, of course. Second, I bet you just cringed listening to it. Not because you suddenly remembered writing public statuses on your Facebook wall, that too, probably, but likely because you remembered how easy it once was to violate someone's privacy. And if you forgot, or weren't around, then here's an image of Facebook's advanced search from 2008, just to illustrate the possibilities. But now we're in 2026, and people are more aware of the dangers of oversharing private information on social media. Even more so, the regulatory environment has also changed, and so did the privacy settings platforms like Facebook enforce, especially by default. As a private person and a parent, I'm glad to see these changes. But as a fraud professional, that makes my job harder nowadays. It's much harder to find those same breadcrumbs good users leave behind. Some would argue that you still have very similar capabilities today. There are plenty of vendors out there who collect identity information, including which online services you subscribe to. You can also still use Google search and the limited search functionality that different social media platforms offer. And lately, we've also seen how AI tools make OSINT analysis easy and fast. All of these are great, but privacy settings will still limit your reach, mainly your access to the behavior and social network of the user. Now think about how important those are. Knowing where a person is right now can remove a suspicion of fraud in cases of geo mismatch. Knowing what they like to do for fun can show direct relation to a purchase made. Knowing who their close friends are can shed light on that suspicious P2P transfer. Today it all sounds cringe-worthy, but back in 2009, it was our secret weapon in fighting fraud. So, is privacy obstructing us from fighting fraud? There's another way to look at how our culture evolved around privacy concerns, and that has to do with the rise of scams as the prominent fraud threat of our day. Just imagine what scammers and con artists could achieve if we had the same level of data exposure we had 20 years ago. Now that could make my skin crawl. You see, as fraud fighters, we tend to complain about privacy regulations and how they stand in the way of implementing effective fraud prevention solutions. But maybe it's the other way around. Maybe they give us a fighting chance. Optimized Episode Title: KYC Fraud Prevention: Dark Web Services Can Beat KYC for $150 Episode Title: Dark web services bypass KYC checks for $150 Meta Title: KYC Fraud Prevention Using Layered Fraud Signals Meta Description: KYC fraud prevention now requires layered defenses, device intelligence, behavioral biometrics, and KYC vendor orchestration to stop modern fraud kits... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/kyc-fraud-prevention Keywords Primary Keyword: kyc fraud prevention Secondary Keywords: KYC bypass prevention KYC fraud detection KYC fraud checks KYC vendor orchestration device intelligence behavioral biometrics behavioral signals identity intelligence device telemetry synthetic identity detection post-signup fraud monitoring high-risk user monitoring step-up verification fraud prevention strategy KYC vendor strategy multi-layer fraud defense single point of failure fraudster ROI fintech fraud prevention account opening fraud Dark Web Services Bypass KYC Checks For $150 Release Date Episode #5 A year and a half ago, I wrote that for around 150 bucks, anyone could buy a service on the dark web that bypassed a KYC vendor. People were shocked. Today? Honestly, not so much. Now the threat is cheaper, faster, and harder to spot. Document checks can be bypassed. Selfies can be bypassed. Even 3D liveness checks, the ones that looked unbeatable not that long ago, can be bypassed. Not a good look. So in this episode, I want to talk about what fraud teams actually do next. Because if your KYC fraud prevention strategy still assumes that a clean KYC pass means a clean user, you are already behind. The answer is layering. But not the lazy version where you just buy more KYC vendors and hope one of them saves you. I mean real multi-layer fraud defense: device intelligence, behavioral biometrics, behavioral signals, identity intelligence, device telemetry, post-signup fraud monitoring, and KYC vendor orchestration used in the right sequence. Because a KYC check is a signal. It is not a verdict. What you'll hear in this episode: ● A breakdown of why KYC bypass prevention has become harder as fraud kits get cheaper and more specialized ● Why KYC fraud checks, document checks, selfies, and 3D liveness can no longer carry the whole fraud prevention strategy ● How device intelligence asks different questions than a KYC vendor ● Why behavioral signals and behavioral biometrics can expose what a document check misses ● How identity intelligence helps connect emails, phone numbers, addresses, and documentation into a more cohesive picture ● Why post-signup fraud monitoring and high-risk user monitoring matter after account opening ● How step-up verification can add friction only when the risk actually justifies it ● Why KYC vendor orchestration can be useful for a small, high-risk segment ● How fraudster ROI changes when fraud teams stop relying on a single point of failure A practical conversation about layered fraud defense, operational blind spots, and why modern KYC fraud detection depends on connecting signals instead of trusting one onboarding result. Who should listen: ● Fraud leaders and fraud operators ● Risk and compliance teams ● FinTech teams managing onboarding and account opening fraud ● Trust and safety professionals ● Identity verification and KYC teams ● Teams evaluating behavioral biometrics, device intelligence, and synthetic identity detection Basically, if your fraud stack still depends heavily on one KYC vendor, or if device telemetry is collected but barely used, or if onboarding and transaction monitoring teams are still operating in silos this episode is probably going to feel uncomfortably familiar. Honestly, that stack fails every time eventually. Episode notes: (Accordion dropdown) KYC fraud detection is changing Fraud teams need to stop treating KYC checks like a final answer. The issue is not that KYC is useless. The issue is that fraudsters now have operational kits specifically designed to beat certain onboarding flows. If your entire defense depends on one KYC vendor strategy, you have created a single point of failure. Layering Device intelligence asks different questions than document verification. Behavioral signals ask different questions than identity intelligence. Once you combine those signals with post-signup fraud monitoring, high-risk user monitoring, and step-up verification, you start forcing attackers into a much more difficult position operationally. KYC vendor orchestration Using a second vendor for a very small high-risk segment can actually make economic sense. Key takeaway: Fraud is economics. A $150 bypass kit only works if the math works for the fraudster. Every layer you add is a tax on fraudster ROI. Stack enough of them, and maybe they take their business somewhere else. At least that is the idea. Am I being too optimistic here? Probably. But that is still the game. Accordion dropdown close Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Transcript > (Accordion dropdown)

A year and a half ago, I wrote that for 150 bucks, anyone could buy a service on the dark web that bypassed your KYC vendor. People were shocked. Today, nobody's shocked. It's just another Tuesday. Actually, today it's even worse. The threat got cheaper, faster, and harder to spot. The question then is, what should fraud teams do about it? Today, I want to talk about the word "layering," and how it can mean several things. All of them are worth considering. So let's get the easy part out of the way. Document checks can be bypassed. Selfies can be bypassed. 3D liveness checks, the ones vendors called unbeatable just two years ago, can be bypassed. The going rate is $150 to $600 per verified account, depending on the vendor and how many checks need to be bypassed. The fidelity is good, really good. I've seen examples of fraudsters generating high quality 3D video from faded 2D photos. So if you're still building your fraud strategy on the assumption that a clean KYC pass means a clean user, you're already behind. But that's the part nobody really disputes anymore. The harder question is, now what? And the answer to "how do I stop these kits" is one word: layering. Layering doesn't mean buy more KYC vendors. Layering means introducing different approaches, defenses that ask different questions about the user. Think about it like this. Your KYC vendor asks one set of questions. Does this face match this document? Does this document pass as a genuine one? And so on. Now let's take device intelligence as an example. It asks something completely different. Have we seen this device before? What was it doing? Was the device tampered with? Where was it located? The fraudster who beat the document check doesn't necessarily control the device the way they think they do. Different example: behavioral signals. Does this user act like a human? Typing rhythm, pasting versus typing hesitation. I'll give you another example: identity intelligence. Do the email, phone, and address present a cohesive identity that matches what appears in the documentation? Does it match the device intelligence? By layering different detection signals, we challenge the fraudster to a level of sophistication their tools might struggle to overcome. Now there's also another kind of layering we can resort to, one that has to do with the sequence of our defenses, and specifically, monitoring new accounts and how they behave after sign up. What is the user actually doing? Funding an account at 3am, requesting a payout from a high risk foreign country. If something suspicious surfaces, you should escalate it before you allow them to exit funds from your platform. Additional friction, additional verification, or a manual reviewer who looks at it with human eyes. Now, if you've done all of that, you've covered the basics, and you're starting to look at optimization. In that case, you may want to think of another layering approach that involves orchestration. Here's the thing: once you've layered your defenses, your system can do something most incumbents can't. It can identify a small segment of users who are genuinely high risk, let's say 5% or less of your total onboarding events. That's a population worth spending extra money on. For that segment, what you can do is send those events to a second KYC vendor. Now, I realize it may sound like the opposite of optimization, but hear me out. These KYC bypass kits are designed to attack specific vendors. It's very likely that they would be much less successful against others. So not only do you run two checks, but you also run a check that the fraudster doesn't expect and isn't prepared for. And if you're able to do so quite accurately, again targeting that small, high risk segment, then you can really mess with the fraudster's ROI while keeping your costs relatively low. Now let me tell you what I usually see under the hood when I look at a fintech's stack: one KYC vendor doing all the work, device intelligence that is collected but only used, best case, for multi accounting prevention, and silo teams that manage onboarding and transactions separately. That stack fails every time. Because, let's face it, the economics of fraud are getting better for the attacker every day. A stack that is designed around a single point of failure, and KYC checks are just an example, will eventually meet a kit designed to defeat that specific defense. So if you only remember three things from this video, remember this. One: a KYC check is a signal, not a verdict. A clean pass should raise your confidence in the user, but it shouldn't close the case. Pair it with at least two other approaches before you treat someone as trusted. Two: don't treat the different signals as check boxes you need to tick. Compare identity intelligence to your KYC results, device telemetry to known addresses. It's about cross referencing signals and building a 360-degree, cohesive view of your user. And three: if you've covered the above already, consider vendor orchestration. Get the layered defenses in place first. Then, for that small, high risk segment, the signals identified, send it through a second KYC vendor. That's where the extra cost can earn its keep. Remember, a $150 kit only works if the math works for the fraudster. That means every layer you add is a tax on their ROI stack. Enough of them, and they take their business somewhere else. And that's the whole game. I'll see you in the next one. YouTube Description Dark Web Services Bypass KYC Checks For $150 Release Date Episode #5 A year and a half ago, I wrote that for around 150 bucks, anyone could buy a service on the dark web that bypassed a KYC vendor. People were shocked. Today? Honestly, not so much. Now the threat is cheaper, faster, and harder to spot. Document checks can be bypassed. Selfies can be bypassed. Even 3D liveness checks, the ones that looked unbeatable not that long ago, can be bypassed. Not a good look. So in this episode, I want to talk about what fraud teams actually do next. Because if your KYC fraud prevention strategy still assumes that a clean KYC pass means a clean user, you are already behind. The answer is layering. But not the lazy version where you just buy more KYC vendors and hope one of them saves you. I mean real multi-layer fraud defense: device intelligence, behavioral biometrics, behavioral signals, identity intelligence, device telemetry, post-signup fraud monitoring, and KYC vendor orchestration used in the right sequence. Because a KYC check is a signal. It is not a verdict. What you'll hear in this episode: ● A breakdown of why KYC bypass prevention has become harder as fraud kits get cheaper and more specialized ● Why KYC fraud checks, document checks, selfies, and 3D liveness can no longer carry the whole fraud prevention strategy ● How device intelligence asks different questions than a KYC vendor ● Why behavioral signals and behavioral biometrics can expose what a document check misses ● How identity intelligence helps connect emails, phone numbers, addresses, and documentation into a more cohesive picture ● Why post-signup fraud monitoring and high-risk user monitoring matter after account opening ● How step-up verification can add friction only when the risk actually justifies it ● Why KYC vendor orchestration can be useful for a small, high-risk segment ● How fraudster ROI changes when fraud teams stop relying on a single point of failure A practical conversation about layered fraud defense, operational blind spots, and why modern KYC fraud detection depends on connecting signals instead of trusting one onboarding result. Who should listen: ● Fraud leaders and fraud operators ● Risk and compliance teams ● FinTech teams managing onboarding and account opening fraud ● Trust and safety professionals ● Identity verification and KYC teams ● Teams evaluating behavioral biometrics, device intelligence, and synthetic identity detection Basically, if your fraud stack still depends heavily on one KYC vendor, or if device telemetry is collected but barely used, or if onboarding and transaction monitoring teams are still operating in silos this episode is probably going to feel uncomfortably familiar. Honestly, that stack fails every time eventually. Episode notes: (Accordion dropdown) KYC fraud detection is changing Fraud teams need to stop treating KYC checks like a final answer. The issue is not that KYC is useless. The issue is that fraudsters now have operational kits specifically designed to beat certain onboarding flows. If your entire defense depends on one KYC vendor strategy, you have created a single point of failure. Layering Device intelligence asks different questions than document verification. Behavioral signals ask different questions than identity intelligence. Once you combine those signals with post-signup fraud monitoring, high-risk user monitoring, and step-up verification, you start forcing attackers into a much more difficult position operationally. KYC vendor orchestration Using a second vendor for a very small high-risk segment can actually make economic sense. Key takeaway: Fraud is economics. A $150 bypass kit only works if the math works for the fraudster. Every layer you add is a tax on fraudster ROI. Stack enough of them, and maybe they take their business somewhere else. At least that is the idea. Am I being too optimistic here? Probably. But that is still the game. Accordion dropdown close Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Transcript > (Accordion dropdown) Episode Title: Real-Time Fraud Prevention: Zero to Hero w/ Matt Vega Meta Title: Real-Time Fraud Prevention: Strategy & Scale Meta Description: Real-time fraud prevention from tactical friction to tech stacks and AI readiness, how teams can build simpler systems before money movement creates losses... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/real-time-fraud-prevention Keywords Primary Keyword: real-time fraud prevention Secondary Keywords: real-time fraud detection real-time payment fraud prevention real-time transaction monitoring upstream fraud detection reactive vs proactive fraud prevention fraud prevention strategy fraud prevention tech stack fraud prevention vendor strategy fraud prevention rules engine fraud detection rules fraud prevention guardrails fraud investigation process tactical friction payment risk management fraud risk management AI fraud prevention machine learning fraud detection agentic AI in fraud prevention device intelligence behavioral biometrics account takeover prevention Real-Time Fraud Prevention: Zero to Hero w/ Matt Vega May 16, 2026 Episode #4 Guest: Matt Vega This episode is a bit of a full-circle moment. Years ago, Matt Vega interviewed me on one of my first podcast appearances. And now, somehow, here we are, roles reversed, with Matt joining me for the first full interview episode of The Saturday Fraud Strategist. Honestly, not a bad way to start. In this episode, Matt and I talk about what it actually takes to build real-time fraud prevention from zero. Not the polished vendor version. The real version. The one with hiring decisions, messy processes, fragile fraud prevention tech stacks, disconnected vendors, and systems that look impressive right up until they break. Not a good look. While real-time fraud detection sounds like a technology problem, the conversation goes deeper. We talk about people, process, product, real-time fraud monitoring, tactical friction, fraud prevention guardrails, AI readiness, and why teams need to move upstream before the money is gone. Because once the payment moves, especially in real-time transaction monitoring or real-time payment environments, you are not preventing fraud anymore. You are documenting the damage. What you'll hear in this episode: ● A breakdown of Matt Vega's people, process, and product framework for real-time fraud prevention ● A practical discussion of how to build a fraud prevention strategy from zero ● Insight into hiring for curiosity, trust, flexibility, and actual problem-solving ability ● A conversation about reactive vs proactive fraud prevention in real-time payment environments ● A focused look at upstream fraud detection, tactical friction, and why friction done right can increase trust ● Practical considerations for building a fraud prevention tech stack where vendors, signals, and workflows actually communicate ● A discussion of AI fraud prevention, machine learning fraud detection, and agentic AI in fraud prevention Listeners can expect a conversation that moves from theory to operating reality, and from operating reality to practical decisions fraud teams can actually use. Who should listen: ● Fraud leaders and fraud professionals ● Risk, compliance, and cybersecurity teams ● Fintech, banking, and payments teams ● Product leaders building real-time payment experiences ● Fraud operations teams moving from manual review to automation ● Founders, operators, and executives building fraud prevention programs from scratch Anyone evaluating fraud detection rules, behavioral biometrics, device intelligence, KYC fraud prevention, account takeover prevention, or the best fraud prevention tools for their stack. The discussion is designed for professionals who are committed not only to detecting fraud, but to building systems that can scale without becoming fragile. Episode notes: (Accordion dropdown) This interview with Matt Vega gets into what real-time fraud prevention actually requires when payments move fast, fraud teams are under pressure, and the old "wait for the escalation" model is already too late. We talk about why upstream fraud detection matters, how real-time fraud monitoring and real-time transaction monitoring change the fraud investigation process, and why tactical friction can build trust when it's done right. Matt also gets into the uncomfortable stuff: reactive vs proactive fraud prevention, hiring for curiosity instead of résumé theater, and why a fraud prevention rules engine can still beat a shiny machine learning fraud detection setup when the basics are broken. Honestly, not everything needs to be AI. Strange, I know. We also touch on AI fraud prevention, agentic AI in fraud prevention, fraud prevention guardrails, fraud risk management, payment risk management, and how signals like behavioral biometrics, device intelligence, KYC fraud prevention, and account takeover prevention fit into a practical fraud prevention strategy. Key takeaway: The short version? The best fraud prevention tools do not matter much if your fraud prevention vendor strategy creates a stack that cannot communicate. Real-time fraud prevention is about building systems that act before the loss, support the people making the decisions, and scale without collapsing under their own complexity. Am I being too optimistic? Probably. But that's the conversation. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Accordion dropdown close Transcript > ( Accordion dropdown)

All right, what's up, everybody? And welcome to the first full interview episode of The Saturday Fraud Strategist. I'm super excited to have a very special guest dear to my heart, Chief Fraud Strategist and Chief of Staff to the CEO at Sardine, the man, the legend, Mr. Matt Vega. Matt, it's super great to have you on the show.

Always a pleasure, man. Super, super excited to be here. I really appreciate it.

No, that's awesome. And I know, I don't know if you are aware, but this, I am having a bit of a closure moment here, because I think my first ever podcast, if you remember that, my first ever podcast I appeared, you were the host. So here we are probably like six or seven years later, and switched roles.

I totally forgot about that. But yeah, that was seven years ago, and I tell you, that was actually a great podcast. We got tons of views. I think we had like 10,000 downloads on that podcast. We pumped on that, to be completely honest.

I don't even remember what we talked about. It was probably machine learning. We probably called it AI.

It was. The most funny, like the best clip of my podcast ever was actually a statement that you made, and like everyone was commenting on it. We were arguing about how companies were using Equifax data to train models. And you came back, and you were like, "Dude, I can just go in the dark web and train a model for you off the Equifax data, because it had all been breached." And that one crushed. It was so funny. The way that you said it, like the comeback was so good, was so good.

That's great. A comeback. I have no recollection.

Yeah, great having you on the show. For those listeners that don't know who you are, maybe you can tell us a bit about yourself.

Sure. Yeah. So I have been in the industry for about 17 years. Started off in e-com, primarily by accident. Got a job as an executive assistant, basically, literally started off by accident. But, you know, literally, it's like one of those things where you take a job, but you don't really know what you're getting into. So I fell into fraud. And then I went into military intelligence doing SIGINT, cyber DNI, digital network intelligence. Worked with some three letter agencies, which was a blast. And then I spent many, many years since then in usually three primary pillars that I usually get brought in for. One, it's a new startup, usually a high risk startup, and I have to build everything, right? Build the tech stack, the team, the operation, the processes, everything. I have to build it from zero to 10. Two, they've had major fraud losses, and they need me to come in and basically rebuild their system, right? Their tech stack, their people, their operation, because whatever they implemented is just not working. Or the latter, which actually happens more often than not, they took some significant losses, they overcorrected, and now their revenue has taken a significant impact, right? And so my job is to actually come in and unwind everything that they've done. So those are really, I would say, the primary areas. The last 10 to 15 years of my career has been in that area. I've worked with a ton of different companies. I've launched Major League Baseball's sports and collectibles platform as their head of fraud risk, building out from zero to 10, like we talked about. I got to work with the Fanatics and Topps team, which was awesome. I launched SOR Technologies' fraud team. Worked with Instacart during the pandemic to do what I was telling you about. They had 1,000% growth, but they had 1,000% fraud during the pandemic. From BlueSnap Payments and working with the FTC, which was an interesting story. And then all the way through to Novo Business Banking and building out their entire tech stack and process. And then I was Sardine's first customer. So that's why I know the Sardine team. When they went to market, I basically worked with Suhas to kind of launch the technology, become a design partner. So a lot of Sardine's products are kind of near and dear to my heart, because, you know, I was the direct feedback mechanism to improve those products year over year. And so, yeah, that leads me to today.

That is awesome. That is such a journey. I'm sure that you have a lot of very interesting stories to follow. And I think also, Matt, that you're the perfect person to speak to about today's topic, which is real time fraud prevention. And we titled this topic as like zero to hero. Basically, what do you need to do from, you know, like having nothing in place to, you know, basically crushing it. And, you know, I want to kind of like share how we came about this topic. And I think it was about a month ago or so, where we were both participating in this webinar where we shared our latest report for a minute, and we were kind of like, you know, rushing through the slides and through the talking points, and we had a lot to cover. And I think we really came down to the wire, right? We meant it to be like 45 minutes and 15 minutes of questions. I think we ended up talking about 58 minutes.

Yeah.

And covering a lot of different points, like deepfakes and distributed autonomous campaigns and polymorphic attacks and so on. And then in these last two minutes, there was this guy, and I completely don't remember who it was, so if you're listening, I apologize in advance. And they were, you know, coming up with this question. And I think it was something along the lines of, you know, but you keep talking about real time fraud detection and fraud prevention, but what does that actually mean? I mean, I already got the chargeback, so how can I do something with it in real time? And I remember that the first thing that was going through my mind at that point was like, what are you talking about? I've been doing it for 16 years now. It's always been real time. I'm not the first person to introduce this concept. So like, where is this coming from? And I was almost starting to kind of like give my answer, which was like, what are you talking about? And luckily, luckily, you started. You kind of like cut in and started giving your answer. And I don't remember exactly what it was. I just remember it was like very, very generous. Just after we finished the webinar, and you know, for me, it was already clear that you were gonna be my first guest. And I said, this is what we need to do, like the zero to hero. How do you go about real time fraud prevention? And you're the perfect show.

Yeah. Appreciate it. This will be fun.

Absolutely. So maybe to kick things off, let me ask you this, like, you know, when you're thinking about your mental model, and you know, you did it a couple of times, right? You were starting from nothing in a new place. How do you go about it?

Yeah, so I've had many, many, many lessons learned over the years. And let me say that in order to get to where I am today, I had to fail many times, right? And as fraud fighters and leaders in the industry, you really, like to be a leader, it just means you failed a lot of times, honestly, right? And I have yet to meet anyone that builds a high performance team without having tons of lessons learned and tons of failures to get to that point. And so luckily, you know, the key is, did you learn from it? And are you able to prevent those landmines from being stepped on again in the future, right? And so that's something that you acquire over the course of a long career. So I would say the first piece, you know, from a mental model standpoint, is I have seen the highest performing teams that I have come across, and I'm talking best in the industry, best in class, it comes down to simplicity, as crazy as that sounds. So a lot of leaders really lean into, it's almost like because fraud is so complex, and because the attack vectors are so complex these days, they lean into this highly sophisticated tech stack and highly sophisticated operation, just like tons of process and tons of nuance and tons of deep dive. And I have found that to fail almost every time. Eventually, those will start, like they just unweave, right? And things get missed, and that process is extremely hard to follow, right? And there's just so much nuance. And so really, what it comes down to is like old school business techniques, like the three P's, right? People, process, product. And focusing on those three pillars is critical to building your foundation, right? And if you have those simple fundamentals, right, you can't build a house on sand, right? So if you have really shaky ground, right, and you have all these weird processes, you have a bunch of bureaucracy, you have a bunch of approvals, right, it's just going to fail. I guarantee you, as you start scaling, you're going to have problems every single time. Like, you know, in the military we say, keep it simple, stupid, right? And you really have to do that. And at the end of the day, the more core foundation that you can build around the simplicity and, call it elegance, of your process and product, the more likely you're going to be to succeed. Because as you go throughout your process of building and scaling, you're going to have to add in new technologies. There are going to be those arguments where you're going to have to change the user journey, and you're going to have to put in step ups and all these things. And if you already have a really complex process underneath that, you're just adding more complexity and more layers on top, and you're going to fail. But if you have a really simple, elegant process, product team underneath that, then every time you add to it, it's really not a big deal because you've built it on a very strong foundation, if that makes sense. So that's kind of like my initial zero level mental model.

Super interesting. And I must say that, you know, I very much relate. And it reminds me that I also always say, it's always about the basics, and you always need to go back to basics. That's the only thing that should be interesting, or the only thing that you need to focus on as a fraud leader. I do wonder if, you know, over the past, let's say decade or so, where we've seen a lot of, let's say, trends, and also a lot of development on the fraud side, I do wonder if this simplicity looks different than how it looked 10 years ago, or is it virtually the same things?

No, I think it's different. I think the people are the same. That definitely has not changed, right? And I can kind of give you some examples of those three P's, right, of how I see it. So on the people side, you want to surround yourself with, you want to focus on two key areas, right? One, you want to get rid of your ego and know that you're not an expert in everything. It just doesn't exist. Anyone that says they're an expert in everything, I start scratching my head, because I guarantee you I could find gaps in their knowledge base, right? So because of that, you want to surround yourself with people that can fill your weaknesses, right? So you may be really strong operationally, but you may not be really strong from a data perspective, or vice versa, right? So as you're building your initial team, you want to surround yourself with people that really supplement your skill set very well, right? Not to mention, you want to find people that you can trust, even at the analyst level, even at the junior level. You want to find people that you can trust, that you can rely on, that are going to make good decisions even when you're not there, right? That's the key. And because if you're one of those leaders that, like, doesn't trust your team or your people, what ends up happening is you end up doing 10x more work, and you just can't scale that, right? And it's like, you know, I've heard people say trust but verify everything, right? And it's like, well, that's great, but like you're leading a team of 50, good luck with that, right? There's just no way to do that, right, as you start scaling and building up bigger teams. So, you know, I can give you an example, like my hiring practice. What I look for is, I look for curiosity above all else. Okay, so at the junior level, especially, right? So I'll give you an example, like I will hire a kid that is asking me a lot of questions and shows curiosity. And like, one of the other data points that I use is, like, video gamers tend to be really good. There's like a bunch of little nuances like that. Gamers are really interesting. People that kind of like tend to go to ask why, right? So there's like two personalities. Some people are told something, and they just do it, right. And then some people kind of want to push back and ask why. I personally, and I'm not saying this is the right way, I actually prefer the people that want to push back, that want to know why, that are actually trying to apply pressure, because it's never my way or the highway. Because let me tell you, throughout my career, how many times I put in a process where I've told someone to do something, and they found a much more elegant and faster and cleaner way to do it, and they were much more junior to me, right? So get your ego out of the way. Let people find their own path, right? So the key is giving them the North Star of where they need to go, right, but if they want to take a different route than you to get there, as long as they're arriving at the same finish line, it doesn't really matter, right? As long as they're covering the guardrails, right? You can put in guardrails of like, this is the areas they need to cover. This is what they need to investigate, whatever it may be. At the end of the day, how they arrive there, let them use their own strengths to be able to achieve that, right? And I have built, you know, if you lead with that, right, of that trust, and building out that trust with your team, and giving them the right guidance and making sure that they're key, you'll end up having a situation like where I do, where I have analysts that have followed me from five companies, right? So I have team members. There's one in particular that has been with five companies with me, right? So, like, literally she will follow me from company to company. And I've had managers do the same thing. And so I think that really building that trust is critical. So that's the people side, right? And you really want to lean into, you know, if you're hiring bodies just because you have a need, be careful on that, especially in today's world. Like, I have found that people still weigh education pretty heavily, and I have not found that to be actually valuable in the fraud space. In fact, like some of the best fraud fighters are probably high school dropouts, in my opinion, right? So like I would hire a very curious, very scrappy 18 year old kid out of high school more than I would a Harvard graduate in many cases for most of the positions. Not saying that the Harvard graduate is not scrappy and super hungry and all that, so that also exists. But in general, in general, you know, focus on what actually matters for the role. Like, what kind of personality traits are you looking for? Not so much of who's checking the boxes in your interview process, right?

I fully agree with you, and I think that, you know, trust on one hand, also looking at curiosity as a lead indicator when hiring is something that I also used very, or leaned into very heavily during my career, and I completely agree with everything that you said. My observation, though, is that, you know, I'm like thinking about a fraud leader that is now, you know, they are required to start this journey of going into real time fraud prevention, and supposedly they don't have the experience, they don't have the skills. Otherwise, they wouldn't be listening to us. And I think it is very easy in this situation to fall, quote unquote, into this trap of, okay, I have these like scrappy juniors, but what I really need is someone with experience. And what I see happening very often is that in these situations, you tend to hire someone with 15 or 20 years of experience that sounds on paper like exactly what you need. They end up just, you know, dragging their feet and kind of like costing you a lot of very expensive time. So what do you have to say to this? Like, how can we identify these red flags or these misfits, right? It might be like super great in an established organization that is already running, but they're not necessarily the right fit to start things zero to one.

Yeah, so first off, like 100% aligned, and that's why I always say that I started my answer with finding people that fill your weaknesses, not the latter. And I think so definitely 100% aligned. I think the key is don't be too worried about total time in career. So, like, that's a trap. Like someone that has 30 years of experience isn't necessarily better than someone with five years experience. Someone with five years experience, from what I've gone through, I would say also understand the org that they have been brought up in and trained on. So, for example, someone that comes from Chase, they're going to be much more, you know, very process heavy, very bureaucratic, right? There's going to be a lot of approving authorities. There's going to be a really long tail. It's going to take two years to bring in any technology, right? So that's going to take a different type of skill set, and that skill, by the way, can be insanely valuable in the right org. However, if you're trying to move fast and you're trying to be scrappy and you're trying to spin something up because you're trying to be like a real time payment rail, right, that's probably not the resume I would go after, right? I would go after someone that knows how to wear multiple hats, that can really step in. And most importantly, you want people that don't stay within their vertical. You want people that, even if their job is just fraud analyst, I want them to also be a data analyst, an operations analyst, a fraud analyst. They might even be customer service at some point, right? So I want someone that has the flexibility to be able to understand the needs of the business exceed the needs of their position, right? And you really have to hire for that. I would also say for, like, when you get into the more management level, et cetera, you know, you want to look for people, and I just want to actually have one caveat, that if someone has spent time in banking in their career, that's actually really beneficial, but you don't necessarily want someone that spent their entire career there.

Yeah.

Right. So, like, there's actually really a lot of value in people that have both sides of the fence, and the reason for it is they understand what does an effective process look like, right? Because there's no way that Chase is moving $100 trillion and all these crazy things without a pretty effective process, right? So I'm just using that as an example. But, you know, at the same time, leaning into kind of the more startup vibe of how they scale, right? And understanding, you know, really, and the questions that you ask them, by the way, I think, like when we're talking about the red flags, is make sure you dive into the how when they describe something. So if they say, you know, I'll give you an example. I was doing an interview not that long ago, and the person was like, "Yeah, I saved the company $65 million over the course of a year by reducing fraud losses," et cetera, you know, the kind of the standard. And I was like, "Okay, now, how did you do that?" Right? And you just keep asking, "How did you do that? How did you do that?" Right? And make them go deeper and deeper and deeper. And I very quickly came to the realization that they didn't do that, like their team did that, right? And they actually really were not in the weeds and actually understand what needed to get done to achieve that. And that's okay. But like, when people are taking credit for other work, that happens a lot, right? Like, "I am responsible for saving $100 million." And it's like, is that true, or was that like a data scientist that spun up a really sick model, right? And like you were just sitting back and maybe adjusting some thresholds, right? And that's like a little bit different, right? That's a different mentality. And that's actually a trick I stole from Elon Musk, actually. So that's one of his interview tricks. He doesn't really care as much, at least what he claims, he doesn't care much about some of the stuff that we talked about and where you came from, et cetera. He wants to know, like, if you achieve a certain benchmark, he dives really in depth into how you actually solved it. And you can figure out really quickly, and it works. I've been using it for years now. You can figure out really quickly if they actually were the ones that drove those pillars, right, and really actually understand the problem and solution, right? And then the other thing is, like, give me a complex fraud case or an interesting fraud attack and walk me through the full lifecycle, right? So you had a major attack. And if someone says, like, "We haven't really had any attacks because our system has been so good," that's obviously a major red flag.

Yeah.

So, yeah, that's like an immediate no go. But you want them to walk you through, like, what is the actual process of how did you respond? How did you identify it, right? How did you mitigate it? How did you detect where it was coming from? What was the source, right? And really leaning into that, you can get some amazing insights into the person and how they actually, like their mental model of how they think through things. And that's where curiosity will really stand out in questions.

Love it. I'm gonna report you to Elon, of course, but awesome. So you were just mentioning process, how to speak to candidates about processes, and how to kind of like think about your team and the makeup of your team in terms of, you know, do they know how to run and build and maintain the right processes? So what is the right process?

Yeah, so there is no right process. That's the answer, right? There is no, like, this process is the industry standard, right? And so I'll give you a perfect example. So I've been to companies where they put a lot of weight and value on a verified customer. Okay? So, for example, the process is completely different if it's a verified, let's just say IDV, doc verified customer versus a net new customer. Okay, and those are the kinds of things that are immediate red flags to me, because let me tell you that the majority of fraud comes from verified customers or verified accounts, right? So it's, you know, a huge percentage of fraud out there now, especially nowadays. So like, it doesn't really add any value that they're verified, to be honest. And like the age of the account, we need to get rid of that. Like the entire industry should just remove that. "Oh, there's a net new account is not necessarily higher risk than an account that's been under property for a year," right? But they equally will commit fraud, right? So you need to focus on their data points. But anyways, when you're trying to build out a process, the key is you want to focus on efficiency and simplicity, right? So now lack of process is just as big of a major problem in an org when you scale, okay? So there is this mentality, and I think a lot of people will disagree with me on this, actually, and I think that's totally great. I love having people disagree with me, is I think that, especially in the startup vibe, startup world, their argument that you shouldn't put in process at all because it can stifle innovation, and you want to kind of give everyone the freedom to wear multiple hats and just figure things out and dive in and figure things out. And I think that's okay. I think you can actually do both, because my opinion and what I have seen over the years is that you, like, process, sometimes you have to slow down just a little bit to speed up. And what will happen is you will start having multiple team members kind of cross lanes and be duplicating work because of that lack of process, right? So sometimes the goal is to just get them a rail, right? Like give them their pathway with the North Stars, right? So, like, here are the North Star benchmarks that we're trying to achieve, right? Like, if you're doing an investigation, what should be the outcome of that investigation, right? It should be, do we want to accept the customer on our platform? Do we want to reject a payment? Whatever it may be, whatever the type of investigation is, right? And so I think, like at a startup level, you can still give them some freedom to kind of move within that guardrail, as long as they're achieving the same result. Now, the other thing that you want to do is you want to set like kind of benchmarks to peers. So, for example, what you don't want is someone that's so curious where they're spending an hour looking at one account, and I've had that happen tons of times throughout my career, right, where they just like deep dive in. Fraud fighters are infamous for this, right? We go down these rabbit holes, and the next thing you know, you find some North Korean backed hacker that's moving $5 on your platform. And it's like, dude, it doesn't add value. It doesn't matter, right? So I think that sometimes you need to set, put in those guardrails to make sure that they're not going down those deep rabbit holes. And I think the other thing is, give, allocate a percentage of time so that they actually can do that. So I have found that the best way, because almost every team is reactive. Almost everyone in the industry is reactive. Like, anyone that says they're proactive, you really got to question it, because you have to be proactive, by the way, in today's, which we'll talk about, especially in your technology. But most operational teams are reactive, right? They're waiting for the indicators. They're waiting for the dispute to come in. They're waiting for the customer escalation to come in. And then they initiate their investigations. So allocating a percentage of your team's time, like let's just say at the analyst level, let's just say I want 15% of your time on a daily basis to just investigate, just go down a rabbit hole and have fun, right? I want you to just find something interesting. And it may just be a trend that you see, and maybe it's not even a fraud trend, but like those lessons learned. So like I stepped into an org, I can't call out the org, but I stepped into a company as an advisor, and I spent like five hours with the leadership team, and they couldn't really tell me what's going on. And we went around and around and around, and then I ended up basically pulling one of the junior, someone with six months of experience, off to the side. And I was like, what do you see? And we solved it in like 10 minutes, right? And so sometimes your team and your boots on the ground, your junior people, will likely see things much faster than the head of frauds or the fraud managers, et cetera. And so making sure you're giving them a little bit of flexibility and freedom to stretch their investigative legs is really key. So I don't think there's, you know, to bring it full circle, I don't think there's a process that is like, here's the industry benchmark. You have to build it based on efficiency. You do not want to over engineer it. Do not over engineer. Like, full stop. You over engineer it, it's going to fail, and it will not scale 100% of the time. If you keep it simple, have checks and balances. Make sure there's guardrails in your process. Make sure there's benchmarks to prevent spending hours on something versus spending five minutes on something, right? So you kind of base them based on the average of their peers, right? And also competition is very good in teams, right? Like competition, humans are very good. Even the ones that say they don't like to compete, it drives performance. So, you know, making games of it is really interesting, right? Like, I built, for example, what we call it, like State of the Union, where it's just like a dashboard, and it says like, you know, total investigations for the day, or whatever it may be, and total time spent on X, Y, and Z. And they end up trying to compete with their peers, and it just drives more performance. And there's no negative outside, like if someone doesn't show up on the board, it's not like they're doing bad, right? But it's just like that subconscious, like, oh, I really want to perform well today, right? I want to be in my home, my name on the board, right? And so those are the kind of things that I think add the most value.

Yeah. I mean, interesting that you say that there's no right process, because, you know, I have these five rules, and they go with me whatever company I'm at, and they crush fraud, right? So.

Yeah, exactly, exactly. It's only five rules.

Yeah. I want to pull on two threads because there's a lot to unpack in what you said. And I want you to kind of like maybe expand more on how it translates, like in teams that deal with real time fraud prevention, because you said two things. You said one that, you know, let me start over. What is it that you said?

Are you talking about when I was talking about, you're talking about the initial part when I was saying, shoot, did you miss something?

It was definitely the proactive and reactive. There was one more thing. I think I know it was more general. I wrote, there is no right process. Focus on simplicity and efficiency. Let's take that. Okay, so I 100% agree with everything that you said. There are two threads that I want to kind of like pull on, and maybe to be more specific, when it comes to how this might look like when it comes to teams that manage fraud in real time. And one, you talked about the fact that the process needs to be simple. And two, you also mentioned that, and by the way, I totally agree with that, that your process needs, at least to start with, needs to be much more reactive in how you model your operations than proactive. Because proactive is, if you're a world class team and super mature, maybe you can start thinking about it, so it's a long term goal.

Yeah.

Yeah. But I would guess that, you know, some of our listeners that are thinking about real time fraud prevention, they would say, okay, but if I'm doing that in real time, but at the same time, I'm reactive, so how does that come together?

Yeah. So basically, I think the best way to think of this is in the difference between traditional, call it like payment rails, right, where there's more reaction time, is at the time of payment, for example, or at the time of escalation, or at the time of customer, you know, whatever it may be, right? At the time of escalation, regardless of where that sits, you have a window of opportunity to take action, right? So that's kind of the traditional path. Now in real time payments, right, just like, you know, money is moving near instantly, right? And so the time of payment is now too late, right? The time of escalation is now too late. It's very similar to wire transfers. Once the money has been executed out, right, you're done. There's no clawback process. It doesn't matter if your fraud team is able to investigate it at that point, right? So you have to kind of treat every payment as like a wire transfer, or the best way to describe it is like someone is about to hand cash to someone else, and when they do that, they cannot get it back, right? So if you're handing cash off, should you focus your efforts the second that they're handing the cash to someone else, or should you focus your efforts on the drive to reach that person, right? And so you have to change your mental model to move upstream. So the key is a perfect example. Let's just say that you have a customer that, you know, let's just say they transact an average of $900 a month. Let's just say this person's from Ohio, right? And pretty similar spend patterns, pretty similar behaviors. And then you randomly see a new device logged in, getting authenticated, let's just say went through a step up process, so it was added as a trusted device. And then it adds a new payee, and then it attempts to move $10,000, right? Let's just use that as an example, right? But if you're looking at it at the time of moving $10,000, you're too late, right? So that money movement time, that's your final last gut check that you have to be able to take action. So what you kind of have done is you can decide, for example, wait a minute, new device logging in, followed by a new payee, like that is enough of a red flag right there to start putting in significant friction. Okay? And what we have found, which is really interesting, is that it used to be, especially back when you and I started, it used to be like friction was so much the enemy that companies were willing to take significant losses to not touch friction. And what we have seen time and time again, and I've done studies on this, even at Novo, where we actually did a study on this, and what we have found is actually friction can increase spend and usability on your product. So I'll give you an example. We tested like, okay, how many times can we actually do a step up, for example, a customer on a debit card before they just get so frustrated? And so what we found is, if we step someone up two times a month, we saw an 11% increase in card spend. And we were like, how is this humanly possible? Like we're stopping more money, right? We're stopping the transactions, adding friction, but they're spending more, right? And what it came down to when we did customer interviews on this is that they felt like they had more trust in the product, right?

So friction done right increases trust. Friction badly reduces trust.

100%. And I think that's where, you know, that's why I always lean into, I call it tactical friction, right? Where if you can apply friction at a point where it's reasonable, right, then it is not a bad thing. It'll likely be a good thing. You'll actually build trust with that customer. I'll give you an example. If a new device logs into my account, adds a payee, and is attempting to move money, like you better believe I want my bank to add friction. I want my bank to challenge me. I want my bank, if I'm moving $25,000, I want my bank to verify that I want to authorize this, right? If they don't, it's not a user experience plan at that point. It's just like unnecessary risk. And let me tell you, everyone's like, oh, well, like friction causes churn of customers. Let me tell you, nothing causes churn more than your account getting taken over or losing money on the platform, right? So you actually look at it. And by the way, most of the time when I hear people talk about churn, it's usually like a growth team, where it's like someone outside of fraud that's applying pressure, saying like, no user experience, right? Or it's going to impact user experience, et cetera. But they haven't come to that conclusion because of actually testing. They have come to that conclusion because that's just been the industry standard for a long time. But if you actually stress test those arguments and actually put in some process, so like, for example, you could say like, hey, let's add 5% of our traffic to a new friction process, right, and see what happens, right? And again, if you're doing it tactically and you're applying friction at the right time and when it actually matters to where you want it to be, to where your customer says, okay, I get why my bank or my fintech, or whatever it may be, like I get why Instacart or whoever it is, right, is asking me to verify this, right? If it's like I'm buying a $5 item and they're blocking me, and I'm having to go through this process and I'm standing in line at 7-Eleven, that's a different conversation. That's like frustration, right? Now, like I'm spending $1,000 at Target, for example, and I normally only spend $100, it makes sense that they would say, hey, just want to make sure you're good with this, right? I'm not going to be like, oh, I'm going to spend a month changing my platform and going to a different bank, though, because like, God forbid, they asked me if I want to verify. It's just not true, right? It does not happen in real life. And so I would say, as a fraud fighter, challenge that status quo, especially with product people and growth people, and force them to realize that, like, okay, let's prove that your theory is right. And let's actually do like an A/B test. Let's do like a champion challenger, right? Let's allocate a percentage of our traffic to a process that I want, like let's just say friction steps or step ups throughout the process for certain high risk data points. And then we'll allocate the rest of the traffic to something else, like let's just say the current process. And about nine times out of 10, you're going to find that you're actually going to probably have less churn and make this. Because, again, like almost guaranteed churn when you have, especially on cards, if you have more than two fraud events on a card, like two fraud events, is like the kiss of death. Now, it could be over a period of time and you're okay. But if you have like two fraud events within, let's just say six months, you got a serious problem on your hands. That's like a high risk of churn at that point.

Absolutely. So let me see if I kind of like got your thoughts here, because I think that proactive versus reactive, especially in real time, is where a lot of teams fail, at least fail to grasp what it really means. And I think that, you know, your example of friction as kind of like an extension of how much trust we put in you, the customer, but also how much trust the customer is putting into you as my provider. And I think what you describe is two things. On one hand, you want to be proactive in the sense you want, because you are operating in real time, because if you have loss leakage, you don't have a lot to do with or to do about it. You need to go upstream. And this is the way to be proactive, meaning you need to kind of like start thinking about building multiple lines of defenses before the monetization or the accident happens. But on the flip side, the learning, you mentioned that through kind of like the A/B test, the learning is done reactively, because you need to kind of like watch the population, whether this is like the fraudsters or how normal legitimate users behave when you introduce friction, and based on these learnings, you know how to tweak the system. Did I get it right?

You're spot on. Yeah, move upstream. And you're exactly right. That's how I would delineate proactivity versus reactivity, right? I think that proactive is like proactively moving out, right, out from the blast radius, right? Go to the farther rings, right, and start. The best way to think of it is like a castle, right? You're trying to protect the king, and let's just say the king is the real time payment in this case, right? You want to put multiple walls around, and maybe right outside of that. Now, there should be pathways where the drawbridge drops down, and it's just a straight shot right to the king. That's okay, right? You need to make sure that they are vetted multiple times throughout that process. And that's where tracking that user journey is so critical, right? It's like what is happening in order to get to that payment is way more important than the payment itself nowadays, right? And so I would say, like again, the payment is actually the last point in time that, like, if you're only focusing on the payment, you're really behind the curve right now. Because you have to move upstream into these attack vectors in order to be able to, like, that's like your proactivity right now, is moving upstream and approaching it from a user journey perspective. And I think that, like that, I would agree with you on that delineation between proactivity and reactivity.

Awesome. So should we move on to your last P, the product, or do we have more things that we want to cover? Okay, then product wise, what does that actually mean?

Yeah. So it means a couple things, right? It depends on the company. It depends on the way that you're building your team. But in general, I actually, and I may consider this uniquely, but in general, in fraud in particular, when we talk about the three P's, I want to talk about the product from a fraud prevention technology stack standpoint, and also the product as like what is the business that you're working for selling, right? So I include both in my product, right? And you have to build your, so it's like, for example, if you're at a fintech that's building some sort of RTP payment rail, right? That RTP payment rail is obviously the product, but so is your tech stack, your mitigation tech stack, your fraud prevention tech stack, right? And so when we talk about building that initial guardrail, there's a couple approaches that I've seen, okay, and some are very effective. Some are not effective. One is like, let's put in five layers. Let's just say, like we're talking about that castle doctrine, right? Let's put in five layers from account creation to identity verification to login protection to whatever it may be, right, as you're going through these layers, right? And some will use one platform that offers those different layers, right? Some will just integrate five vendors to plug in. And I don't necessarily, at this point in my career, think that there is necessarily a right or wrong way on that one. I actually think the key is, are they communicating? So you need to have the, you know, I think what's so critical is making sure that you have one centralized location to be able to understand and synthesize the findings of those layers, right? So you want to be able to see like, okay, wait a minute, this is a high risk login, okay? And then we see, like, okay, this is an unusual behavioral biometric compared to what we normally see. And then you say, okay, this is like a high risk new payee that we've never seen before, and this person has never interacted with, right? And okay, now there's like, they're trying to initiate a much larger dollar amount than their general spend. But if those are different vendors, right, and those vendors are not communicating effectively, right, in your tech stack, you're going to miss it. It's really, really hard, right? And then you end up having to, and I see this all the time, then you have fraud teams that just focus on each one, right? And then you end up having like five fraud teams, because you've got like an onboarding fraud team, a KYC fraud team, and then you got, right, and that's not really necessary if they're all communicating effectively, right? And so I think especially early days, if you have the capability to do that, and that's where the value of going into one platform approach to where that you can do that is where it's pretty close to guaranteed to communicate, but it's not necessarily required, as long as you're building it effectively, and as long as you can speak with your vendors, for example. Have those conversations of like, I want to make sure that vendor X can communicate with vendor Y, and they'll tell you how to do it. They'll say, like, yeah, well, like our SDK is pretty much data agnostic, so you can just plug it in and feed that as like a custom field for the API, right? And so, like, you know, we do that at Sardine all the time, right? And I know there's tons of vendors that do this where you can say, like, if you have signals that you want to pipe in as a part of your risk stack, just plug it in as a custom field or a user defined field in the API, and just send us those signals, right? So I think making sure that everything is communicating effectively is critical. And then also, I think that, so I guess that's like the lesson learned, is that if you build in a tech stack that's not effectively communicating, you're going to have problems with scaling. You're going to miss things, to fine tune each layer, and it's really hard to do that. It takes a lot of time. And the problem with that is you can get caught in this trap of like, I'm going to build, and here's like, I would say, my next token of wisdom for everyone, is that if you want to build a best in class tech stack, right, from the product side, going out and getting the best vendor for each layer will not lead to the best process or product. Okay? So if I were to go out and say, okay, I'm going to get the best vendor on the market for IDV, then I get the best vendor on the market for account security, then I'm going to get the best vendor on the market for KYC, then I'm going to get, right, you just keep going down that process. Nine out of 10 times, that's not going to lead into a best in class tech stack. And so sometimes you want to sacrifice, right? You have to have some sacrifice in your tech stack if it means that you're going to gain some sort of value for the greater good, right? So like, let's just say that maybe the best in class vendors, because they're so competitive, maybe they don't play nice together, and maybe the communication lanes are really challenging. You're going to have to build a pretty sophisticated internal dashboard or model to be able to synthesize all these signals, right? You may be better off maybe going to like one tier down, right, and focusing on someone that actually is more data agnostic, right, or that has more flexibility to kind of work with other technologies. And so I would say, like, I've seen that many times again where fraud fighters, like, "Yeah, I'm going to build the best in class tech stack, and I'm doing that by just getting the best in class vendors of each category." And like, I've never seen that actually work, as crazy as that sounds, you know.

I don't think it's crazy at all. I think, I mean, I'd say first of all, I think it is such an important point that actually teams that go from manual operations to automated operations can easily miss, because when you only have a fraud ops team and they have their own like four different tools, like an admin and this vendor's portal and SEON's portal and whatever, and they just copy paste data fields between them, that's something that is like very easy to think would work also in real time fraud detection. And it doesn't. It breaks immediately, obviously. And I also think that, you know, the last point that you made about a best in class collection, eclectic collection of vendors, doesn't necessarily mean best in class stack, to me actually goes back to what you started with, and that is to keep it simple. And many times by doing exactly what you described, by going to five different vendors that don't talk to one another just because they're the best, actually you're creating a very complex system, and a very complex system is also very fragile.

100%. And let me tell you, I want to double tap on what you just said because that's so important. Not only is it fragile and very complex, but actually the vulnerabilities will start expanding substantially. So the exploits that you can actually get in those weeds, right? So like the best in class vendor doesn't necessarily protect you best in class, right? Because it's based on how you're actually integrating and using that across your platform, and are they communicating effectively? So the best way to think of it is, if I have five best in class vendors, but I'm not linking them together effectively, those chains are weak. And when those chains are weak, those are exploitable chains to fraudsters, right? They're going to penetrate those weaknesses all day long, right? So you want to have really strong bridges between those vendors if you're going to use the multi vendor approach instead of a platform approach, because otherwise, those vulnerabilities are just going to each.

Absolutely. Yeah, I've seen it with so many clients. It's honestly, it's a bit depressing. Yeah, yeah, yeah. Okay, so we want to keep the tech stack cohesive. We want to have a central hub. What else, like when it comes to real time fraud prevention, what else would you say is important to think about when you're starting out this journey?

Yeah. So one thing that I would say is some of the traditional signals. There's two approaches. So first off, sometimes the most sophisticated attack vectors and the most sophisticated fraud, let's just call them exploits, can be solved with, okay, I have seen time and time again where, because it's a really new like polymorphic agents show up in the market, and some of the banks, like I was just talking to the head of banking at Google, and she was telling me that she has multiple financial institutions that are kind of pulling their hair out as they're starting to see this down with her. And, you know, we were talking about this. At the end of the day, sometimes going back is actually far more effective than spinning up some new AI machine learning model to solve it. Wow, wow, right? And so I think that that's one mentality that you have to always pay attention to, is don't get caught up so much in the hype. And just because something is super complex doesn't mean a simple mitigation step won't be just as effective. So like when you're building your tech stack, I would say like a good rules engine can knock out a huge portion of your fraud, right? And people are always like, oh, we want AI and machine learning, and we want all this stuff. And it's like, that's great, but a rules engine will get you 70 to 80, maybe even 90% there. The machine learning model will squeeze out that last 10%, right? Or the AI, right? Where AI comes in is AI will supercharge your team to be able to focus on higher ROI things. It'll basically compile data more effectively. It'll help you. It's like giving your team a steroid injection to be able to perform better. So that's where we can talk a little bit about the agentic side. But that's just, I think, like the key takeaway here is just because the fraud trends are adapting and becoming faster and becoming more aggressive doesn't mean that simple solutions no longer work.

100%. It's funny because I keep saying exactly what you just said, keep repeating myself, and it feels like it always very weird. And, you know, especially when I was a CTO of an AI company, it was always very weird to say, look, maybe like a couple of fraud rules in place and that will solve. And it feels like the more experience you have in the industry, the more you understand that also your solutions, by being very simple, that also means that they're, you know, it's easier to monitor them. It's easier to make sure that they are not misbehaving. It's easier to tweak them if they need some tweaking. A very sophisticated machine learning model, that's all great and dandy, but retraining it, or making sure that it is fresh, or that now, perfect also for this new payment method that you have, or this new region where you deploy, that now means to basically retrain the model. And that's a whole lot of work. That's not a half a day of an analyst, right? That's a completely different scale of work. So I am completely with you on this.

Yeah. And one other thing that you just said that I didn't think about, but you're spot on, is like with machine learning, it's really challenging, even AI in some cases, but mostly on the machine learning side, both supervised and unsupervised. It's really hard to have like a plug and play model, like even at the vendor side, right? I'll give you a perfect example. In issuing, you know, when I was working with Sardine as a design partner on some of the machine learning models, we could not, for example, take, so Brex was a similar company, or Mercury, or some of these others, right, where it's business banking, okay? A machine learning model trained on, let's just say, Mercury's data or Brex's data to stop business banking exploits would not work on Novo, which has the same business banking exploits. And the reason for it is your customer risk is different, right? Your user journey, there's just too many known unknowns in that funnel to where it's just not a one to one plug in. And so what ends up happening is, when you get these out of the box models, is that you end up having to set the threshold so high to where like it has to have a 99.9% confidence before I'm actually going to consider it very high risk. And by that point, like, you probably should just build some really cool rules, and that would have probably done the trick far more effectively.

Absolutely. I think this traces back exactly to the point that you started with, where product is first and foremost the product company, right? Because that basically dictates what are the threat vectors, what are the use cases, how they manifest, and so on. Completely agree. Just because you mentioned that, and, you know, I can't hold myself. It's such a huge topic, and we can go into a lot of different routes, but zero real time fraud prevention. I'm starting from zero. Do I need to consider agentic, or is it something for year two?

Great question. So the best way I could describe it is as follows. In order to have a very high performing agentic layer in your tech stack, right, you have to have a really, let's just call it AI friendly ecosystem. And so what happens is the companies that are building, let's just say they have gone the traditional route first, it's much harder to then pivot to agentic and AI than it is if you build everything from the beginning to actually get to that path, right? So I would say you have to make a decision. Like, are we, even if we don't do it now, in the next three years, for example, are we likely going to pivot? And that's something like, I promise you, your CEO is going to say, yes, you better believe we're going to pivot to AI. I've yet to see a CEO be like, no, I don't want anything to do with it, right? Because they see that, like, okay, wait a minute, if we can squeeze out more performance and reduce headcount, maybe, like these are all kind of things that seem like the no brainer for an executive. So building up front is going to be really critical, right? And that's also where, again, it falls back to keeping your processes simple and elegant. Because if you have these really complex processes, and you have these multiple teams working on different things, on the agentic side, you're gonna have to build agentic agents like each of these, and that's going to really squeeze the ROI of that agent down substantially, versus having an entire rail to focus on. So at the end of the day, this is what we're seeing in the industry. First off, let's set the benchmark. We are not seeing agentic AI replacing units in fraud prevention. In fraud prevention in particular, I have not seen it once yet. In compliance, I have not seen it once yet. Now, don't get me wrong, there are absolutely leaders out there that have terminated people or laid people off for AI. But if I was a betting man, I would bet my entire salary that there was some other reason, and AI was like a good argument to go down that route. I know there's a Fortune 500 company that recently said, like, hey, you know, we were implementing AI and we laid off either 10 or 20% of our workforce because of it. My bullshit indicator went up instantly. And I was like, no, no, this is post pandemic bad hiring practices, and they're using AI as their cover story, right? So let's just be very clear on that. So now that we have that as like a benchmark, right, I think there's two things. One, to protect yourself in an AI future, you don't have to be a data scientist. You just have to be a world class user of AI. And I think that like that's where, you know, see how you can improve even your daily life. How do you be more efficient on things, right? Your work, right? Even if you're just self prompting stuff, those are all really good ways to start learning how to interact and engage. And you'll find there's different tricks and techniques that you can use to squeeze out performance. And, you know, like your prompt instructions, how you actually write those to actually get to where you want, right? And you want to like prompt out, like, you know, sometimes what will happen is you can get down this layer within large language models where, like Frank McKenna and I went down this rabbit hole. It was so funny. So Karisse Hendrick reached out to us, was like, "Hey, have you guys heard of spider web fraud?" And I was like, spider web fraud. Like, I'm thinking maybe some sort of network graph type thing that they're talking about. And so both Frank and I were like, no, we haven't heard of it. And then I went on Google spider web fraud, and the AI response was like, it's this new trend of fraudsters using Halloween spider webs to commit fraud. Then I'm thinking in my head, like, dude, there's no way this is real. And then Frank went online and was like, what is hamburger fraud? And what was happening is the model was just reinforcing our prompts, right? And it was connecting the bridges between us. So you got to be really careful on how you do it. And so we went down this rabbit hole for like five hours laughing about all these new fraud trends. I think even Frank posted on this. So I remember, you know what I'm kind of talking about. So, you know, I think that you have to be really careful and understand how to effectively prompt. So there's something called prompt instructions in the settings of your large language model. Almost all of them have this, not at the business level, but they definitely on the personal level will have it. Basically, you give instructions in the settings of your account to basically tell the large language model how you want it to interact with you. Okay? And by the way, you can use large language models to help you write that prompt anyways. But what you want is you want something that's going to challenge you. You want the model to challenge you. You want it to say, like, don't reinforce my biases. Like, stop me if I'm going down a rabbit hole that's wrong. If my question is leading to a false answer, challenge it, right? You want it to say, basically, poke holes in my logic, right? And that is a far more effective approach to start using AI. Because, for example, like I'm going into a board meeting, right, and I want to make sure that I am fully prepped on every challenge that I could possibly get on this proposal, right? And these are the things that AI is very good at. It's very good at finding these new arguments and logic, right? But you just got to be careful not to just reinforce yourself and go down this thing of, like, I want this outcome, and so I'm going to prompt it to give me this outcome. And that's what I see the mistake every time is, like, help me, guide me to the outcome that I already want. Like, make it guide you to the outcome you don't want, right? So I think that that's a key piece of like, we're talking about becoming a world class user of AI. Now on the agentic side, so first off, it's not replacing jobs, but what we are seeing is it is supercharging the analyst performance. So every operational fraud analyst is now a data analyst, almost full stop. And so what we're seeing is we're seeing this trend where people that are not data scientists or are not data forward experts, right, or data forward analysts, they now have the ability to make data based decisions versus intuition. And, you know, probably 50% or more of fraud fighting is intuition, right? It's a huge percentage of it. But when you don't have the data to back up your intuition is where you start getting into the traps. And so where agentic agents can be incredibly powerful is they can provide you the reinforcement data to help your intuition kind of guide you to the correct answer and have a much higher confidence and certainty that your answer is correct, versus before you're going off like gut feeling and like whether or not I get goosebumps when I read this account, right? That's not a very effective way to do it. That's kind of the traditional way to do it. So it can be incredibly effective. I think that there's some really interesting stuff happening. So right now, there's like, I would say there's a lot happening on the regulatory side on this as well. But let me tell you that the OCC, FDIC, CFPB, all of these financial agencies, they are all in on AI. They are improving AI use cases right now in financial use at the fintech layer, at the sponsor bank level, and at the regular banks, right, credit unions. So now is like 2026 is probably the best time, especially with some of the natural world changes that are happening that can give you some firepower to push on your org, push on your leaders, to start adopting more agentic framework. And then also, I think that there's going to be some interesting things happening on the Reg E side, right, that liability shift side. And then you're also starting to see like, even on the agentic commerce side. I have a difference of opinion on agentic commerce than most, but from utilizing it in your team, like to me, if I was building a new team, if I were to start tomorrow and build a new team, I would 100%, every single time, I would build my team, my process, my product, to be able to be highly effective for the use of agentic agents.

Absolutely. Well, folks, you heard it first here. You want to double down on AI, so make sure you do it tactically. Matt, this has been like, you know, we have a lot to cover here. Or actually, we covered a lot. We have a lot to summarize here. So let me kind of like collect all the takeaways that we covered in the last hour or so. So we talked about fraud prevention in real time, zero to hero, how to get started. And I think the red line throughout this entire conversation was, keep it simple, stupid. That is my takeaway from this conversation. You worked your mental model of people, process, and product. On the people side, we talked about hire people for curiosity, hire people for trust. For the people side, we talked about hiring people for curiosity, we talked about hiring people for trust, and we talked about how we want to prefer flexibility over experience. We talked about processes and the fact that there is actually no right process in the industry, even though a lot of teams think that there is and they expect that there is the right process. Actually, there is no such thing. Again, we talked about focusing on simplicity and efficiency, not to over engineer, at least not at first. We talked about the balance that you need to strike between being proactive and being reactive, and we talked about that in real time fraud detection, real time fraud being proactive means that you want to layer your defenses and make sure that when it comes to the monetization, to the loss event where you can experience leakage, you want to make sure that by then, the user underwent some risk checks where you already have a clue. But on the other hand, also that in the end, learning, relearning, whether this is about new fraud attacks or whether this is about how legitimate users behave, almost sorry, not almost always, happens retroactively. And you gave a very good example about friction. We talked about friction and the fact that friction done right can increase trust. And lastly, we talked about product. Two things. A, you need to consider the product of the company and what kind of surface of attack it creates, what kind of fraud threats it brings to the fore, and of course, also to consider your tech stack, your fraud prevention tech stack. We talked about the fact that it's okay to have different vendors, as long as you have one central hub where you can have all the data and all the reporting and all the monitoring, and that these vendors can communicate between one another, whether this is through vendor partnerships or a certain platform, or whether these are pipelines that you built yourself. We talked about that having the best vendor doesn't necessarily mean that you would have the best fraud prevention stack, because you can create a lot of complexity. And here again, we return to the theme of keeping things simple. We talked about not getting too much caught up in the hype, because sometimes, or actually very often, our experience as fraud fighters is that sometimes you can prevent, or not prevent, let's say mitigate, very sophisticated fraud attacks with one simple rule instead of trying out to spin up a complete machine learning model from scratch. And lastly, we touched some bonus material when we talked about agentic AI, and you said that while AI is not a must right now in terms of combining it within your fraud stack, you put a lot of emphasis on the fact that this is the time right now to become a world class user, to become literate, to upskill not only yourself, but also your team in the usage of AI. And lastly, you also mentioned the fact that already today, agentic AI can be very helpful in taking folks who are maybe domain experts, or at least, I would say, fraud specialists that don't necessarily have the data background, and give them the tools, the capabilities to back their intuition, back their experience, their gut feelings, with data through the use of agentic AI. Hopefully, I got everything and I didn't leave anything out.

No, you nailed it. I think at the end of the day, keep it simple, stupid. Focus on the three P's, right? And really, don't over engineer things. Make sure things are communicating effectively. And that's off to a really good start, right? That'll help you achieve what you want to achieve. It'll allow you to scale. Don't over engineer, right? And absolutely, if you're not using AI today, you really need to, because the technology is going to improve, right? So even if you're a non believer right now, I assure you, the technology is doubling, right? So by the time you get to the point where you become a believer, it's already too late. So you need to start building the framework and the fundamentals to make sure that you're going to be ready when you are ready.

Matt, it's been awesome. Thank you very much for sharing your experience with us. For you folks, sorry, let's do it again. I want to give you actually. Matt, thank you very much for sharing your experience with us. It's been a blast.

I really appreciate you. Yeah, this has been a lot of fun, and it's always fun jamming with you, because I know you and I come from similar backgrounds and different similar mentalities of the way that we approach things, so it's fun to bounce ideas off you.

Wow. 100%. Folks, hopefully you enjoyed our first full interview episode. There's a lot of things cooking and more to come. In the meantime, you know the drill. Smash the like button, comment, subscribe, save, download, send it to your mother, send it to your grandmother. Leave a review and see you next Saturday. YouTube Description Real-Time Fraud Prevention: Zero to Hero w/ Matt Vega May 16, 2026 Episode #4 Guest: Matt Vega This episode is a bit of a full-circle moment. Years ago, Matt Vega interviewed me on one of my first podcast appearances. And now, somehow, here we are, roles reversed, with Matt joining me for the first full interview episode of The Saturday Fraud Strategist. Honestly, not a bad way to start. In this episode, Matt and I talk about what it actually takes to build real-time fraud prevention from zero. Not the polished vendor version. The real version. The one with hiring decisions, messy processes, fragile fraud prevention tech stacks, disconnected vendors, and systems that look impressive right up until they break. Not a good look. While real-time fraud detection sounds like a technology problem, the conversation goes deeper. We talk about people, process, product, real-time fraud monitoring, tactical friction, fraud prevention guardrails, AI readiness, and why teams need to move upstream before the money is gone. Because once the payment moves, especially in real-time transaction monitoring or real-time payment environments, you are not preventing fraud anymore. You are documenting the damage. What you'll hear in this episode: ● A breakdown of Matt Vega's people, process, and product framework for real-time fraud prevention ● A practical discussion of how to build a fraud prevention strategy from zero ● Insight into hiring for curiosity, trust, flexibility, and actual problem-solving ability ● A conversation about reactive vs proactive fraud prevention in real-time payment environments ● A focused look at upstream fraud detection, tactical friction, and why friction done right can increase trust ● Practical considerations for building a fraud prevention tech stack where vendors, signals, and workflows actually communicate ● A discussion of AI fraud prevention, machine learning fraud detection, and agentic AI in fraud prevention Listeners can expect a conversation that moves from theory to operating reality, and from operating reality to practical decisions fraud teams can actually use. Who should listen: ● Fraud leaders and fraud professionals ● Risk, compliance, and cybersecurity teams ● Fintech, banking, and payments teams ● Product leaders building real-time payment experiences ● Fraud operations teams moving from manual review to automation ● Founders, operators, and executives building fraud prevention programs from scratch Anyone evaluating fraud detection rules, behavioral biometrics, device intelligence, KYC fraud prevention, account takeover prevention, or the best fraud prevention tools for their stack. The discussion is designed for professionals who are committed not only to detecting fraud, but to building systems that can scale without becoming fragile. Episode notes: (Accordion dropdown) This interview with Matt Vega gets into what real-time fraud prevention actually requires when payments move fast, fraud teams are under pressure, and the old "wait for the escalation" model is already too late. We talk about why upstream fraud detection matters, how real-time fraud monitoring and real-time transaction monitoring change the fraud investigation process, and why tactical friction can build trust when it's done right. Matt also gets into the uncomfortable stuff: reactive vs proactive fraud prevention, hiring for curiosity instead of résumé theater, and why a fraud prevention rules engine can still beat a shiny machine learning fraud detection setup when the basics are broken. Honestly, not everything needs to be AI. Strange, I know. We also touch on AI fraud prevention, agentic AI in fraud prevention, fraud prevention guardrails, fraud risk management, payment risk management, and how signals like behavioral biometrics, device intelligence, KYC fraud prevention, and account takeover prevention fit into a practical fraud prevention strategy. Key takeaway: The short version? The best fraud prevention tools do not matter much if your fraud prevention vendor strategy creates a stack that cannot communicate. Real-time fraud prevention is about building systems that act before the loss, support the people making the decisions, and scale without collapsing under their own complexity. Am I being too optimistic? Probably. But that's the conversation. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Accordion dropdown close Transcript > ( Accordion dropdown) Optimized Episode Title: AI Fraud Prevention: Why Leaders Choose Worse Fraud Tools Episode Title: Why Leaders Choose Worse Fraud Tools Meta Title: AI Fraud Prevention: Why Control Matters More Than Accuracy Meta Description: AI fraud prevention only works when teams trust the decisions. Why control, KPIs, explainability, and strategy visibility matter more than scores... URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/ai-fraud-prevention Primary Keyword: AI fraud prevention Secondary Keyword: rules vs AI in fraud prevention fraud prevention AI AI fraud detection machine learning fraud prevention fraud detection rules fraud risk management fraud AI tools fraud decisioning chargeback rate optimization AI trust in fraud prevention Why Leaders Choose Worse Fraud Tools Release Date Episode #3 In this episode, I start with a slightly strange moment at the Mastercard offices. I was catching up with someone I know and he told me I had started pushing a new narrative. Okay. Apparently, the narrative was that rules are better than AI. Honestly, I get why it looked that way. I talk about rules vs AI in fraud prevention quite a bit. But that is not really the point. The point is control. AI fraud prevention, fraud prevention AI, AI fraud detection, machine learning fraud prevention, all of it sounds great until the person responsible for money movement and customer acquisition has to approve the change. Then accuracy is not the only thing that matters. Trust matters. Explainability matters. Strategy visibility matters. And if leaders do not feel in control, they will choose worse fraud tools. Not because they are irrational. Because breaking the business is, technically speaking, not a good look. What you'll hear in this episode: ● A breakdown of why the "rules vs AI in fraud prevention" debate misses the bigger issue ● Why leaders often choose fraud detection rules over stronger AI fraud tools ● How fraud risk management changes when the process touches money movement and customer acquisition ● Why fraud decisioning depends on trust, not just model accuracy ● What fraud AI tools often get wrong about explainability ● How chargeback rate optimization can become more useful when users can compare low, medium, and high-risk strategies ● Why AI trust in fraud prevention depends on clear KPIs, plain answers, and visible tradeoffs ● Listeners can expect a conversation that moves from "which tool performs better?" to the more uncomfortable question: who actually feels safe enough to make the decision? Who should listen: ● Fraud leaders and fraud operators ● Risk and compliance teams ● Product teams building fraud AI tools ● Financial institution leaders evaluating AI fraud prevention ● Fraud technology vendors and solution architects Anyone involved in fraud decisioning, chargeback rate optimization, or machine learning fraud prevention Basically, if you have ever looked at a model and thought, "The performance is better, so why won't they use it?" this one is for you. Episode notes: (Accordion dropdown) This episode is for anyone who has ever watched a "better" fraud tool lose to a simpler one and thought, wait, how did that happen? I'm unpacking why AI fraud prevention does not win on accuracy alone. In fraud decisioning, control is part of the product. If leaders cannot see the strategy, understand the KPI tradeoffs, or explain what happens when something goes wrong, they are going to choose the tool that feels safer. Even if it is worse. We'll get into why fraud detection rules still have so much staying power, why machine learning fraud prevention often feels harder to trust than it should, and what fraud AI tools need to show users before anyone is comfortable handing over decisions tied to money movement, customer acquisition, and chargeback rate optimization. No dramatic anti-AI rant. Just a practical look at why AI trust in fraud prevention is harder than vendors want it to be, and why the path forward probably starts with simpler strategy visibility, clearer KPIs, and fewer dashboards that look like they were designed to impress a conference audience. Plain answers people can actually use. Key takeaways: The AI model may be stronger. The machine learning fraud prevention strategy may perform better. The optimization may be real. But if the person making the decision does not feel in control, none of that matters very much. So maybe the better question is not, "How do we prove AI is more accurate than rules?" Maybe the better question is, "How do we make AI fraud prevention feel safe enough to use?" Less exciting. Probably more useful. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Accordion dropdown close Transcript > ( Accordion dropdown)

Catching up with friends is one of my favourite "business" activities. When I was a tech founder, I was pretty bad at finding time for friends. It was all planning sessions, one on ones, and endless emails. But as an advisor, catching up with friends is 50% of the job. And what's even nicer is that I get to meet a lot of new, interesting people as well. So here I was at the Mastercard offices, catching up with a friend I got to know through LinkedIn, but actually meeting them in person for the first time. Then at a certain point in our conversation, he tells me, "I see that you recently started pushing a new narrative in your writing." And he said that with this knowing smile. And that caught me by surprise, because I wasn't aware I'm consciously pushing any narrative. So I asked, "Oh really? Which one is that?" "Rules are better than AI." "Ah." I can see why people would think that I think that. But honestly, I don't think it's about rules versus AI. It's about something entirely different. Because here's the thing: people say they care about performance, but in my experience, what they actually care about is control. Here's what I mean. I tend to talk quite often about the ongoing debate of "rules versus AI." That's because there are a lot of things to unpack here: accuracy, coverage, speed, explainability, costs, performance degradation, etc. But it all comes down to one point: control. Or, to put it more accurately, the sense of control. Here's what we tend to forget: in fraud prevention, we touch the most sensitive business processes, money movement and customer acquisition. If you mismanage one of those, you hurt your business. And why do we tend to forget about this fundamental reality? Because as fraud fighters, all we see in front of us is optimization. Shaving a few milliseconds here and a few basis points there. So while fraud fighters often want to optimize by introducing changes, leaders see the changes themselves as a risk. The risk of accidentally breaking the system. And that tension, between small optimization gains versus the risk of complete business disruption, is exactly what drives paralysis. And I get it. Sometimes the gamble, what you have to lose, seems too high. Now, fraud teams, both internal and on the vendor side, often fail to realize that. They focus on what can be gained and ignore what can be at risk. And this is especially true when fraud teams push for adopting new technologies like AI or machine learning. Because let's face it, the vast majority of companies don't use AI or ML as part of their core product, and they're simply not used to it. And if they are not used to handling AI/ML, they won't understand it. And if they don't understand it, they won't trust it. And if they don't trust it, they won't feel like they're in control. And how can we expect a business to give up control over its most critical processes? So it's no wonder, then, that companies choose to utilize rules over AI. It's not about understanding the score of a specific event, it's about having a solid understanding of how decisions are being made as a whole. Rules offer exactly that: simple heuristics, deterministic outcomes, transparent monitoring, straightforward KPIs. Are rules always more accurate? Are they always catching more fraud? Do they adapt well to mutating fraud patterns? Absolutely not. But they are safe. And "safe" wins RFIs. So, does that mean we should give up on AI? No, but we first need to think about how to have our users trust it. How? Well, that's the question I hope anyone building fraud tech is thinking about. I've seen some vendors take a servicing approach, either by offering a "bring your own model" configuration or by letting users train models directly on their platform. And indeed, being more familiar with the model definitely increases trust. But teams that have their own models are the minority. And let's face it, these are the teams that can, and are, used to handling complexity. But what about the ones that don't? How do we get them to adopt AI and ML? Personally, I find that a big part of it is helping the user understand which strategies are available to them, and what would be the effect of each. For example: choosing a KPI to optimize around, and then suggesting the user low, medium, and high risk strategies. Let's be more concrete. Say a user wants to optimize around chargeback rate, and they set the different risk appetites to be 0.2% for low, 0.3% for medium, and 0.5% chargeback rate for high. Now imagine if, after setting these requirements, I am presented with this table. Now, this might seem quite basic, but truth be told, you won't find it in most fraud AI tools. And these are just the basics. Imagine if you could double click on a chosen strategy and explore it further, segment it by regions, or flows, or products. That's the explainability that matters, not why a certain case got a fraud score of 12. Because, like with rules, the decisions we implement with AI are about top-level KPIs. Without knowing exactly how to change those, there's no chance I'm going to make any changes. So how do we retain the sense of control while using AI? We let the user dictate which KPIs are important to them. We allow them to explore a number of alternatives. And we help them monitor the performance. No complex heatmaps, no overlapping ROCs, no fancy graph networks. Just plain answers anyone could understand. Because it's that "anyone" who will end up making these decisions, not the data scientist that trained the model. YouTube Description Why Leaders Choose Worse Fraud Tools Release Date Episode #3 In this episode, I start with a slightly strange moment at the Mastercard offices. I was catching up with someone I know and he told me I had started pushing a new narrative. Okay. Apparently, the narrative was that rules are better than AI. Honestly, I get why it looked that way. I talk about rules vs AI in fraud prevention quite a bit. But that is not really the point. The point is control. AI fraud prevention, fraud prevention AI, AI fraud detection, machine learning fraud prevention, all of it sounds great until the person responsible for money movement and customer acquisition has to approve the change. Then accuracy is not the only thing that matters. Trust matters. Explainability matters. Strategy visibility matters. And if leaders do not feel in control, they will choose worse fraud tools. Not because they are irrational. Because breaking the business is, technically speaking, not a good look. What you will hear in this episode: ● A breakdown of why the "rules vs AI in fraud prevention" debate misses the bigger issue ● Why leaders often choose fraud detection rules over stronger AI fraud tools ● How fraud risk management changes when the process touches money movement and customer acquisition ● Why fraud decisioning depends on trust, not just model accuracy ● What fraud AI tools often get wrong about explainability ● How chargeback rate optimization can become more useful when users can compare low, medium, and high-risk strategies ● Why AI trust in fraud prevention depends on clear KPIs, plain answers, and visible tradeoffs ● Listeners can expect a conversation that moves from "which tool performs better?" to the more uncomfortable question: who actually feels safe enough to make the decision? Who should listen: ● Fraud leaders and fraud operators ● Risk and compliance teams ● Product teams building fraud AI tools ● Financial institution leaders evaluating AI fraud prevention ● Fraud technology vendors and solution architects ● Anyone involved in fraud decisioning, chargeback rate optimization, or machine learning fraud prevention Basically, if you have ever looked at a model and thought, "The performance is better, so why won't they use it?" this one is for you. Episode notes: (Accordion dropdown) This episode is for anyone who has ever watched a "better" fraud tool lose to a simpler one and thought, wait, how did that happen? I'm unpacking why AI fraud prevention does not win on accuracy alone. In fraud decisioning, control is part of the product. If leaders cannot see the strategy, understand the KPI tradeoffs, or explain what happens when something goes wrong, they are going to choose the tool that feels safer. Even if it is worse. We'll get into why fraud detection rules still have so much staying power, why machine learning fraud prevention often feels harder to trust than it should, and what fraud AI tools need to show users before anyone is comfortable handing over decisions tied to money movement, customer acquisition, and chargeback rate optimization. No dramatic anti-AI rant. Just a practical look at why AI trust in fraud prevention is harder than vendors want it to be, and why the path forward probably starts with simpler strategy visibility, clearer KPIs, and fewer dashboards that look like they were designed to impress a conference audience. Plain answers people can actually use. Key takeaways: The AI model may be stronger. The machine learning fraud prevention strategy may perform better. The optimization may be real. But if the person making the decision does not feel in control, none of that matters very much. So maybe the better question is not, "How do we prove AI is more accurate than rules?" Maybe the better question is, "How do we make AI fraud prevention feel safe enough to use?" Less exciting. Probably more useful. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Accordion dropdown close Transcript > ( Accordion dropdown) Optimized Title: Fraud Prevention Strategy: Why I joined Sardine Episode Title: Why I joined Sardine Meta Title: Fraud Prevention Strategy: Content and Real-World Risk Meta Description Fraud prevention strategy requires aligning product with real-world needs. How practitioner-led content shapes what teams actually need from vendors. URL: https://www.sardine.ai/media/the-saturday-fraud-strategist/fraud-prevention-strategy Focus Keyword: fraud prevention strategy Secondary Keywords fraud prevention platform fraud prevention consulting fraud prevention solutions fintech fraud prevention enterprise fraud prevention Why I Joined Sardine Release Date Episode #2 I wanted to take a step back and talk about something a bit more personal, but also very relevant to how I think about fraud prevention strategy. It's been six months since I joined Sardine. And I figured it makes sense to explain how I got here, because honestly, the decision wasn't hard, but it wasn't simple either. This isn't just about changing roles. It's about moving from fraud prevention consulting into something broader, where I can connect content, product, and strategy in a way that actually helps fraud fighters do their job better. And along the way, it raises a bigger question: what does an effective fraud prevention strategy actually look like when you've been on the practitioner's side for long enough? What you'll hear in this episode: ● A personal breakdown of why I moved from solo consulting back into a team ● Why "freedom" in consulting isn't always what it seems ● How I think about fraud prevention strategy after years in the field ● What made Sardine stand out as a fraud prevention platform ● Why practitioner-led content matters more than ever ● How fraud prevention solutions should actually be built and evaluated Who should listen: ● Fraud fighters working in fintech fraud prevention and enterprise fraud prevention ● Risk, compliance, and product teams evaluating fraud prevention solutions ● Professionals working in fraud prevention consulting ● Anyone thinking about the gap between vendor promises and real-world fraud operations ● Anyone trying to build or choose a fraud prevention platform that actually works If you've ever asked yourself whether the tools you're using really solve your problems, this one will probably resonate. Episode notes: (Accordion dropdown) This episode is for anyone who has ever looked at the "freedom" of consulting and thought, okay, but why does this still feel weirdly stressful? I'm talking about what it felt like to join a team again after 11 years on my own, and why that decision was not as simple as "employee good" or "consulting bad." Honestly, nothing in B2B fraud prevention is ever that clean. Annoying, but true. We'll get into the uncertainty behind consulting, the strange emotional math of revenue forecasting, and why building useful fraud prevention content is harder than just publishing another polished thought leadership piece that says almost nothing. The bigger point is this: fraud teams do not need more noise. They need clearer thinking, better strategy, and vendors who actually understand practitioner reality. So this episode is partly personal update, partly fraud prevention strategy conversation, and partly me admitting that, six months in, joining the team actually makes sense. And at the same time, it feels like we're just getting started. Suspiciously positive. But here we are. Key takeaways: So yeah, this episode is a bit different. More personal, less technical. But I think it ties back to something important. Fraud prevention strategy isn't just about tools or models. It's about understanding the problems, the tradeoffs, and the people doing the work. And for me, joining Sardine was really about that. Finding a way to take everything I've learned from fraud prevention consulting, and apply it in a way that's a bit more focused, a bit more structured, and hopefully, a bit more useful. Am I being too optimistic here? Probably. But so far, it feels like the right call. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Accordion dropdown close Transcript > ( Accordion dropdown)

So it's been exactly six months since I joined Sardine. And I wanted to take the opportunity and share how and why I made this decision. Now, before we get started, I want to make it clear. Making that decision wasn't hard, but it wasn't simple either. And I think it makes sense for me to share my thoughts around it. Why wasn't this decision so simple? Well, for the first time in 11 years, I was going to become an employee again. When I confided my decision to some of my closest friends, I got a lot of questions. Why join a team? Why Sardine? Why now? What about Native Risk? And I'm sure that some of my readers had the same questions going through their heads when they saw my announcement back then. So I want to share my thought process, how I reached this decision, and what's next for me. Let's start with why join a team in the first place. The million dollar question. To answer that, I need to share why I started solo consulting in the first place. When Fraugster, my previous company, went through bankruptcy, I wasn't exactly sure what I wanted to do next. Everything was on the table, from starting a new venture, to joining a scale up, to even becoming a woodworker (real story). But after a few months of soul searching, and speaking about it with my partner, I realized that I want to orient my life around my family for a change. In the nine years I've been a founder, it always felt like I was juggling two competing sets of priorities: work and family. And to be frank, usually work won. Unfortunately, but that was reality. But regardless of which side "won" my attention, I always felt guilty towards both parties. It felt like I'm always coming up short. My life also changed. I became a father, we were planning a second kid (who is now almost two years old!), and we were thinking seriously about leaving Berlin, which we did last year. To me, working solo seemed like the answer for it all. No co-founders, no employees, no boss, no investors. I can work with whomever I want, whenever I want, from wherever I want. This was the freedom I yearned for. And consulting worked out great for a while. I had a blast, I grew tremendously, I re-ignited my passion for fraud prevention, and yeah, I also made a decent profit. But after two and a half years, I realized that the sense of freedom I was holding onto was an illusion. How come? Well, if you've ever worked in the B2B services business, you know that forecasting your revenue is practically impossible. Some six-figure clients would materialize from thin air, and we'd start an engagement the following week. Other clients could take long months to decide if and what they wanted to do together. It put me in an awkward position. Even when I signed big deals, I was still anxious. "What if that's the last big deal for this year?" The result was that I held myself back from anything but basic spending. Not really investing in my business on one hand, but also not taking much-needed holidays on the other. And it all hit me when I took my first vacation this last summer. Even though I was making good money and in control of my time, it was the first family vacation in three years. That's when I realized I craved some certainty in life. And that's when I started thinking seriously about joining a team. OK, why Sardine? Looking at what kind of role I wanted to take on, I knew two things. It had to be a remote role (you know, I live in southern Spain, and the biggest tech company around me is the local garage), so it had to be remote, and it had to be part time. That meant choosing between two options. The first would be to go back to my roots and take a leadership role in data, product, or engineering in the fraud prevention space. And while there were plenty of such open positions, I figured that the likelihood of me keeping it part time was low. The second option would be to capitalize on the brand and presence I've built over the years and take an evangelizing or strategy role with one of the vendors in the space. Not so many opportunities, but it got me really excited. If there was anything I really enjoyed in my solo career, it was delivering high-impact, educational content. It was always a passion of mine, and the last few years have been immensely fulfilling in that regard. But here's the interesting part. I actually only thought about such a role because, just before that eye-opening summer vacation I mentioned, the team at Sardine reached out to see if there was a chance for us to collaborate. Back then I was hesitant, as many of my prospects were competitors. But once I made up my mind, they were my first call. But it wasn't only that. Watching Sardine throughout their, our, journey left me with a strong impression. First, they really knew what they were talking about. So many founders today come from outside the industry, and it's definitely not the case with Sardine, and it shows. Second, their product was incredibly impressive. The first fraud orchestrator, designed for fintechs, FIs, and enterprise merchants, low code and yet sophisticated. I had never seen something like that before, and to be honest? I was glad Fraugster went out of business just before we had to compete with them directly. But the thing that I liked most was different. They were absolutely killing it with their marketing and branding. Every bit of it reminded me of what guides my own content: value first, educational, and practitioner led. This was exactly the team I knew I would be proud to be part of. So, what do I actually do at Sardine? Glad you asked! My role bridges two worlds I'm passionate about: creating content and shaping fraud prevention strategy. This means that on one hand, I'm contributing to Sardine's educational content, publishing best practices, insights, guides, and from now on also interviews, all designed to cut through the noise and speak directly to you, fraud fighters. Beyond content, I'm also working with the product and leadership teams to make sure that what we build actually solves real problems. Believe me, after 16 years in the trenches, I know the gap between what vendors promise and what practitioners need. My job is to close that gap. And how's it been in those six months I'm here? Simply put? I'm having a blast! Looking back at everything I wrote, everyone I met, and now, everything I'm recording, I actually can't believe it's been just six months. And let me tell you this: we're only getting started. YouTube Description Why I Joined Sardine Release Date Episode #2 I wanted to take a step back and talk about something a bit more personal, but also very relevant to how I think about fraud prevention strategy. It's been six months since I joined Sardine. And I figured it makes sense to explain how I got here, because honestly, the decision wasn't hard, but it wasn't simple either. This isn't just about changing roles. It's about moving from fraud prevention consulting into something broader, where I can connect content, product, and strategy in a way that actually helps fraud fighters do their job better. And along the way, it raises a bigger question: what does an effective fraud prevention strategy actually look like when you've been on the practitioner's side for long enough? What you will hear in this episode: ● A personal breakdown of why I moved from solo consulting back into a team ● Why "freedom" in consulting isn't always what it seems ● How I think about fraud prevention strategy after years in the field ● What made Sardine stand out as a fraud prevention platform ● Why practitioner-led content matters more than ever ● How fraud prevention solutions should actually be built and evaluated Who should listen: ● Fraud fighters working in fintech fraud prevention and enterprise fraud prevention ● Risk, compliance, and product teams evaluating fraud prevention solutions ● Professionals working in fraud prevention consulting ● Anyone thinking about the gap between vendor promises and real-world fraud operations ● Anyone trying to build or choose a fraud prevention platform that actually works If you've ever asked yourself whether the tools you're using really solve your problems, this one will probably resonate. Episode notes: (Accordion dropdown) Why joining a team was not simple So for the first time in 11 years, I was about to become an employee again. Not exactly something I took lightly. The illusion of freedom in fraud prevention consulting Revenue forecasting in B2B consulting is... not great. Deals come out of nowhere, or they take months. And even when things are going well, you're still asking yourself, "Is this the last one?" Building content that supports real fraud prevention strategy A big part of what I do now is content. Guides, insights, interviews, things that are meant to actually help fraud fighters. There's a lot of noise in fraud prevention solutions. A lot of claims, a lot of features. Not always a lot of clarity. So the goal here is simple: cut through that and focus on what actually matters in fraud prevention strategy. Connecting product to practitioner reality The other side of my role is working with product and leadership. What vendors promise and what practitioners need are not always the same thing. My job is basically to sit in that gap and try to close it. What comes next So, six months in? Honestly, I'm having a blast. It's one of those things where you look back and think, "Okay, this actually makes sense." And at the same time, it feels like we're just getting started. Key takeaways: So yeah, this episode is a bit different. More personal, less technical. But I think it ties back to something important. Fraud prevention strategy isn't just about tools or models. It's about understanding the problems, the tradeoffs, and the people doing the work. And for me, joining Sardine was really about that. Finding a way to take everything I've learned from fraud prevention consulting, and apply it in a way that's a bit more focused, a bit more structured, and hopefully, a bit more useful. Am I being too optimistic here? Probably. But so far, it feels like the right call. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Accordion dropdown close Transcript > ( Accordion dropdown) Optimized Title Dating App Identity Theft: My Stolen Identity is Cheating on My Wife Episode Title: My Stolen Identity is Cheating on My Wife Meta Title My Stolen Identity Was on a Dating App Meta Description A fraud expert woke up on a dating app he never joined. What follows is a trail straight to APP fraud and a tale of stakes the dating industry keeps ignoring. URL https://www.sardine.ai/media/the-saturday-fraud-strategist/dating-app-identity-theft Focus Keyword dating app identity theft Secondary Keywords fake dating profiles online dating scams romance scams app fraud prevention identity verification systems identity verification failure account takeover detection My Stolen Identity is Cheating on My Wife Release Date Ep. #1 A few months ago, I woke up registered on a dating app. I'm in a 13-year relationship. I did not sign up. Someone used my email to create a profile on Coffee Meets Bagel. When the platform froze the account, the scammers opened another one. Same email. Same details. Instantly. By the third account, customer support still hadn't replied. What starts as fake dating profiles doesn't stay there. They become romance scams. Then a loss your FI absorbs while the dating platform moves on with no accountability for the identity verification failure and no skin in the game. Dating platforms have no chargebacks, no regulatory pressure, and no reason to fix the lack of account takeover detection. So who actually bears the cost, and why are we still waiting for them to care about APP fraud prevention? What this episode covers ● How dating app identity theft becomes a gateway to romance scams and APP fraud ● Why dating platforms have no structural incentive to prevent account misuse ● What the upstream fraud trail looks like, and who ends up paying downstream Who should listen ● Fraud ops teams at FIs and fintechs ● Teams working with identity verification systems ● Risk and compliance professionals tracking APP fraud vectors ● Anyone watching romance scam trends Key takeaway Dating platforms are the upstream origin of a fraud chain that ends at your institution, and they have every incentive to keep ignoring it. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Transcript > ( Accordion dropdown) Links Mentioned Musubi - AI moderation for dating apps Tech Against Scams coalition

This story happened a few months ago. I woke up registered on an online dating app. Now, it was kind of odd, because I didn't really remember doing that, and not to mention, I've been in a happy relationship for 13, 14 years now. "Did you sign up? You signed me up to a dating app as some kind of a joke," I asked my better half, and the passive aggressive stare I got back told me it was a mistake. So I took a closer look. Someone definitely used my email to sign up for Coffee Meets Bagel under my name. Wait, are they using my pictures as well? I tried downloading the app to see if I could access the account, immediately notifying my partner, of course. But weirdly, logins aren't even connected to email. You could either log in through Facebook or directly with a mobile number. Okay, this got me anxious. Is there someone else impersonating me who's in the process of scamming people right now? What happens if I get blamed for it? Honestly, I didn't really know what to do. I tried contacting customer support through email, but I only got a generic auto reply, and I didn't really think I'd get much from them. Anyway, a day later, I got an email that my account was frozen. And guess what? The scammers already opened a new account with the same details, and here's how it looked in my inbox. I gotta say, it was pretty surreal watching my identity being stolen right in front of my eyes without being able to do much about it. By sheer chance, on the very same week, I got to meet the team behind Musubi. They are experts in the space and recently launched their AI moderating product for dating apps and other digital services, and they weren't surprised. This happens all the time, they told me. I showed Alice from the team the screenshot you've just seen, and she wrote back saying, looking at this, it definitely seems like: A. At least three accounts were created with your email. B. Two of these three were suspended, yet they were able to create another account with the same email right away. C. Weirdly, it looks like at the end, even after two were frozen, you were still getting two of every email. So it's possible that one was frozen and then unfrozen, in addition to the one that was never frozen. Not a good look, especially if the third account remained frozen and all were under the same email. Lastly, she writes, they didn't get back to me the same day, which left time for another account to be created yet again. And this matched my impression. I found it especially surprising that multiple accounts can exist with the same email, and even more so, as Alice mentioned, while some of them are frozen. Now you've got to ask yourself, if the same email was used, what other assets were linking all of these accounts? And I could imagine that the device and the phone number were also the same, at the very least. And speaking of which, it's not even clear why the scammers chose to use my email. They weren't required to go through any email verification, so why bother, anyway? The only thing I can think of is they wanted an account with proven history to bypass any risk checks at sign up. In any case, here are some best practices Alice shared with me, with which I totally agree. If an account is frozen, all metadata linked to the account should also be frozen, such as email, IP address, device ID, phone number, etc. Customer support tickets should feed into moderation systems, so emails like yours are prioritized and looked at quickly. Email verification is a big one. Don't allow two accounts at the same time with the same email address. Okay, yeah, it's a cool story. A fraud expert whose identity got stolen, all right. But this is LinkedIn post material, not necessarily a full video. So why did I choose to dedicate an entire video to discussing it? Because what starts as identity theft on a dating site usually turns very quickly into romance scams, and then into app fraud. And who's bearing the damages? The victims and the financial institutions. Thing is, while these scams originate on dating sites, we see how poorly both of them are equipped to detect and properly stop them. So it's not a surprise, given they're not exposed to any damages or scrutiny. Will that change anytime soon? From what we've seen so far, it's hard to imagine. But what I think is possible is to exploit the collaborative potential that hides here. Think about it. What have we actually seen here? It appears that upstream in the scam journey, fraudsters are acting carelessly. And imagine how much data we can gather about both fraudsters and victims in such cases. What if we could expose it downstream to block monetization attempts? The challenge here, as always, is to foster collaborations that don't seem trivial at first. In fact, last year we saw a huge announcement from Facebook, Venture.com, Coinbase, and others about launching an anti-scam alliance, exactly for this reason. I'm not exactly sure what happened with it outside of the PR story. But still, in that regard, we often talk about the convergence of fraud and cybersecurity spaces. But what about fraud and trust and safety? Except for a few organizations, these teams usually don't find themselves collaborating much, if they even exist in the same business together. But maybe it's time to explore such initiatives more seriously, with the rise of romance scams and pig butchering scams and other forms of app fraud. Am I being too optimistic here? Probably. I just wish my identity wasn't out there doing who knows what. YouTube Description My Stolen Identity is Cheating on My Wife Release Date Ep. #1 A few months ago, I woke up registered on a dating app. I'm in a 13-year relationship. I did not sign up. Someone used my email to create a profile on Coffee Meets Bagel. When the platform froze the account, the scammers opened another one. Same email. Same details. Instantly. By the third account, customer support still hadn't replied. What starts as fake dating profiles doesn't stay there. They become romance scams. Then a loss your FI absorbs while the dating platform moves on with no accountability for the identity verification failure and no skin in the game. Dating platforms have no chargebacks, no regulatory pressure, and no reason to fix the lack of account takeover detection. So who actually bears the cost, and why are we still waiting for them to care about APP fraud prevention? What this episode covers ● How dating app identity theft becomes a gateway to romance scams and APP fraud ● Why dating platforms have no structural incentive to prevent account misuse ● What the upstream fraud trail looks like, and who ends up paying downstream Who should listen ● Fraud ops teams at FIs and fintechs ● Teams working with identity verification systems ● Risk and compliance professionals tracking APP fraud vectors ● Anyone watching romance scam trends Key takeaway Dating platforms are the upstream origin of a fraud chain that ends at your institution, and they have every incentive to keep ignoring it. Not ready to stop the conversation about my, and hopefully your, favorite subject? Subscribe to The Saturday Fraud Strategist newsletter. Transcript > ( Accordion dropdown) Finally! I am so happy to share with you all that The Saturday Fraud Strategist is now a podcast! WHAAAAAT??? YEAH! On top of my weekly newsletter, you could now listen to AND watch me talk about my - and hopefully your - favorite topic - fraud strategy. AND, from time to time, I'll be hosting operators and founders to discuss where the industry is headed and what we - fraud fighters - should pay attention to. I must say - I'm super excited, and if I'm being honest, a bit nervous about all of this. I've been debating with myself whether to start a podcast FOR AGES but kept putting it off. But now - this teaser is out, so I guess there's no turning back. So, if you want to join me for the ride, head over to Sardine's website and subscribe now! Are you ready? Am I ready? We'll find out next Saturday! General for reference Fraud strategy isn't just a risk problem. It's a business problem. After 16 years in the industry and a year of building one of LinkedIn's most engaged fraud communities, Chen Zamir is going deeper. No news digests. No vendor pitches. No fluff. Just practical, experience-backed thinking on fraud strategy, and what it actually takes to get it right. Each episode covers the stuff that doesn't fit in a LinkedIn post: the nuance, the hard calls, the frameworks you can actually use. The goal is simple: leave every episode with something worth acting on. Whether you're a fraud professional, a product manager, or a C-suite exec trying to understand the risk-growth tradeoff, you're in the right place. New episodes every Saturday. YouTube Fraud strategy that's actually practical. 16 years of experience, zero fluff, one actionable takeaway every episode. Chen Zamir goes deep on fraud strategy at the intersection of business growth, customer experience, and risk management. No news digests, no vendor pitches, no recycled content. Just real talk for fraud professionals, product managers, and executives who want to think sharper about risk. New episodes every Saturday. Subscribe and join a community that's already changing how the industry thinks about fraud. Got a question or topic you want covered? Drop a comment or reach out directly. Chen reads everything. Spotify/Apple Fraud strategy. No fluff. Real talk from 16 years in the industry, every Saturday. Chen Zamir breaks down the decisions, frameworks, and hard calls behind fraud strategy for professionals who want practical insights they can actually use. Whether you work in fraud, product, or the C-suite, every episode leaves you with one clear takeaway. New episode every Saturday. Subscribe so you never miss one. @nathan.destigter@sardine.ai "really hard for an AI agent to behave exactly like a human," @vini.carvalho@sardine.ai - note the slight edit to the title