68 min
Fraudologyへお帰りなさい。
今回のエピソードでは、何年も前から番組に再登場してもらいたいと思っていたゲスト、ホリー・サンドバーグさんをお迎えしています。ホリーは不正対策オペレーションを率い、MRCのアメリカ地域諮問委員会のメンバーを務め、さらにReverbでトラスト＆セーフティ部門のディレクターを務めてきました。ですから、不正対策においてAIにできること・できないことを語るうえで、まさに私がその議論に参加してほしいと思う理想的な人物なのです。
今まさに、誰もが「AIは不正対策チームを置き換えるのか」「不正対策チームをもっと効率的にするのか」「あるいは、この10年間ずっと私たちを悩ませてきた問題を、なぜか全部解決してくれるのか」を判断しようとしている、あのよくある局面にいます。そう、まるで不正対策の戦略は、チャットボットが現れるのをずっと待っていただけかのように。
そうでもありません。
このエピソードの本質は、人間が介在する形での不正検知と、そこで人間の役割が「念のためのバックアップ」ではなく、まさに中核となる戦略そのものである理由についてです。AI は、不正対策チームのスピードを上げ、パターンを要約し、調査を支援し、シグナルを浮かび上がらせることができます。しかし、ドメインの専門知識そのものを置き換えることはできません。AI は、あなたの不正対策チームが知っていることを知りません。あなたの会社のリスク許容度、顧客層、イレギュラーなケース、過去の攻撃、そしてチームが時間をかけて構築してきた独自のシグナルを理解しているわけではないのです。
そこからが本当に面白くなるところです。
ホリーと私は、LLMがあふれるこの世界において、なぜシニアレベルの不正対策リーダーシップが依然として重要なのか、なぜモデルドリフトを軽く見てはいけないのか、そしてなぜ不正防止戦略には、現実世界で不正がどのように行われているかを本当に理解している人たちが不可欠なのかを掘り下げていきます。オープンソースのデータだけではなく、モデルの出力だけでもなく、実際に不正オペレーションが日々行われている、あの混沌とした現場環境においてこそ、です。
このエピソードでお届けする内容：
- ホリー・サンドバーグが、シニアレベルの不正対策リーダーシップはLLMに置き換えられないと考える理由
- 人間参加型の不正検知は、どのようにAIによる不正防止を強化するのか
- なぜドメインの専門知識はオープンソースデータから得られないのか
- 不正対策チームが理解すべきAIの幻覚とモデルドリフトのポイント
- 不正対策の専門家がAI運営委員会や部門横断的な戦略策定にどのように関与できるか
- なぜAIにおける「確実性への偏り」が不正対策業務にリスクを生みうるのか
- 変化の激しい市場環境で、キャリアのレジリエンスやレイオフ、静かな再採用、そして自分の価値をどう証明するかを考える方法
このエピソードは次のような方におすすめです：
- 不正対策業務に携わっており、AIが実際にどのように役立つのかを理解しようとしている
- 不正対策、トラスト＆セーフティ、リスク、または決済チームを率いており、現実的で実行可能なAI不正防止戦略を必要としている
- LLMが人間の不正対策の専門性を置き換えてしまうことを懸念している
- モデルドリフトやAIの幻覚、そして不正検知モデルのガバナンスについて、より深く理解したい
- 自分の不正対策の専門性が今も重要であることを、リーダー層に実践的な方法で示したいと考えている
このエピソードを気に入っていただけたら、ぜひ購読して、iTunes、Spotify、YouTube、または普段お使いのポッドキャストアプリでレビューをお願いします。番組を多くの方に知ってもらう大きな助けになります。
エピソードの概要と主なポイント
なぜ人間を介在させた不正検知が重要なのか
ヒューマン・イン・ザ・ループ型の不正検知は、形式的に人を関与させておくことが目的ではありません。不正対策の専門知識を持つ人が、ツールが出力した結果をきちんと解釈し続けていることを確実にするための仕組みなのです。
AIはパターンを見つけ出すことができます。活動内容を要約することもできます。データをより速く処理できるよう、チームを支援することができます。
しかし、不正対策チームは依然として、より難しい問いを投げかけなければなりません。
このシグナルは本当に重要なのでしょうか？この顧客行動は、このビジネスにとって通常のものと言えるでしょうか？モデルは有益なものを捉えているのか、それとも単なるノイズに反応しているだけなのか？このパターンは新しいものなのか、それとも以前にも見た手口なのでしょうか？
それが、AIを道具として使うことと、AIに静かに意思決定を任せてしまうこととの違いです。
- 人による監督があることで、チームは不正の兆候を文脈に即して解釈できるようになります
- 不正防止の戦略は、自動化だけでなく人間の判断に依然として依存しています。AI は調査を支援することはできますが、最終的な責任を負うことはできません。
- 不正検知における人間の専門知識は、自信はあっても誤った判断をチームが避けるのに役立ちます
なぜAIの幻覚は不正リスクを生むのか
AIの幻覚は、単なるチャットボットの問題ではありません。
不正対策の現場では、AIの幻覚は誤った指示や質の低い調査サマリー、誤解を招くリスク解釈、そして自信満々な文章ゆえに正しく聞こえてしまう誤った判断へとつながりかねません。
そして、それこそが不正対策チームが重視すべき点です。
不正に関する知識の多くは公開されていません。そうした知識は、社内のケースノート、顧客の行動パターン、チャージバックデータ、加盟店の行動、トラスト＆セーフティのワークフロー、そして長年の実務経験の中に蓄積されています。そのため、不正防止におけるLLMが、不完全な情報やオープンソースの情報だけをもとに判断しようとすると、本当に重要な文脈を持ち合わせていない可能性があります。
そしてそれがなければ、彼らは足りない部分を想像で補ってしまうかもしれません。
それはたいてい良い結果にはなりません。
- AIの幻覚は、不正に関する判断に誤った自信を与えてしまう可能性があります
- LLM は、公開データでは利用できない独自の不正パターンを見逃してしまう可能性があります
- 不正対策チームは、AI が生成した要約、推奨事項、およびリスク解釈を検証する必要があります
- ドメインの専門知識があれば、チームは回答が現実とかみ合っていないときにそれを見抜くことができます
なぜドメインの専門知識が依然として不正防止の手段となるのか
ドメインの専門知識は、単なる「あると良いもの」ではありません。これは、不正対策チームが持つ最も強力なコントロールの一つです。
経験豊富な不正対策リーダーは、犯罪者がどのように手口を変えてくるかを理解しています。支払いフローがどのように変化するかも把握しています。指標が一見安定しているように見えても、その裏側で行動が変わり始めているタイミングを見抜きます。また、コントロールが技術的には機能していても、もはや本質的な問題を解決できていない状況も理解しています。
そのようなパターン認識は、汎用的なモデルの出力から得られるものではありません。
それは実際に手を動かして取り組むことで身につくものです。
これこそが、AI 主導の環境においても不正防止のリーダーシップが依然として重要である理由です。シニアの不正対策担当者は、単にキューを確認しているだけではありません。彼らは、リスク許容度、顧客体験への摩擦、エスカレーション、ツール選定、ベンダーのパフォーマンス、そして次にどこで新たなリスクが顕在化しうるかについて、判断を下しています。
- ドメインの専門知識があれば、詐欺対策チームは新たな行動パターンをいち早く見抜くことができます
- 不正対策部門のリーダーシップは、データをビジネスの文脈と結び付ける
- モデルのドリフトには、「正常」がどうあるべきかを理解している人材が必要です
- 不正検知モデルの監督は、経験豊富なオペレーターが早い段階から関与していると最も効果的に機能します
なぜ不正対策チームはAI戦略に参画すべきなのか
もしあなたの会社がAIロードマップを策定したり、AI推進委員会を設置したり、自動化をどこに導入すべきか検討しているのであれば、その議論には不正対策チームが必ず参加すべきです。
ローンチしてからでは遅い。
前に。
不正対策チームはすでに、生成AIを悪用した詐欺によって攻撃の規模と質がどのように変化しているかを目の当たりにしています。より巧妙なフィッシング、より精巧ななりすまし、より高度なソーシャルエンジニアリング、より高速なテスト、そしてより説得力のあるストーリーが生み出されています。
そのため、企業が社内でAIの活用を始めるとき、不正対策の専門家は他のチームにはない視点をもたらします。彼らは、自動化がどこで役に立つのかを理解しています。また、自動化がどこに新たな死角を生み出すのかについても理解しています。
- AIリスク管理には、不正対策、トラスト＆セーフティ、法務、プロダクト、エンジニアリングを含めるべきです
- 不正対策チームは、自動化によってリスクが生じうる箇所を特定できます
- 部門横断的なAI戦略は、適切な監督なしにツールが導入されることを防ぐのに役立ちます。また、不正防止戦略は、顧客対応や取引ワークフローにおいてAIをどのように活用するかを方向付けるべきです。
なぜ不正対策の専門家は将来価値を示す必要があるのか
このエピソードでは、今まさに不正対策の担当者が直面している非常に現実的な課題についても掘り下げています。
AIはキャリアに関する議論のあり方を変えつつあります。
「これまでに自分が何をしてきたか」を語るだけでは不十分です。それももちろん大切ですが、より強い立場になるには、自分の専門性がこれからそのビジネスに何をもたらし、次に何を可能にするのかを説明することが重要です。
AIによる不正防止ツールの評価を手伝うことはできますか？ 損失につながる前にモデルドリフトを見抜くことはできますか？ 不正のパターンをビジネスリスクに落とし込んで説明できますか？ プロダクト、コンプライアンス、法務、エンジニアリングの各担当者に、不正リスクが実際にはどこに存在しているのかを理解させる手助けができますか？
それがその値です。
なぜなら、不正対策の専門性とは、単なる個別案件の対応ではないからです。それは、組織として蓄積された知識であり、現場での判断力であり、パターンを見抜く力でもあります。そして企業は、どれだけツールで代替できると信じ込もうとしていても、結局のところそうした専門性を必要としているのです。
最終的なポイント
AI は今後も不正対策の業務を変え続けていくでしょう。その点は明らかです。
しかし、最も強力な不正対策プログラムとは、人をプロセスから排除してモデル任せにするものではありません。AIを活用しつつ、適切な人間をプロセスに関与させるプログラムこそが、最も強力なものになるのです。
ビジネスを理解している人たち。不正のパターンを理解している人たち。何かがおかしいと感じ取れる人たち。結果をただ受け入れるのではなく、疑問を投げかけて検証できる人たち。
なぜなら、不正対策では、たとえスピードを上げても、正しい方向に進み続けていなければ意味がないからです。
つながる：Holly Sandberg | LinkedIn
- 不正対策業務のリーダー、MRC米州諮問委員会メンバー
- Reverbの元トラスト＆セーフティ部門ディレクター
つながるKarisse Hendrick | LinkedIn
- 「Fraudology」ポッドキャストのホスト
- 受賞歴のあるサイバー詐欺対策の専門家
- Eコマース不正防止コンサルタント
- スタートアップアドバイザー、基調講演者、そしてフォーチュン500企業向けコンサルタント
Guests
Episode transcript
Karisse Hendrick
00:57
Welcome to Fraudology Podcast, where we dive into the science and study of online fraud from the perspective of an e-commerce fraud fighter. I'm Karisse Hendrick. Welcome back to Fraudology. Well, today I have a guest that I have been wanting to come back on the podcast for literal years, and it's just hard to get a hold of, hard to nail down. But my guest today is Holly Sandberg. She is legitimately one of my favorite people in the world, not just in the fraud world. I have learned so much from our friendship, and when it comes to fraud leadership, and you know, fraud operations, and strategic direction. And so I wanted her to come on to help me answer a few listener questions. Holly has been in the fraud industry for over 15 years. She was the director of fraud for Paciolan and Ticketmaster, both in the ticketing industry, which, if you know e-commerce fraud, you know that's one of the trickiest industries to be in fraud. And then most recently at Reverb as the director of trust and safety, and has the next stop coming up soon. So, Holly, welcome to the podcast.
Holly Sandberg
02:06
Thank you. It's good to... I don't know how many years it's been, but I think we're going to do like a twice-a-decade thing as an ongoing commitment between the two of us, and I work really hard for that twice a decade, so I apologize and apologize at the same time.
Karisse Hendrick
02:22
It's okay. No, I'm just happy to get to when I can get you. If I had my way, it'd probably be twice a year. So, it's okay to play hard to get, but you're also super busy, so that's why it's hard. You know, I wanted to get you in between jobs, so that way you would actually have some time on your calendar.
Holly Sandberg
02:37
That works for me. That's when you can say all the good stuff, too, which is my cue to say that my views are my own and not the viewpoints of any of my past, present, future, current. Did I cover all the times there? Karisse Hendrick: Yeah, so fine print, you know, checkbox checked. Holly Sandberg: Then, yeah, we can just jump in.
Karisse Hendrick
02:57
Okay, cool. So, like I mentioned, I often go to you with questions, like a lot of people come to me with questions, and I'm their phone a friend, but you're my phone a friend. And so I thought that it would be, I've been saving up a few listener questions, and I wanted to have you answer them with me. So the first one has been asked in various ways multiple times over the last, like, six months, especially, and that is, I'm worried AI is going to replace my job. What can I do to convince my bosses that it shouldn't? And is that even the right question to be asking?
Holly Sandberg
03:30
I think it's a great starter question, and one of the things that I've said many times, and you've certainly nudged, is a kind word for it, and not just you, but other people have kind of said, like, you know, speak up more on LinkedIn and kind of put more content out there, and I think about it every once in a while. So maybe this will be my springboard to, like, it's the right first question to ask, but it certainly should not be the last, and I think there are better questions, but it is the question that's on a lot of people's minds right now. So I think it's a fine starting point, just don't let it be your ending point. So if you're worried, first of all, congratulations, you're sane and normal. Because if you're not, then maybe you need to send me a message, or find a therapist, or some other kind of mental health outlet, because everybody's worried right now. And, you know, most of the people in the fraud fighting world, in the payments world, are very smart, they're very astute, they're very analytical, they're exactly the kind of people who would read the room before maybe the average person who doesn't have that same kind of mindset. So it makes a lot of sense. You're in really good company, and it's certainly not something that you should feel bad about. As for what you can do to convince your bosses that AI shouldn't replace you, we could do five of these. Don't get any ideas, Karisse. I'm not saying I'll do five, but we could do on this. I think the first best advice that I would give is, volunteer, be visible, be as indispensable as you can be. Speak to, I guarantee you, you may not know that it exists or not, but you'd be hard-pressed to convince me that almost every company out there doesn't have a working group or a steering committee that is specifically working on bringing the company into an AI-forward sort of stance. Ask about it. If it's out there in the open, cool. Find out who runs it. Volunteer if it isn't. Ask if there is one. And if you find out the answer is yes, then say, I'd love to be a part of that. Nothing ventured, nothing gained. If you do succeed in getting a seat at that table, fantastic. I think that the trick then is show up with openness rather than cynicism, don't be a naysayer, bring solutions instead of problems, and be very careful, I'll say, about the way that you frame it. Like venting with your friends about being freaked out that the robots are going to replace you is cool, but there's a more professional and maybe more diplomatic way to handle that in the workplace, and, you know, save the venting for the happy hour. And think about all of the advances that you've made in the way that you've grown over your career. This is really no different. Show up and, you know, put your best foot forward in as many of those conversations that you can get yourself into.
Karisse Hendrick
05:54
I think that's really good. I think that's a really good starting point, is, you know, volunteering and working more cross-functionally as well. So I would add to that, the more you work with other departments and teams, the more visible you are, and they recognize how valuable fraud can be, and just how much of a landscape you can see in the transaction for e-commerce, and then just, you know, throughout the customer journey as well. You know, offer those insights to other departments and demonstrate that the robot can't do everything that you do, right?
Holly Sandberg
06:26
It can, and I think my last boss, who's amazing, changed my life, fantastic leader, would very often say her directs, you always assume good intentions, even if you find out later that they're maybe they're not quite as good, but they come to the table, assuming good intentions, right? Come to the table, assuming that it isn't, you know, some corporate overlord kind of Austin Powers-esque villain sitting on a velvet chair with a fluffy white cat and a monocle, who's just, like, drumming their fingers together and figuring out how much headcount they can cut, and that they really are approaching it from a how do we stay ahead of the curve. We not let this, the hyperspeed with which everything is happening right now, kill the company, let alone thinking about individual teams. Like, you have to think you're scared, but your leaders are probably scared too, and they're asking themselves, am I doing enough? Am I investing enough? Am I making the right calls? Do we have the right things on the roadmap? So, you know, approach that as a partner rather than in an adversarial way. I think that's probably the best way of doing it. And find a way to really love, and I don't think this is that difficult, actually, for people with an analytical mindset. Like, if you really actually start to work on upskilling, which hopefully everyone listening has, but even if you haven't started, there's a lot to love about how fast technology is moving. And if you think about the thrill of the catch for fraud fighters, the thrill of finding really rare, relevant precedent. If you're a lawyer, that makes the case for your client. The thrill of finding some sort of little understood function. If you're a neuroscientist, like these are all the things that make, if you love what you do, make you love what you do. And there's a way to incorporate the speed with which everything is advancing right now as an amplifier and a force multiplier, much more so rather than a replacement. And I think if you lead in that regard, it's going to be noticed, it's going to benefit you. And then I think what I would try to convey directly to your boss is the biggest mistake is treating AI as a budget-cutting exercise before understanding, before having a really firm understanding of what you're doing, before understanding the operating model. AI can absolutely reduce manual effort. Experienced risk teams do so much more than process queues. They identify emerging attacks, they interpret weak signals, they test controls, they spot drift. That's huge, huge, huge, huge. That one, if I remember one thing I said here, like, they spot model drift, that's a really big thing right now. They understand customer harm. They understand trade-offs. And if you take, like, one sentence to your boss, it's please do remember that the feedback loops that make AI useful are experienced risk teams. Stop, and like, really, really think about that. Like, if AI is useful for fraud, what got AI there was learning from experienced risk teams. So, I actually don't think it's as scary as maybe some people might think it is right now. Karisse Hendrick: Can you explain model drift a little bit for those that don't understand that? Holly Sandberg: Probably not as someone super technical. I think it's just, you know, your fraud analysts probably, if they're working with a model, whether it's a model that's been provided to you by a vendor, solution provider, partner, whether it's something that's bespoke and it's been built in house, may not be able to explain to you exactly how the model works. They're hopefully very, very involved in training it, but they can absolutely tell you when it suddenly isn't as effective as it used to be, and hopefully when they do spot that, they're listened to rather quickly, because that has a compounding effect. The longer it goes on, where you're going, oh, this is great, we had the eternal build versus buy sort of debate, we built something, it's really cool.
Holly Sandberg
10:00
Where I always say that build for build and buy is better than either build or buy. And then, but then you suddenly see that the things that it was knocking out of the park that you were so excited about aren't quite happening as much anymore, and that's, that speaks to having a regular cadence with the technical teams who can understand model drift better than I can explain it. It speaks to just establishing an operating cadence at the very beginning of rolling something like this out, rather than just being like, "Oh, we built that, it's cool, it's kicking ass, it's doing awesome things." And then when it isn't, you haven't had a conversation with the data science team that built it in five months, and you have to go be like, "Hey, remember me? I'm Dave," and start all over again. So I think that that's where, you know, your AI should upskill. There's so much talk right now about upskilling on AI. Your AI should upskill on your smartest humans and make sure that you lay out a really, really structured plan to make sure that that happens, and that you're not caught, you know, everyone's saying, on your back foot. Yeah, something about feet.
Karisse Hendrick
10:59
Yeah, and I know, like, in speaking with merchants, that some companies are putting out directives, like, you have to, you must be using Claude or Copilot, you know, for 25% of your job by the fall, things like that. And then there's also, because there's different AI sources, right, because there's also, you know, third-party vendors that are claiming to have AI, and I would caution, you know, we, I know of a merchant that actually, a couple that have now created AI governance groups to vet vendors who say that they do AI, and to understand, well, exactly how are you using AI? Because it's so new, you know, we can't just trust AI to do everything, and so they're, you know, really getting under the hood. How are you using it, and are using it the right way, and is it, you know, we're trusting you with our company data to make some really big and expensive decisions. Are you, do you actually use AI? If you don't, you know, why? Why say you do? Because in some cases, I think you and I both agree that fraud should be fought with both technology and humans, and you can't just do all of one, all of one without the other. And so I would suggest, you know, vetting your vendors for that, but then also holding them accountable and having those regular meetings with the technical team to understand, you know, is our model drifting? Are there hallucinations in the AI, right? Is AI just making shit up? Are they jumping to conclusions that they shouldn't be jumping to, or is it jumping to conclusions that it should be jumping to, because AI is still in it, not an A. It's not attacking the robots just yet, but yeah, that's those are a couple things that I would add. As far as that is, you know, depending on what type of AI you're being asked to implement. The other thing I would say too is, if you are using something like Lexis or OpenAI, you know, Claude, whatever, that you understand that the feedback loop is, or that not the feedback loop, you said the feedback loop, the information where they're getting the data, their data sources are usually open source. We don't talk about the good shit in fraud, right? Like, we don't talk about that, we don't publish it, right, because we don't want it out in the open. We have very few advantages against the bad guys, and how our technology works, and how to optimize that technology, and how to, you know, understand and diagnose new fraud attacks and other things like that, and then what to do about that specific type of fraud attack. All of those things we don't publish, right? So, instead, what you're getting as a data source is a lot of blog articles from vendors that are pretty vague. You're getting Reddit threads, which are coming mostly from fraudsters themselves. You know, you're getting that type of information. That's your data. So, you know, a couple weeks ago, or no, maybe a couple months ago now, Frank McKenna, Matt Vega, Mary Ann Miller, and I were on a group text thread, and I think Matt started it, but he started putting into, and I can't remember if it was ChatGPT or Claude, but he started putting in, you know, what is pizza fraud? Tell me about what you know. What's pizza fraud? And like Claude, or whoever he was using, made up some kind of, like, you know, it's when somebody steals a pizza from a local pizzeria. What? That's not really pizza fraud. Like Frank did a whole article on it, if you see it, but it's basically demonstrating, like, how much AI can just hallucinate. It'll never tell you that something doesn't exist. It'll just make shit up. And so that's really, you know, if you're depending on one of those, you know, LLMs to answer fraud questions, you need to know that the data source is, you know, not, it's got to come from somewhere, and it's usually coming from open sources. So, if that doesn't exist, it's just going to make something up.
Holly Sandberg
14:29
Sure, I mean, you're giving it a task, and it has a very binary way, I think, of looking at that task, and it has a bias towards certainty. So, I don't know as much, and anybody who has ever reported to me will roll their eyes out of their head and tell you they've heard me say a zillion times, I love nothing more. No answer will make me happier than a confident I don't know. Yeah, and you, I won't give you that. And the sources that you're talking about, they're so just prone to subtle manipulation once someone kind of realizes that. And, you know, to me, it's not hard for me to think of ways that I would go ahead and attack that, the speed that everything is moving at. What worries me most, I think, is just laziness, and not, not even laziness, like just folks not knowing what to do. Like, my mind goes back to you saying there are people who are working for companies, and those companies are telling them by X date you need to be doing 25% of your work with AI. Gosh, I really hope that the questions immediately after that you need to be doing 25% needs to be, there's going to be QA on that 25%, and to quantify the impact of that 25%, and you're going to be asked to provide some data around accuracy of that 25%, and you're going to be asked to provide some data around what did you learn and what went wrong. Going back to the point of, you know, I certainly don't consider myself an expert at all, but I've worked on upscaling myself enough that I've seen AI directly lie to me. I've seen results come back that didn't make any sense. And, like, the iteration of those kind of working sessions, and those prompts are actually, that's something that I really like, and I really think a lot of fraud fighters will, if they don't already, will actually really enjoy. Because it's the same, even though we're talking about faster-moving technology and technology with greatly enhanced capability. The old adage is always still always true: garbage in, garbage out. Yeah, no matter how much faster the garbage hopefully comes out or doesn't, in your instance, that still really is like there's a responsibility on you. And I think a lot of companies are going to, you know, are really in peril of screwing it up, like you just kind of turn in your time card, so to speak, and say, here's my, you know, this week 78.8% of what I did was not AI, and, you know, 20-whatever percent was. If that's the last and only question that someone's asking, I just think that's a recipe for disaster. And I realize I'm, you know, in this conversation I'm actually super optimistic about a lot of these things, but that, that I think needs to be said. That's probably an instance of not worrying about humans being lazy and humans not really realizing how to prompt and how to upskill enough, so that you're really getting the best results out of it. But that's probably an organizational example, that if it's just like, well, everybody says they're doing shit with AI, we're good, everything's going to be fine now, like there's probably going to be some pretty unpleasant surprises, I think, happening in organizations that aren't being much more structured about it than that.
Karisse Hendrick
17:18
That's a really good point. Yeah, there should definitely be some QAing, and not just training, because I also know of a merchant who put in a math equation for VAMP and put in their data, you know, into AI, and the math was wrong. As a devout math hater, I remember hearing that, and it really pissed me off. Honestly, I was just... I know, right? Because, like, if it can't do math, like, what, what's it good for? Seriously, like, take over my math for me, and it, like, calculated it wrong, and for them, there wasn't this grave, there wasn't this, you know, outcome of, and then they were on the VAMP program, and they got charged as shit done for, you know, being over the threshold, like that wasn't the case, but still the math was wrong, and the equation was right, the input was...
Holly Sandberg
18:00
Right, but the outcome was wrong, and it could have had some detrimental consequences. Where no, we think we're okay, we tell our bosses, "Oh, yeah, no, we're under the threshold, no need to worry." And then you get a fine for $8 per incident for last month, and that adds up real quick. And it can go both ways. I think it's the input is right and the output is wrong, or it can be the output is actually right, because the input was wrong, and that's where it can get really, really dangerous. And if you were to do some sort of an audit and have someone very technical with, like, a way better understanding of how this all works than me, they would just say, like, nope, it actually, you asked the wrong question, it gave you the right answer to the wrong question. So you asked it to measure month-over-month change in fraud rate, or whatever the case may be, and because you prompted it incorrectly, or you didn't pull in the right data sources, you now think that you've got a 50% problem, or maybe you've got a 5% problem, and the AI actually didn't do anything wrong, you did, right? Yeah, that's a really good point, and that's another, you know, reason for that upskilling, right?
Karisse Hendrick
19:00
Like, for, you know, really understanding it and understanding how to prompt it accurately. I know that's a big part of it, right, is having to use that very analytical brain of, you know, it's almost like if this and this, then that, you know, and knowing how to prompt it correctly to get the right answer. You know, in this case, we're talking about LLMs, and not necessarily vendors that have AI, but, you know, the same thing goes for vendors when they have AI as well. Are they queuing it right? Are they making sure that their tools and their systems that use AI, and that, you know, can actually write rules on behalf of you and everything else, like, is there a feedback loop to ensure that those rules were accurate and they're not misfiring, or that they're not creating false positives?
Holly Sandberg
19:44
Yeah, and I think that, and there's two people who come to mind, you know them both, one especially well, and one who you've met, who both worked for me, who are both brilliant analysts and brilliant strategists. So, not to go too far down, though, like, what can go wrong, sort of pessimism path, right? There are certainly some vendors doing some really cool things in the space, and I've had some conversations with some of them, and it's really just about, you know, in days when rules engines were all that we had, and we had to kind of explain to leadership, like, if you have a robust rules engine, what you can do with it is really pretty infinite as far as what you can create. But it's, yes, but it's what you can create that's the limitation there. So I look at, like, those two ladies who are just absolutely brilliant, and think, like, as good as they are at combining model scoring with supplemental rules, and everything that they know, and their amazing ability to spot trends, and their amazing to sort of, I believe it's a hunch. I could go off on a whole tangent about brains being, you know, not understood in the sense that they are computers, but what they can do with that, because now they don't have to go and do this detailed kind of busy, not busy work. It's valuable work, so calling it busy work is wrong, but sort of the foundational work that they do, that then they say, okay, this is what I've learned, this is what I pulled together, now I go write that rule. They can, so that kind of challenge that always existed with rules, where it was hard to explain and hard to pitch the value right to leadership about, like, what can it do? Well, I don't know, why can't it do? It's as good as whoever's driving the car. You've now taken that person, you've taken them out of a Toyota Camry, and you've put them in a Ferrari. So that's kind of where my mind goes on the optimism side when I think about amplification, and it's much less about the AI replacing that super brilliant strategist. It's about, oh my gosh, like we can tap into that super brilliant strategist in amazing ways that we couldn't before, just because of the limitations of space and time in reality.
Karisse Hendrick
21:42
Yeah, if your compliance team is drowning in manual reviews and outdated tools, it's time for an upgrade. Sardine offers a fully integrated suite of products for KYC, KYB, AML compliance, and risk operations all in one platform. With Sardine, you can perform global watch list screenings, document verification, and customer due diligence in real time. You can also configure flexible onboarding flows, monitor transactions, file suspicious activity reports, and centralize your case management, all from a single dashboard. Their real-time risk scoring also helps you make smarter authorization decisions on the fly. To simplify your operations, reduce false positives, and stay ahead of fraud and compliance issues, visit www.sardine.ai to learn more, or to book a personalized demo. That's www dot s a r d i n e dot a i. So, switching gears a little bit, you know, another, like, a follow-up question to the original question about being worried if AI is going to replace your job, and I think we went off on a little bit of a tangent of like, hey, here are some things that you should use it for, and here are some other things that maybe you should be cautioned by. There are a lot of layoffs, and there are people who are being laid off because companies think that they can, you know, they don't need them, and they can replace them with AI. So if that does happen, the question is, what should I do if that happens?
Holly Sandberg
23:05
Don't vent to ChatGPT about it. This is the time to reach out to other actual humans.
Karisse Hendrick
23:13
Especially don't name your employer while you're venting to ChatGPT.
Holly Sandberg
23:18
That you, that you're, your privacy settings, whether it's Claude or ChatGPT or Gemini, or whatever the case may be. Like, first of all, just don't talk to it about things that you shouldn't talk to it about. Understand that you're absolutely right to point that out. But also, there's been certainly a lot of talk and a lot of news about, you know, the sort of sycophantic nature of AI, and, you know, as a matter for some people, it might actually be tempting, maybe to go and probably type a lot more than what you should about, you know, about getting caught up in a restructuring or in a RIF. So, the best thing that you really can do is talk to other humans. You'll find a lot of support out there right now. Reach out. There are groups, they're not hard to find on LinkedIn, and probably on other big social media platforms, but I'm sure they exist out there. Reach out to your network, however, in whatever style that is, whether it's reaching out broadly, whether it's just having one-on-ones with people that you trust. Like, you do you, and you do what you need to do to take care of yourself. And then do your research. I think there's been some really interesting information recently, in March. Gosh, March seems so far away, and I thought it was still early. I think it was March, there was a really good Forbes article, and that Forbes article actually keyed a lot off of some Forrester research around AI layoffs, and there, because we love these, like, quiet firing, quiet quitting, quiet everything, now not only quiet rehiring, but there's some actually really interesting, really data-backed forecasting out there around the fact that that will happen, because there will be people who just hit the gas pedal, maybe a little bit too hard, and there's news about that happening already. There's, if you don't have to look very far to see that some of the bigger companies that have done layoffs have also reached out to some of those people very quickly and rehired them.
Karisse Hendrick
25:00
Thinking of Twitter, that's the first example I could think of. They're not the only one, so I can say it, right. Yeah, we know they're not the only one, but that was public, so I can use that as an example. Oh, whoops, we fired the guy that oversees this.
Holly Sandberg
25:15
Some other very recognizable names in that group, and, you know, I think you're going to have good days and bad days, probably. If you get impacted by something like this, on your bad days, you can maybe look at it and just be like, and be a little snarky about it. But on your good days, you can just say, if you research any huge leap forward in the history of humanity, not just this one, the story's kind of always been the same, about, you know, the early adopters, then the rush to catch up for the people who aren't early adopters, and then overcorrections, and overpromising, under-delivering, and just a lot of, like, you know, progress is a messy thing, especially progress at hyperspeed. So, if you find yourself in that position, you'll have some time on your hands, and you'll find a lot of people, myself included, I've offered to, like, do mock interviews with people, offered to help people with resumes. I'm happy to do that. I've been a hiring manager for probably longer than a lot of people have been alive, and I never mind doing that for people who reach out to me, and I know I'm not the only one. So just take care of yourself, really, and find the support in whatever way is best for you. But it really isn't the end of the world, and if you do some research, specifically on your instance, you know, you may find maybe it doesn't have as much to do with AI as what you thought, or as what it was presented as on its face, and you've got time to upskill, right? So, there's that too. So that when that correction happens, when there's a lot of rehiring because people over-indexed on cutting headcount, you've got time to position yourself in the best possible way. And that kind of goes back to, you know, the advice that I gave at the very beginning of this. You'll have more time to do all of those things, and be ready to hit the ground running.
Karisse Hendrick
27:00
Yeah, I think I would add that there are things that you can do even before that happens. Like, I think you have nailed it on the head that it's not just fraud professionals that are nervous and scared, it's, you know, every level of employee is nervous and scared about either, you know, risks or, you know, restructuring or AI adoption, and, you know, laying off people because the company thinks the AI is the answer and they're diving in faster than they probably should. There's just a lot of fear out there, and so there are things that you can do before that happens, right? I mean, obviously look at the job market and update your resume, and, you know, all of those things, but also you can start networking, right, reaching out to people in your industry that work at companies that you're interested in, or that, you know, we're pretty small and friendly bunch overall, you know, maybe like a couple of exceptions of some grumpy people. But for the most part, you can also get involved in organizations and attend conferences as much as possible to do the face-to-face networking, which is so important. There's also, you know, you can start to interact on LinkedIn, as you know, I've mentioned to you once or twice, that you can do more often. I mentioned to everyone, so it's not just you, but I do think that you're full of many pearls of wisdom that people would find interesting and impactful. But, you know, doing that, coming on the podcast, like there's lots of things that you can do to kind of up your visibility. But at the same time, you know, during that time, while you're scared, go back to Holly's answer for the first question and make sure that you are not teching out of your current job.
Holly Sandberg
28:15
I went back and I looked at, I have one thing on a Post-it beside me, and I went over it six or seven times with the pen, and that's what I said earlier about the feedback loops that make AI useful are experienced risk teams. And the way that I would apply that to this is, you want to make yourself feel better right now. And what I find super interesting in amongst all this talk about headcount cutting and about AI can solve fraud, there's actually very interesting information out there about the fact that generative AI can solve fraud a whole lot better than machine learning could. And there's some very good research coming out of Princeton and a couple of other areas on that subject, which is good for any one of us who ever had to explain why a machine learning model wasn't reinventing the universe in the way that maybe somebody thought that it would, and that we probably, as fraud fighters, probably knew from the beginning that it wasn't going to. It was an advance, absolutely. But I think that goes to more optimism for the generative AI side of things. It's actually vastly more skilled at it than machine learning is. But you know who's hiring right now for fraud fighters and trust and safety? If you scroll the job listings, Anthropic and OpenAI, and Google. So, going back to the, you know, if there's no need for people who are skilled at trust and safety, or who are skilled at fighting fraud, why is it that the people, the companies, these huge companies that are generating thousands of headlines every single day, they're all hiring for fraud fighters and trust and safety? Which tells you, I think, an awful lot. May not be what you want to do if you're looking for a job, but I think it's very interesting that there's, it's not hard to see right now that they need it. They really need it badly. And if their products and their models and their LLMs had figured it all out, you wouldn't see them hiring in the way that somebody's just going to say that. If they...
Karisse Hendrick
30:00
If they knew how to fix it, they wouldn't be hiring the way that they are. That's a really good point. So, I've been in the same role for several years, and think I'm ready for the next title up. I think this is probably similar to the last question, but a little different, but it doesn't exist in my company. How can I pitch my value to my boss to show them I deserve a raise?
Holly Sandberg
30:17
Well, do I go optimism first or pessimism first, right?
Karisse Hendrick
30:24
In this economy.
Holly Sandberg
30:26
I'll go optimism first. I think quantify, quantify, quantify. Don't just say I'm great, just say I'm 37% great, and I increased efficiency by X, and I reduced cost overhead by 10X and reduced, you know, fraud rate or charge, increased charge, whatever that is, wherever you can quantify it. It's going to be a better conversation. But I think what I see a lot of people do that I think needs to be part of the conversation, but maybe not the beginning of the conversation, and, you know, at the beginning of a conversation is the hook, right? If you're a snooze fest for the first five minutes of a conversation, and you say really great, amazing things in minutes six through 10, you might have already lost your audience. So, I think you start the conversation not with all the great things that you've done, you start the conversation about what you are going to do, and that's uncharted territory. It's not nearly as probably familiar for some people, especially people who have any kind of imposter syndrome. Like, and I'm contradicting myself a little bit, like, on one hand, saying quantify, quantify, quantify, but I'm also saying they know you're great at what you do, and that's why they have you doing it. So, what you really need to convince them of is that you'd be great at something that they don't even know that they need yet, and that's a different conversation. And it probably doesn't start with, it starts with maybe sort of alluding to the great things that you've done, but anchoring too much on that, I don't think is necessarily, you know, the strategy that I would go with. Leaders hear that, not to discount that in any way, but when they do, you know, mid-year reviews or annual reviews, or, you know, go through SMART goal setting, if that's something that you do with your team, you want to approach it literally like a net new conversation, because it is. You're not talking about let's evaluate what I'm doing currently, you're talking about why am I ready for the next thing, and that can catapult off of what you've done, but it shouldn't be the entirety of the conversation.
Karisse Hendrick
32:12
Right. Right. I think that's a really good point, and not one that I would have thought of. But, you know, here's what I can take on extra, here's what I can do, and by the way, I think that the more appropriate title for that would be X, right? You know, or hey, I have been asked to take on XYZ. I think the more appropriate title for that is this, and here's what I'm going to do with these extra responsibilities. I like that. Yeah, if you've got examples, whether it's at the company they're at currently, or even in a previous role, I mean, it probably has a little bit more oomph, maybe, if it's been at the company that you're at. But if you've previously taken on things before, if you've had your remit expanded, if there was a thing that needed to get done, and everybody was kind of looking around, going, I don't know who's going to do that, and you stepped up and said me, and then you did amazing things with it, I mean, that's...
Holly Sandberg
33:00
The story there, I think, is about the amazing thing you did, whatever that may be, but it's also about the fact that, like, you were given some room, right? Some breathing room to run with something, and you proved that you do that. You don't just need someone to hand you a playbook. Yeah, that you're able to take what you've done, that you've knocked out of the park, and apply the decisiveness, and apply the judgment, and apply the way that you dig into things, and the way that you research things, and the way that you show up prepared, and the way that you ask. You don't have to know everything, you know it's okay, going back to me saying I love a confident I don't know, or acknowledging that you're asking them to invest in you, saying here's other times that you've invested in me, and it's paid off. I'm asking incrementally a bigger investment, but you've got some historical precedent that says that this is a good bet for you to make on me. I like that, that's really good. So, what was your pessimistic approach or pessimistic response? I should... I, yeah, I really am. I think people who are very sarcastic, which maybe I am a little bit, get sometimes.
Karisse Hendrick
34:01
Yeah, no, not sarcastic. Oh, wait, I'm being sarcastic about you not being sarcastic.
Holly Sandberg
34:05
Get a bad rap for being pessimistic. I really am an eternal optimist. So I think my, but where my mind goes that would be somewhat pessimistic, that I would try to turn into optimism, is the reality is you may do everything right, you may follow all the advice that I just kind of laid out, and put something together that's absolutely amazing, and for reasons that have nothing to do with you, they could be budget, they could be internal pressure, they could just be your boss isn't smart enough to realize how amazing you are. It could be any number of things. The answer might be that you have to grow elsewhere. Yeah, that's okay. And especially if you can say to yourself that you did all that you could, then that, I think, is especially okay. So I wouldn't just look at it as a win-or-lose sort of a scenario. You go, you try to pitch this, you get shot down, and you just say, well, I'm just going to go back to my desk and keep doing what I do. Then maybe that's time to look at doing something different, and look at doing it elsewhere, and that's okay. Not to do in any sort of rash, reactive way, but it's certainly an okay thing to think about. It happens all the time, and I think people, particularly people who are loyal, which a lot of people in our industry really are because they have a really strong moral compass, sometimes take longer than others to realize that that maybe is the best pick for them.
Karisse Hendrick
35:22
You're so right that I have to remind several fraud fighters on a regular basis that their company is not going to be as loyal to them as they're being to their company, and that sucks. But at the same time, I think it's a good reminder that you do have to look out for you. So if you have done those things and you've pitched this new title, or this new, you know, job duties, or these new job duties, and your boss has not, you know, glinked on, or maybe they've said, "Sure, I'd love for you to do that, but we have no room in our budget to pay you more." That seems to happen to me a few times, especially around 2008, 2010, you know, it might be time to grow elsewhere, like you said. Speaking of you being sarcastic or cynical, you know, I will. One of the things that you said to me once, and I will never forget it, and you probably totally forgot about it, was once you said, "I'm not cynical, I'm just skeptical." And that, I think, sums up the way that fraud fighters think in general, right? Like, it can come across as cynical and pessimistic, but we're skeptical because we have to be, because there's, you know, always fraud right around the corner, or a scam, or something like that. Like, we have to be skeptical about the truth, but I do think a lot of us are eternal optimists, you know. We wouldn't be continuing to show up for this battle that is grossly outnumbered every single day if we weren't.
Holly Sandberg
36:42
Yeah, I agree. I mean, that goes back to why I think the human support with everything that's going on right now is a good thing. And there is, I think that's, you can't be cynical if you're cynical. Show me somebody who's cynical, particularly on the fraud side of things, but maybe I could make some connections even to the payment side of things, but definitely on the fraud side of things, I would bet you that they probably have a higher false positive rate than some other companies. So that's really the key difference, I think, between cynicism and skepticism. If you're cynical, you're going to start, you'll find, there's another kind of old adage that I really love, which is like, you will find what you're looking for. If you're looking for negativity, you'll find negativity. If you approach fraud by saying that the very best way to find bad actors isn't to know what the bad actors are doing, it's to know every possible thing you can know about your best, most trusted, most loyal customers. That's how you spot anomalies way, way better. If you approach it only through that cynical lens, you're going to start seeing ghosts where they don't really exist, and I think most people who have been around in the industry have probably figured that out by now.
Karisse Hendrick
37:45
Hopefully, especially those that are in leadership positions, but yeah, I think it's similar to AI hallucination, right? Like, you'll find what you're looking for if you're looking for just fraud, right, but that, yeah, to your point, you'll probably, if you're only looking for fraud, you're going to see it in places where it doesn't exist, and that's going to create false positives. Well, I'm not surprised that we blazed through the time that we had today. We had one other question, but we can answer it another time, maybe in five years, or maybe two.
Holly Sandberg
38:17
I mean, you know, maybe, maybe we don't go five years on.
Karisse Hendrick
38:20
Well, you know that the door is always open anytime you want to come on. I am here for it, and I think my audience will be too, and understand why we typically, when we talk on the phone, it's a good two- to three-hour conversation. I don't even remember what our record was, but I know we've been up in the fours before. I think our spouses both appreciate it, though, because we're not talking to them about fraud as much as we could, so there is that. But no, I appreciate this, and I, you know, I hadn't thought of a few things that you said, so I know that other people will think that too. I will make a point to add your LinkedIn, a link to your LinkedIn profile in the show notes for anyone to connect with you if they aren't already. And thank you for coming on the podcast. I really appreciate it. It was fun.
View full transcript
