What is up fraud fighters and welcome back to Fraud Forward.
I have been looking forward to today's conversation for a while.
So I originally discovered Assaf Kipnis through
an episode of, of course, the OG Fraudology.
Um and I've followed his work ever since.
And recently he shared something about realizing he was more
than just an investigator, and it resonated with me.
You know, I spent most of my career in
fraud investigations and operations.
And for a long time, that's how I saw myself, too.
It took a mentor pointing out that my practical knowledge of
banking, payments, fraud, and the processes behind it all gave
me a much broader understanding than even I realized.
And that's actually what prompted me to reach out to Assaf today.
So, then we had a conversation even before recording and he
challenged something that I hadn't really considered.
While we're debating whether AI will replace fraud investigators,
the people committing fraud are using AI to experiment,
innovate, and improve their operations.
So, what happens if we automate away the people
responsible for helping us stay ahead?
Today, we're talking about human innovation, building AI that
investigators can actually trust, and why the future of fraud
fighting may depend on how well we combine human expertise with technology.
So, Assaf, welcome to Fraud Forward. I'm so glad you're here.
Thanks, Hailey. I'm I'm really happy to be here.
I've been following you for a really long time
now, so I'm super excited about this.
The the inner monologue of me just went, Ah!.
Um, for for those of you that or for those in the audience
who may not be familiar with you, I'd love it if you know
before we dive into the conversation, if you could introduce
yourself, give them a little bit about your background
and, you know, why it is you do what you do.
Sure. So, my name is Assaf Kipnis.
Uh, I've been in the security, integrity, trust, and safety
and fraud space for about 13 years right now.
I spent my career uh doing mostly social media investigations
at the highest tiers of the investigation teams.
Started at LinkedIn for several years.
Most of the time I was at uh Facebook.
Uh both as an investigator on the e-crime
team and a manager on the e-crime team.
Doing scaled mitigation, tracking threat actors,
ident basically investigating things that other
teams just couldn't figure out at that point.
Then I spent some time at Google and ElevenLabs and now I am running
my own consultancy doing both investigations, AI first
investigations, because I really got into that in the last couple of years.
And AI builds and threat intelligence and things like that.
Just things that make me happy.
Uh well it makes me happy because now we get to
benefit from your expertise and knowledge.
And again just so grateful that you're here with us today.
So when we first talked, you know, during our earlier
conversations, you challenged the perception of fraudsters
as individuals that are working alone.
You described them as businesses with goals,
processes, and even innovation cycles.
What do you think are we misunderstanding about
how modern fraud operations actually work?
So I don't want to say that everybody misunderstands
that, but I struggle with working only in the largest
companies that have the largest scales.
And for that reason also have the most
complex attackers attacking them.
But from my experience, especially after 2020, fraud operations,
scam operations, they're no longer this person sitting in a in a
garage or someone in West Africa doing it by themselves, which
in South Africa, I don't even know if it was ever that case.
Fraud operations are running like companies.
Whether it's the Yahoo boys that are when we look
at them, we're like, "This is not a company.
" But when you look deeper into them,
they have a a corporate structure. They have goals.
They have people that do grunt work.
They have people who who are leaders, people who do the
communication, the the the recruiting uh and infrastructure.
So, a lot of times when I talk to to to people in this space,
I feel like we're still I'm still hearing people saying,
"Oh, I'm going to take this account down or I'm going
to block this uh device and we're good to go.
" And I'm seeing that I'm still seeing that at scale
when when uh companies even the biggest companies
celebrate taking down a thousand accounts.
And that's um from at least from my experience, that's not
something that harms uh a scaled operation at all.
Yeah, I I can see exactly what you mean and and for for those
in like the community banks and credit unions, we struggle
and we are just trying to take down usually the one.
But you're right, they are operating at these massive scales
and so if we stop one, what is it a drop in the bucket?
And also they don't have the same regulatory hurdles
that we have and rules in place that prevent us
from innovating at scale like like they have.
You know what advantages do you think they have over what we do?
Well, they don't have to deal with growth.
And across everything I've done, uh growth
growth has always been the the hindrance.
And I'm not saying that from an adversarial
perspective, growth has to happen.
But it was always how do we minimize false positives?
How do, they don't have users.
We have people that we need to serve and we need to work around that.
And the more the attackers try to look like
real users, then the harder it is for us.
Um the other thing from from an attacker and a defender perspective,
an attacker just needs to find one entry way.
A defender needs to defend all possible entryways and
think about what other entryways might there be.
So as far as tactics and organizations, they have less
limitations and they also have the opportunity to
to continuously innovate and and and check.
The other thing that comes to mind is fraud
prevention is never the business.
So it's always a part of what we do and
for them fraud is the business.
So they have a lot more opportunity to direct
resources towards that than we do.
You know, the way that my mind works, I always
try to um bring in practical things.
And what you just reminded me of of like, you know, the defenders
have to defend every open vulnerability and whatnot.
I just thought back whenever my husband and I found out we were
having a girl and his mom came up to him and said, you know, if
you were having a boy, you'd have to worry about one boy.
Now you have to worry about every boy
in the world with your one child. And so I I feel the same way.
It's like God with us as as fraud people is especially if we
don't have the the holistic understanding of the payments and
processes and infrastructure of your own organization.
It's it's almost impossible to try to protect every vulnerability
you may have because you don't necessarily understand it.
Making sure we've got, you know, the right people in place, but
it's very easy for that little snake to kind of slither his way in.
And and you can never know the full intricacies
and full structure of your own organization.
Um I always said at, when I was at Facebook, like
the attackers know our systems way better than
we do, and we're also segregated into pieces.
My team deals with this and this team deals with that.
Um they they need to get to to to the point of exploit.
While we have people that are innovating on our product and changing
the product without telling us because that's how products work.
And we are only defending a certain pieces
and everything has its own compartment.
Um so they have a lot more freedom to to play around.
And you know, one thing that we wanted to make sure we
covered in this this first uh portion of our of our conversation
is, you know, we hear a lot about using AI to reduce
that headcount and automate investigations.
You've argued that this approach could actually
create a competitive disadvantage.
I'd I'd love to dive into that for a little bit.
Yeah, I've been playing around building
things with with AI for a while.
And it's hard sometimes because I'm in my own AI bubble and I think,
oh, everybody kind of understands it in the way the same way I do.
And then I talk to people and it's like, okay, maybe not. Um, and
And there's a lot of So, yes, I think yesterday I someone
said something in a fraud group that I'm on that
was very interesting to me and I agree with.
It said um the narrative out there is that you just put
something into a black box and it produces a tool.
And that's just not the case.
Building AI products is a software development product.
It's a it's a software development process.
Especially in our space because there's so many more
decisions that could happen within the process.
That we have to have someone that knows what they're doing.
So we cannot expect while we are told that the AI
is this all knowing uh entity that knows everything
about everything and can just produce stuff.
It doesn't know the the the details of your organization.
It doesn't know the details of everything that you do.
And if you take the person who has the expertise in the
space, and it doesn't have to be the expertise in your
company, it's the expertise in how do we do things?
How do we fight fraud? How do we how do you develop software?
You're going to lose the edge and you're going to end up with AI
processes that are making decisions that you are not ready for.
And the next piece of that that what we're talking
about is, AI cannot innovate on its own.
AI cannot think about oh what was the conversation I had 3 days
ago with someone, and they found something really interesting.
Let me see if I can research that more.
That thing might be known in the in the internet somewhere,
but your agent it, this task has no context of that.
The other thing that I'm constantly noticing about
agents is that we think they think like us. They don't think like us.
They don't say, "Hold on, I did this and then I did that and
that broke, and then I learned something over there.
And the next time that happens, I'm
going to say, Oh, I remember that. " They don't remember anything.
Um even if you are building like you know those brains that
learn and knowledge bases they're still not cognitive to
the point that they can say ah hold on let me innovate
on this piece, or I remember talking to that person, or
making this mistake yesterday, or 10 minutes ago.
Um so when you're doing that and you're taking the person out of the
equation, you're just going to have motion without progress.
You're gonna continue building stuff, but they're not going
to be anything new and exciting or interesting.
Uh, and on the other side of that, the threat actors
are not thinking, how can I reduce headcounts?
They're thinking, how can I increase what I'm getting?
So they're sitting there and using people to build
with AI, and innovate, and find new ways.
Because some they see they see us protecting
and they're like okay that didn't work. What else should we try?
And also the other thing they need to innovate harder with AI.
Because uh current LLMs are built to not
be conducive to malicious activity.
So they need to consistently go around
the AI to figure out how to do that.
So while they doing that and we're saying we're going
to just have less people and have everything completely
automated, we're losing the race.
I can see that. I also think that I'm being gaslit by my LLM.
Because it makes it makes me think
that it that it thinks like I do.
Uh you know, for instance, last night I said,
"I would like to make some brownies. " I'm going to read this.
I said, "I would like to make some brownies,
but I don't have any cocoa powder.
" I do however have white chocolate hot cocoa mix.
Could I use that with some other household
ingredients to make brownies? And it said yes. Laughing emoji.
We are absolutely baking brownies with what you've got.
Now, they ended up being delicious.
But the way that this responded back to me made
me think that it's thinking just like I am.
The way it responds to me is responding the
way that I would respond to people.
I mean, it's laughing at me with a crying emoji.
Like, what is going on here?
So, I think I'm being gaslit by my LLM to to make me think that
it's thinking like I am, but it's not as innovative and creative
as we as humans are and have proven to be in the past.
And also, it doesn't have the like it might have
the context in the in in the training that someone
made brownies like that and like, okay.
But most of the time, it's it's looking to to appease
you to say, yeah, we can make this work.
Yeah, we can make it work.
And it's built to to be that.
It's built to be for some reason, it's built
to be a mass product for everybody.
But what I think is missing here is that there's a very big gap
between the consumer product and the enterprise product.
And we're Yes. As a consumer, you can ask it stuff.
It will talk to you. I've done that.
And you can brainstorm and you can get all
kinds of fun ideas and have it research.
But once you needed to make actual decisions, it's not
just that you needed to make actual decisions.
You needed to make gated decisions.
You needed to make educated and gated decisions.
And when you get there to the point where you can't
have the decision be made with the wrong context.
Or the decision made with context that's irrelevant,
because that's uh that's what would happen.
It would just, oh, I don't have context for this.
Let me take something and then I don't have control of that context.
So at that point you're starting a cascading issue of possible
problems that you're that you're not seeing.
Because it's making these decisions and then it keeps
going down the decision tree, and then you come back
and you're like uh where did this go wrong? Where did it break?
And it's very hard to trace.
I had um a a bank executive tell me once that um the decisions
are harder with these LLMs because they're too agreeable.
They they want it to work.
So, whenever you want it to make a decision, it's going to try to
be agreeable to allow a transaction to go through or whatever.
And I love that you started bringing up context.
Context is huge whenever we're thinking about this.
Fraud fighters, if you're anything like me,
your list of things to read keeps growing.
Your time to read them though probably not so much.
That's why I write The Monday Fraud Fix.
It's the newsletter companion to Fraud Forward.
So every Monday I'm sharing fraud trends that made me pause,
what I'm reading, conversations happening in our community,
and the fraud fighters that I think you should know.
I also bring my take, you know, what it means for the people
doing this work, and the questions we need to be asking.
Every now and again, I might leave you a
little Scooby snack to start your week.
You know, I just can't help myself.
You can find The Monday Fraud Fix on Substack or tap
the link in the episode show notes and subscribe.
I'll see you in your inbox, fraud fighters.
We'll shift and talk now, You know, that AI is only as
reliable as the context and and controls around it.
That's what you've mentioned before.
And you've spent, like you mentioned, a lot of last
year building investigative tools with AI.
What have you been building and and what
problems were you trying to solve?
So, one of the main things that I wanted to do
is I really enjoy open source investigations.
And after being in very large companies and coming out and
only having to use open source, it was very daunting.
Cuz I'm like, I had all the information that I needed to do
investigations, how do I do investigations now with none of that?
And a lot of open- source investigations or open
source investigations general is tedious.
It's opening links, and Google dorking, and finding the
affiliate link, and clicking it, and where does this
go, and then finding I don't know records of stuff.
So I wanted to see how I can automate these in automate
investigations that I can run OSINT uh open source investigations faster.
And also actually run them.
Because if I run if I want to run something just for fun about cuz
that's my fun I find affiliate marketing and stuff like that.
And I I I don't know why it's I enjoy it.
But it's not fun if I'm doing it for three days straight and I'm
hitting my my head and like having spreadsheets that's a pain,
Um so how do I do that that it it works quickly
and I can do multiple at the same time?
So my favorite tool that I built is open source investigation
tool that is connected to multiple open- source intelligence
um MCPs and and resources, and AP and a bunch of APIs into
virus total, and spamhouse and things like that.
And also I entered I kind of brought into it a lot of the
investigative methodology that I know, and investigative
methodology that I've learned from other people.
Like doing a premortem, second-guessing decisions.
And essentially what it does now I say okay open a new case.
Uh it opens all of the context folders that need to go into the
evidence folders, because everything has to have evidence for me.
And then it's I say okay we need to go, here are uh
three links, This is what I'm concerned about, Go.
So, what it does essentially is starts understanding
what I wanted to do for the specific investigation.
Then starts fanning out with swarms of agents to find all the links,
all the affiliates, everywhere that's it's been talked about and
starts aggregating the information and actually saving it to disk.
So everything I have, nothing sits in the context
for the LLM, because I don't trust that at all.
Everything sits on folders on my disk. And then it reasons from that.
Um, but I've spoken to people, oh, but what do you how do
you know that it just went to the direction you want?
So the whole point of it is, I'm in the loop.
It comes to me and says, I found this. What do you want now?
And then I can doubt what it's saying.
I can have it research what it said to me and say, "Show me
the evidence" and then pull it in a different direction.
I'm doing that right now.
This is a big one that I'm actually been working on for 3 days.
And I constantly need to redirect it,
because it's like an investigator.
It goes down rabbit holes and sometimes
we go down rabbit holes together.
But what I really like about it is that at the end it
produces a full document for me uh with evidence.
Meaning screenshots, where everything came from, all the documents.
And that's the most important part to me.
So if if some if I look at the document and say, hold on.
It can tell me exactly with a screenshot where it came
from, and the link, everything, and how it goes
all the way back, because everything is saved.
It doesn't need to go back to its own context
and say, where did I go that? It where did I find that?
It goes directly into the MD files that I
have, and can trace itself backwards. So that's my favorite tool.
It honestly sounds really cool.
Um, and I love though also you talk about the rabbit holes, right?
And one thing that that I know that I've experienced in the past,
and others have too, is the the hallucinations, right?
That that come from uh using these AI tools.
And so during our conversation, you argued that hallucinations
aren't necessarily the the biggest risk.
That that the bigger risk is that AI is making decisions
based on incomplete or incorrect context.
Can you explain that difference there?
Yeah, I don't want to have a sound bite that people grab that
says Assaf said no shouldn't care about hallucinations.
Hallucinations are a thing, but I honestly I've mostly experienced
hallucinations when I'm working with uh something outside my system.
An example would be I put something on on GitHub
and it's been gaining a lot of stars. Another thing that I built.
And I said to, usually I don't play with ChatGPT.
ChatGPT is only like a chat that I talk to.
So, it doesn't have any other context.
And I said, "Can you find who talked about this online?
" And it gave me a LinkedIn account and
said, "This person talked about it. " And I went to the account.
This person doesn't post anything. Like, what what's happening here?
Oh, I saw this person's name somewhere and then I just added it here.
So, essentially, hallucinations are the AI making decisions.
And at this point, we can call it a hallucination, but basically
what it did, it researched, saw a name, and made the decision
to grab that name and say that's that's your resource.
So what I want to say is that the problem
itself is not the hallucination.
It's the decision without behind the hallucination.
And what I see in my work is that the LLM will make
a decision based on incomplete data or even in my system
sometimes it will just grab the wrong data.
Um I have I constantly iterate, and constantly change things,
and I ask a question, and I'm like that has nothing to
do with what, like it has but this is super old data.
Oh, when I made the decision to go find it, the first thing
that came up was this document that still lives here.
So what happens is basically if we compare it to a human, is you go
to an analyst, and say go give me an answer about this problem.
And they went and read a 10year-old uh report
about how this problem was 10 years ago.
And then they made their entire decision based on that.
So essentially it's not,I I really hate when
people say that the LLM is lying to them. It's not lying to you.
It's just it just found the wrong data to make this
decision about, and it really wants to make a decision
and tell you that you find something.
And then it just cascades down from that wrong decision.
It's a really long- winded way to explain like why the judgment
of the LLM is the part that's that I am worried about.
That it makes a judgment in a decision with
incomplete, misplaced, or wrong context.
Which is exactly what happens to people.
I I appreciate that knowing that we have to provide it that fresh,
specific context in order for it to make these good decisions.
Um I I guess it's like you can't tell a fraud
investigator to go and investigate something if
they don't know what type of fraud it is.
We have to explain the fraud.
They have to understand it and the investigation
And I think that's the gap.
The gap is that the kind of chatter around
is the LLM knows everything. How could it make the wrong decision?
It knows everything. It knows everything better than me.
And I've long time ago stopped relying on the agents training.
I think relying on the training of the LLM is where we go wrong
because we're like, "Oh, it should know everything.
" This it's it's I think it's because we've been trained on Google.
Like you ask Google a question and it will it doesn't give
you sometimes they'll give you an answer but usually they'll
give you like this is where here are all the links.
You make the you make the determination.
And we kind of think it's like Google, but it's not.
Um whenever I need to make a decision about research for
example I'm looking at some videos and I needed to understand
how huristics of AI slop, I hate that word, but
AI slop videos what the huristics look like.
So I didn't ask the LLM what the huristic look like. I said start.
I have a research uh skill, open the research skill,
go find everything that known people in the field have
written about this, and synthesize it for me.
Then let's take that context and make the decision
about how to find these huristics.
So essentially we need to tell the LLM, hey go learn
about this now and it's not to have context.
It's the LLM doesn't constantly find context
about everything in its training.
It's there, but um you need to if we if you need
something specific, you need to go and have it researched
exactly like you would tell a person.
I think that's one thing that we miss um is that
we do have to give it additional context.
And we do find that out once the you know the outcome comes and
we're like uh I don't think that was what I was actually looking for.
Here's this you know we got to start with
that and don't just blindly trust.
The other thing that you know everybody seems to
be talking about, and you mentioned it a second
ago, is you know the human in the loop of AI.
And I've also been exploring the concept of human on the loop.
Do you think the industry actually understands
what meaningful human oversight looks like?
No because it's well don't want to be that guy.
I don't know everybody in the industry, but a lot of times I see
human in the loop and I see it as a talking point as a and a catchphrase.
Human in the loop is dead.
As far as a thing, it's become a buzz word.
Uh we just need a human in the loop.
And to me, it's the same buzzword as saying
AI slop, because it means nothing. What do you mean? What is AI slop?
What is human in the loop? What loop?
How are you putting this human in the loop?
So I had a conversation recently with someone and their
perspective of human in the loop is every single decision
that the AI makes, must have a person review it. Which is bananas.
You can, you'll just work very very slow and
it's not, it's it's completely ineffective.
So, usually what I see is when it's with people who are less familiar
with with automation, because you can't have a person look
at something that's automated and constantly review.
I think there's of course there's tears.
Because there's regulation, there's sensitivity,
You have to identify when you want a human decision to be there.
But I think the concept that is becoming a human
in the loop is a human verifying the AI's work.
I think that's fundamentally flawed from multiple perspectives.
And on the, so it's flawed from one perspective
that you just can't do that.
And if you are thinking about I am going to, I don't
trust the AI at all, and I'm going to have it make, I'm
every single output I'm going to have to check.
That's just not happening.
But from a different perspective, at least the way I look
a human in the loop is in the development side.
I want to be aware if something breaks.
I want to be aware if you made a decision that
I already said is the wrong decision.
Or a decision that is gated, or you ran down
a rabbit hole and you just got stuck there.
I want the AI to be able to come back and tell me, hey, I did this.
So, it's it's different because I think there's like
a software development human in the loop, and there's
like a fraud and safety human in the loop.
And I think the way fraud and safety and security think
about it is like how do I reduce false positives?
How do I make sure that I have a human review
in the end and things like that? Those are different.
But fundamentally from a process development perspective,
you can have a human in the loop, but it's a lot
more intertwined than people make it look.
It's like, oh, you just come in the end to say yes, no.
It's a lot more, a human in the loop
is how the AI makes the decisions.
Or more how the AI is allowed to make the decisions.
I think that's a very good point. I do have a followup on that.
You know, AI is essentially going to produce more work than
than we humans can realistically keep up with or review.
How how do we design oversight that's both effective
and and scalable for for these organizations?
The concept that I've been struggling with some of my clients
on, were was oh we just want to make just make it with AI.
Just have the info like the spreadsheets or the logs go into
AI it will do its thing, and it will all come back.
So that's where things break immediately.
When you think that the AI or the agents are going to do all
the work end to end, and all you got to do is prompt it.
Because what happens again is you didn't give it guidelines,
you didn't give it guardrails, you didn't give it context.
And I think what is happening a lot, and this is a big
friction point I see between people who are more technical
like software engineers and people who are not.
And where people who are not software engineers they'll say well
we're just going to put it all together it's all going to work.
And software engineers like ah no everything needs to be software.
There's a middle a middle ground.
So when I build things the even when I'm talking about
my investigation tool the investigation tool uses
The LLM uses the AI in very specific places.
It's not using it everywhere. Um, most of the work is code.
And I haven't written code in many years and I don't plan on it.
I leverage the AI to write the code for me.
But when I build something, most of the output,
most of the build is deterministic.
So an example would be I need you to grab five pages of logs from
somewhere, parse through them and then find patterns and huristics.
I know that I can do all of that with code.
I have done that with code in several companies.
And I can tell Claude, okay, pull the logs.
I will never have Claude read through logs because
that's where it gets confused and starts making
stuff up because it just wants to be good. Parse the logs.
Don't read them. Make a script, parse everything, download it.
Uh use uh a Python library to find the patterns.
And at the end, when you find the patterns, call the LLM to reason
on these patterns against the context of the investigation.
And even then, the LLM is gated and for specific
decisions it just cannot make.
But the the whole point of what I'm trying to say, this is
kind of the definition I'm trying to make between consumer
and builder or enterprise is that as a consumer, you can
just play around with it and build stuff and it's fun.
But as an enterprise, code exists. It's there.
By the way, it's free, and AI will write it for you.
So one, it's free and two, it's deterministic.
And a lot of our work needs to be deterministic.
And we cannot have a probabilistic machine make decisions for us.
It can make assertions and determinations on patterns,
and it can find new things and incorporate new data.
But if you need an answer that's deterministic, a yes or a
no or a number, which is most of the work, that's code.
So I'm not saying you need to write code.
Tell the LLM to write the code for you.
I would have to get the LLM to write code for me.
I'm not writing code. Also, what what a waste of time.
You know, even if I was really good at it,
it would still take me 10 times the time.
Just Okay, just write a script to do the thing and let's go.
No, the last time I did anything with
code was uh 2009 and on MySpace. My my page was really cool.
All the moving things. I remember that.
All the moving things, all the glitter in the background, my songs.
Yeah, I remember that. I had that. Yeah.
You started and there's a song.
You click on your page and that would determine how cool you were.
And then your top eight would change and it's like gh you didn't
want to remove that one person because then they'd know.
I think LinkedIn should do top eight.
Do you know how much drama that would cause?
It would be so dramatic. It would be great.
So dramatic, but it'd be worth it. I think
Okay, so let's talk a little bit more about that investigator role.
That was one of the things that I really
wanted to talk to you about.
Um, you know, you one of the reasons I originally reached
out again was just because your post about realizing
that you're more than just an investigator.
And I had that moment in my career where I knew
that I was more than just an analyst.
Um, what led you to that realization?
So, I remember working at Facebook, which
was definitely the job I liked the most.
And where, that was the place where I always worried
about this is going to end one day, which it did.
Um, and I remember thinking like, okay, if
I'm not here anymore, where do I go?
Like, I know how to do social media investigations.
That's what I want to know how to do.
And I know people that worked with me that went to banking
and went to intel and like intelligence and stuff like
that, but it was always like this is what I do.
And uh went to Google and kind of like was the same.
Uh and then ElevenLabs was the same.
But and this is one of the best things I got from ElevenLabs
because ElevenLabs is a real AI first company.
Everything happens with AI.
And that I kind of took that on when I started working on my own.
The other piece is that as as an investigator
who doesn't code, I always had a ceiling.
And I'm sure you're familiar with the ceiling.
Why don't we query this and put this on it and
this on it and put all these things together?
And you're like, okay, I don't know how to do this.
Maybe I'll do it by hand.
And then do you go to an engineer who's like okay
you don't know what you're talking about.
Or even if they're cool it's like okay I need to
build this for you this going to take a while.
I need to understand what you're doing.
So working with AI, it's all of a sudden the ceiling is gone.
And I I think that's the experience of a lot of people.
That suddenly this this, you're not technical.
And towards the end of working corporate I got a lot of that.
And I really begrudge it, still do, of you're not technical
so you don't know what you're talking about.
And that is where the investigator kind of sits of like I know how to
investigate, but I have this ceiling that I just cannot get through.
And honestly when I was at Facebook I really tried tried
to learn Python and play around with Python notebook.
And it was always just not my thing and I
always say like work to your strength. And that wasn't my strength.
And as I started working with AI all of a sudden
like oh this ceiling is going away.
I'm not saying I'm I'm a very good author of code.
And I'm sure a lot of my code an engineer
will look at it and say oh my god. Which is fine.
But I've noticed a couple of things one
is now I can now I can be a builder.
Like uh at Facebook we had analysts, an analyst builder.
And I remember when I was a manager, I
only wanted to hire analyst builders.
I didn't I didn't want to hire people who didn't know Python.
And it hurt because I'm like that's I'm hiring people who are not me.
I wouldn't, I said that, I wouldn't hire myself.
But now I can be an analyst builder.
But then I realized that having the ability or kind of the over
time the, I guess ability is the right word, to have an adversarial
mindset is extremely powerful when you're a builder.
Because a lot of so essentially builders who
are very good, they just want to build.
Let me build it, and I don't care, let
me just build, just make it go.
And even vi vibe coders are seeing it right
now and they're like things break.
And as a person that's been doing the adversarial living
in the adversarial mindset all the time, all of
a sudden I can apply that to building things.
To say, okay, where is the LLM going to mess with me?
Where's the where do I need to say mm mm.
This is left or right, and I have to enforce this.
And the way I've been looking at that is really perceiving
the LLM itself as I'm building as a threat actor.
Not a malicious threat actor, but a threat actor.
It's going to do whatever it wants to do at some points and I have
to make sure that I know those points are coming and I know how to
produce something that will stop it from producing bad data.
And that's where it hit me that I'm like, Oh, I'm a builder
now and I can build in a very very specific way because
of my experience as an analyst and an investigator.
I can build as an investigator.
Um, someone recently said, um, the next wave of
builders, I forget who he said they were. Oh, it's these people.
I said, no, it's investigators.
Investigators are the next the next wave of builders
because they see the problems before they manifest.
More importantly, they ask the questions.
Investigators are people who ask questions.
And the more questions you ask when
you build with an LLM, the better.
And the questions would be stuff like you built
I built something and it went awry.
Instead of saying build it better, I say stop.
We're doing a root cause analysis, and I built a skill for that. Why?
What? I wanted you to do this, and you did this.
Um, and we stop and we ask why and we
learn from it every single time.
Actually now it just does it on its own.
Um, and I think that's the the evolution at least that I had as an
investigator that I could say I can build securely now and robustly.
And not just for investigations, but for anything. I can build a CRM.
I don't know um if they do that, too.
But uh just building with data integrity
in mind always which is not so common.
Um, I also have I have two follow-up questions on on the building
of this and like thinking with the investigator in mind.
Um, you know, experienced investigators often recognize patterns
or inconsistencies that they they can't immediately explain.
I can remember being an investigator and pulling the NACHA
file and just something felt off when I looked at it.
I had no other explanation other than it just felt wrong.
How do we capture that knowledge and make it
useful beyond the individual investigator?
I think that the best thing that AI development has been
has given us, especially as investigators, but not only,
is the ability to aggregate knowledge and query it.
To build knowledge bases.
So I can figure something out and it gets recorded in the
knowledge base and then when someone else experiences a
similar thing, it can go back to the knowledge base.
You can go back and say, "Hey, I see this.
Has anybody else in my team experienced this, and how?
" So you can go and it will find it in the knowledge
base and it will say, "Oh yeah, Pat experienced this,
in this investigation, and this is what they saw.
" That's extremely powerful and it also grows
exponentially to a lot of things.
I use knowledge base all the time because I don't remember anything.
But to have the ability to have a team, a company, an organization
knowledge base, that's the real unlock that someone leaves
the company, and you're like and you're like shit, Okay.
Uh he's the only, she's the only one,that did this investigation,
and the only one that knows anything about this.
And now we're boned because I need to figure this out.
Or in 2020, I worked on some election stuff and
a friend of mine investigated specific things.
Thankfully, he was still on the team. He's like, "Copy paste.
Here's all my documents. Go figure this out. " And I did.
Instead, I can just go, "What did Ryan find out here?
Give me give me the patterns. Give me the highlights. Teach it to me.
And now, I don't even, that I can just say here's
a thing that I found compare it against the knowledge
base and figure out what what's here.
And that's to me from our organization is the biggest unlock.
And I hope people don't hear it as like oh
so you just don't need people anymore.
No, you still need people um because someone needs
to go through these things and make mistakes.
Someone needs to make mistakes and realize their mistakes.
Some AI makes mistakes and does not realize
their mistakes, a lot of the times.
So someone needs to have that and then the knowledge base
needs, that's how the knowledge base grows and expands.
And that's where you don't get these knowledge gaps or you don't
get, my favorite thing that people said to me when I would
ask a question they would say did you read the wiki?
I want you need to you need to not ask me this.
You need to read the wiki. So, is the wiki updated?
Um, so all of that can be automated and
you don't need to read the wiki.
You just ask a question and the wiki will tell you because it's all
there because it was all processed in while people are working.
Um, I hope that answered the question because the the knowledge
part is the thing that I feel is the biggest thing.
Absolutely. No, it it totally did.
There is one other piece that I wanted to
ask before we kind of wrap this up.
Um, and it was something that you mentioned to me about
turning what people know into infrastructure.
And I think that's what you were kind of touching on a
little bit here, but I' I'd like it if you could kind of
explain exactly or actually what does it mean and how
can organizations begin doing something like that?
Because just like you mentioned, somebody with extensive knowledge,
not just in the industry, but in a specific organization, how it
works, and they leave after 30 plus years, there's no one else in
that building, in that organization that knows what they know like
how do we make sure that we put this into the infrastructure.
So this is the point of the knowledge base.
Is again, I'm not trying to say that this is easy.
It's very hard, especially at scale. It's hard to scale this.
And it's also hard to do it retroactively, right.
But the way I have done it in small scale
and I believe it can grow with work.
Uh I never want to say these things are easy.
So is that the way I look at it is every piece of intelligence every
piece of information any report that anyone creates, needs to live
in a knowledge base that can that can uh compound and aggregate.
And as you're doing that, and you uh you're having people
even talk to the LLM as they're doing an investigation.
And that's the thing, this is where kind this comes full circle to
the conversation of do we make people, do we not need people anymore?
No, we need someone to talk to the LLM
as doing they're doing something.
And that's how the knowledge base, learns from you pushing back,
and asking questions, and all of that getting aggregated.
Like what did Assaf ask? What did Hailey ask a week ago?
And where did that take us?
And as you grow that, you are able to build that into future
decisions, you instead of saying, "Oh, does this person this person was here?
Do they remember what happened when we tried something like this?
" We actually know what happened exactly when
we tried something like this 5 years ago.
Um and that's how this works into kind of corporate infrastructure
or even small company infrastructure that you have the capability to
make informed decisions without, it's not only people leaving.
It's, people don't remember what they did a week ago.
People don't remember what they did a month ago.
But their decisions live a long time.
And also the way that they reach the decision, those
decisions that can aggregate, and is queryable.
And you can have the LLM reason on that.
And the interesting part is, I'm sure you've had this.
You were doing an investigation and there's a threat actor,
and you said the threat actor's name, uh out loud.
And someone from the other side of the room like, oh that guy?
Oh Yeah. I worked in that three years ago.
And if you didn't say it out loud, you would waste a lot of time.
So the knowledge base would say, Oh, that guy?
This person worked at it three years ago. Here's all the information.
Here's everything you need to know.
So that's how you incorporate that into the work.
And it's not only in investigations. You can do that in anything.
Uh it's just compounding and expanding knowledge, live.
Yeah. So true. So true. Okay. Last last question.
What should fraud leaders uh be doing today to develop
their investigators into the people who will help design,
supervise, and improve tomorrow's fraud systems?
They should sit with each one of them and say,
What does our system look like today?
What does our system need to look like if we incorporate AI into it?
What they should not do, is take AI and bolt it on
their current structure and current processes. It's very tempting.
It's fast. It's cheap and it looks good.
And a lot of people are going to do that.
But that's the way to, you need to talk
to the people who are doing the work.
Have them break the work down completely, and rebuild it.
And I know it sounds bananas, because it's it's a lot.
But that's the way to do it.
It's, AI is not something you bolt on top of what you're doing.
I think it's also just recognizing, you know, the importance
of the role of the the fraud investigator in all of this.
And so I I guess I maybe fib to you.
I'll ask you one more question.
5 years from now, what do you think we'll realize that we
completely misunderstood about the role of the fraud investigator?
I think at least in a perspective of what we're looking at it now, I
think what we're looking at in the fraud investigator is like someone
who can make these bounded decisions and provide yes or no answers.
And that's a very simplified way to do it um to say it.
But I think what we're not seeing right now is the capability of
the fraud and abuse investigators, uh the investigator to ask the
right questions, to really really deeply understand problems.
And we are not leveraging that right now.
Um at not everybody is leveraging that to the point of like I can
really leverage this way this person thinks to help at scale.
And what AI is do is doing is unencumbering the fraud
investigator and the fraud invest management from the
minutiae of like the the doing the menial things.
And we're going to see that the investigator mindset is the the
mindset that helps grow processes, and find patterns at scale.
Which I know a lot of business investigators trying to do right now,
and a lot of them are getting burnt out hard, because they see it.
But they can't do anything about it.
So what my hope is that in 5 years all the small things are automated
enough, that we are able to make these large determination use, fraud
investigators to make these large determinations at scale.
I love that that we're proving the
need for the investigator, right?
And the fact that they aren't going anywhere.
I will never forget I I even told you this.
I um posted a picture once on LinkedIn of my daughter.
She gave me a thing for like the Mother's
Day things that they do at school.
And it's like my mom is a fraud fighter and when
I grow up I want to be a fraud fighter.
And I posted on LinkedIn and some jerk, that's the nice
way to say it, commented was like well too bad there won't
be one whenever she's uh old enough to get a job.
And I was like you are wrong.
It's it's a it's a sound bite.
This is the problem I have when people say that.
It's not that they're wrong or right, is that they have nothing
to back it up, and they're just repeating a sound bite.
It's funny because you say a lot of people
say AI is not original thought.
It could be, but then again, that's not an original thought.
You're just repeating something you heard
without any information to back it up. And that's always frustrating.
Um, and and also it's just one of those things
that I think about even when I grew up.
Last last uh episode I talked about being a Gamecock
fan, SEC Gamecock band, and we have been rebuilding
for as long as I've been alive.
And we haven't had very successful years of of training.
And I'm sitting here and I'm going, is it
because it wasn't an original thought?
Did my parents just tell me I was a fan and now
I'm a fan and I can't help but be a fan?
I didn't make my own judgment call and now here I am stuck
being disappointed by my football team every year.
As long as we beat Clemson, I'm okay.
Assaf, thank you so much for joining me.
I think that this is just such an important conversation.
Especially as you know, fraud teams are are being
asked to do more, move faster, and figure out
where AI really fits into their operations.
We we spend a lot of time talking about what technology
can do, but we can't lose sight of the people who understand
the problems that we're trying to solve.
So to everyone listening, I'd encourage you to think about how
much knowledge exists within your own fraud team, that may
never have been documented or shared or fully recognized.
You know, how do we make sure we're using AI to strengthen
that expertise rather than losing it?
So, thank you again so much for joining me today.
All right. Thank you, Hailey. I appreciate it.
All right, fraud fighters, stay vigilant, stay
informed, keep moving Fraud Forward.
Thanks for listening to Fraud Forward.
Remember, every conversation, every connection, and every insight
moves our industry one step closer to stronger fraud defenses.
If today's episode sparked an idea, share
it with your team or tag me on LinkedIn.
I love hearing how you're moving Fraud
Forward in your own organization.
Until next time, stay curious, stay resilient,
keep moving Fraud Forward.