Hi everybody and welcome to another live special episode from
Money 20/20 Vegas with me today Pedro Sanzovo from Plaid.
Welcome to the show. Lovely to meet you.
Excited to be here Money 20/20 and talk to you about fraud.
Yeah. So we just started the conference has just started.
I want to speak to you shortly about found foundation models in fraud.
Tell us a bit about who you are, what you've been up to so far.
Yeah, absolutely. Uh, so I joined Plaid only six months ago.
But I've been in the fraud space for, well my whole career really.
That was my first foray into defensive technologies, let's call it.
Moved back to Brazil and started my own company in the transactional
fraud space focused on the on merchants and servicing merchants,
helping prevent chargeback, uh, all that jazz.
We we ran that company for about five or six years.
Uh and had a successful exit to Signified.
Back in 2018 at Signified, I was leading a lot of the the uh the
product work most focused on machine learning, modeling, making
sure that we're creating top-of-the-line models there.
And I had the opportunity early this year to come over to Plaid
uh and help us continue building this incredible work that
Plaid has been doing over the last few years on building
really good identity and fraud products for the market.
Let's talk about foundation foundation models.
Yeah that's that's the hippest term now in uh in fraud.
Let's start with the basics like what's foundation models?
Uh how are they different from regular legacy machine learning models?
The the way that I like to think about it like think
about the history of fraud prevention, right?
Uh 10 maybe 15 years everyone was basically creating
those set of chained rules, right?
Uh waterfalls that were basically making decisions.
Um at some point the the industry realized that there
were a type of algorithmic approach namely XG boost.
So the uh extreme gradient boosted uh random forests that allowed
for a lot more data to be encompassed into micro decisions that
ultimately resulted in better prediction with super low latency
and those became effectively the state-of-the-art in the
front end industry for the last 10 years at the very
They also look very much similar to rules,
right, from an algorithmic perspective.
Yes. Right. And they have the same downsides of rules.
What are the downsides of rules?
They on one hand they're very they're very labor some to build.
There's a lot of feature engineering
that needs to go into it, right?
Adapting the index, the features, yeah.
You need data scientists to actually be coding up the features,
understanding what the traffic is telling you.
And then like whenever there's a change in behavior, you need to
put your data scientists, your fraud investigators should actually
understand like okay what the hell is happening here?
How can we codify that into a feature? Let's build that feature.
Let's throw into the model. Let's train it offline.
Let's move it online and then try to figure out
if it actually providing lift and precision.
Um and it's also very very dependent on label
sets and very clear label sets, right?
Supervised learning. Exactly. Supervised learning.
So you need to point the model to this is
the thing that I want you to predict.
This is the thing that I don't want you to predict, right?
And you need to keep those models
And that's very labor intense, right? That's not easy.
It's very complicated, especially for smaller companies, right?
Uh and then you get into this game that if you're
labeling things incorrectly, you're screwed.
Trash in, trash out, right? It's a massive problem.
What happened over the last let's say five years or so
is that um super super super smart people figured out
that there was different way of codifying information
that does not resemble those decision trees, right?
And those are the transformer-based models.
Fundamentally those transformer-based models is what is
backing the LLMs right that we all use uh to an extreme
now like ChatGPT, Anthropic, Claude all that jazz.
And all it does to a certain extent is that it codifies
information as a sequence of inputs, right?
And LLM's those are words or those are letters and then tries to
predict what is the next most likely output of that sequence.
Again for LLMs that's a word that's a letter
What what If we could do the same for fraud, right?
What if we could do the same for financial data?
I I think that's the question that everyone started asking.
It's like, okay, instead of codifying letters and words,
what if we codified financial information, right?
So that is what well we at Plaid started doing and a
lot of players in the industry are doing as well.
We're codifying financial information.
And we're codifying our data as the sequence of events.
And then we're creating those things called embeddings, right?
Which ultimately are the inputs for the model
to try to figure out, okay, what is the next
Instead of these handcrafted features that you described before for
post, we have now embeddings that are created by the foundation model.
And the beauty of this approach is that you rely less on
humanmade feature engineering and also label science.
You still need to some labels.
You need to feed into it, but it doesn't need to be the same
degree of precision that actually boost models needed.
There there are a few downsides and we're still
kind of learning how to use them, right?
Uh but actually before we go to downsides, they're good.
They're predictive, right?
At least I think we we're seeing right
now an uplift in precision up to 34%.
That's very much in line with what we're seeing.
That's in line with what we're seeing in industry.
Uh but again, I think what's important to keep in mind is
that the same problem that existed in actually boost trash
in trash out still exists to a certain extent.
Interesting. How does trash look like for a foundation model?
If your data is not clean, if the data quality is poor and on its
essence, the model is not going to be able to codify those sequences.
So a simple example is one of the most interesting elements
of those foundational transform transformer based models
is uh the time based events and sequencing, right?
If you don't have a really good way to key those times and like
that's something that if anyone has ever worked with data, you
know that making sure that your time columns or time data assets
are precise and they're not getting funky with like time
zones and whatnot, it's easier said than done, right?
And that problem still exists in transform models.
So you need to make sure that that underlying data asset is
trustworthy, is rich, and then the algorithm is going to do its work, right?
And it's going to do its magic.
Uh but it doesn't solve for the foundational issue.
And it sounds simple, but I think where where I've seen
organization kind of like get it wrong or get it messy
is exactly where you know like status has arrived.
And you need to kind of like keep the time stamp of the status
or velocity attributes and like when did I actually see these
links or connections and when you don't have that.
And now it's interesting because those models they're able to absorb
a lot more data than actually boost trust or to an extent, right?
It's like the whole kind of dynamic that we're seeing even on
the LLM is that the more compute that we put into it, the more
data that we put into it, the more precision it shows.
In theory, that should be the same that we
should see in the financial ecosystem.
The more data that we put into it, the more compute that we put
into it, the better precision we're going to see, right?
But the problem is that data we don't have a 8:25 limitless
amount of good quality cleaned data, right?
What we're learning is okay like we have our essential
set of that asset that is going to the model.
Now let's bring this new that asset that we have
never been able to incompass an extra boost. Oh damn, okay.
That that asset is not ready for it it needs to be cleaned it needs
to be treated it should like we need to start collecting it.
Data that we weren't collecting before that
we weren't storing the right Hey. Yeah.
Um, so there's there's again a lot of
foundational work to make that work.
And I think that, you know, to an extent it still is
the same problem that we had with again legacy. It's funny.
It's funny to kind speak about like that
with legacy machine learning models.
Also there you cannot really train on fresh data without
auditing the data and making sure that it's it's
clean and the labels are clean and so on. Last question.
Uh super new, like what two months maybe, but we now have Jev.
Uh so you know we're kind of like we we are just getting our heads
wrapped around this concept of foundation models and now we have the
system one or decision models or whatever you want to call it, uh that
you know work differently than uh than you know LLM and foundation models.
Uh they are much more fit uh for a purpose in terms of
um you know like real time operations and so on.
How do you see the play between you know we have like
machine learning models, we have foundation models
and we have now decision decision models.
How do you see like the the scope here?
I mean it's you got to think about what are your target variables
in any exercise and what are your constraints?
For the world of fraud prevention, Generally what I'm
thinking about when I think about target variables,
I'm thinking about precision recall right?
I'm thinking about true positives, false positives,
the quadrants that we all know.
What but I got to think about the constraints.
And a big constraint is latency
So depending on the use case that you're facing latency
constraints are going to be wildly different.
If you're trying to make like transaction transaction decisions for
credit cards for instance or debit cards, your latency budget is
very very minimum generally under 300 milliseconds or or or even lower.
If you're doing onboarding or promo abuse, then
your latency constraints are way larger.
So then you have a lot more room to play with. Okay.
Optimizing the target variables.
And so the way that I think about it is if you have a
good data science team and a good data engineering team,
you're bound where where it's almost mandatory that
you need to explore the options out there. You need to do research.
And then you need to have clarity on what are
you actually trying to optimize for, right?
Uh and honestly, I think the beauty of our of our industry
is that the data tells the truth ultimately.
If you have a model that beats your precision with the same kind of
constraints and respecting the constraints in place, it wins.
Right. Then there's another, if I may,
There's another thing that gets me concerned about this
new age of modeling which is explainability, right?
So like one of the things and XG boost never really solved that but
It's much easier with XG Boost.
It's much easier, right? It's much easier.
You can get a sense like humans can interact with
the outputs of those models in decent ways.
It's much harder to do with trans with foundational
models for transformer based models and with Jev
I imagine that the constraint is the same. The problem is the same.
So, how can we walk that line?
And again, it depends a lot of the use case, right?
If we're talking about risk, credit has a
regulatory mandate for explainability.
That is very different from fraud.
And and I would add maybe to to close uh things off, I would
add that, you know, these embeddings, you know, they they
short like it's a shortcut to feature engineering.
I don't need the talent. I don't need the effort.
I don't need to handcraft it.
But I can also not really use them in rules.
Yep. Exactly. So, so yeah,
So then you need to kind of like go through that
effort in order to to like write rules.
Totally. And then it gets into I think you're totally right.
And that's well immediately I start thinking about agents, right?
It's like okay, you know who can kind of understand some of those
embeddings but in the humans potentially agent, right?
So like things that we're experimenting with is
okay can we feed agents embeddings and can they
transform some of those into rules, right? Can they find the gaps?
Uh and obviously you you still need a human in the loop to
try to figure out okay like is this actually good work.
But I think that's the direction that we need to explore
Yeah I mean folks uh things are not getting simpler
They are getting more fun I think
The good news it's more fun, more options, more toys to play with.
Pedro enjoyed it. We should uh do a proper sit down
In the podcast and enjoy the conference, man.
Thank you very much. Appreciate it.