Se um dos seus agentes de IA estivesse apresentando uma queda de desempenho há seis semanas, você saberia? A maioria das pessoas me diz que não, e é esse o problema que estou investigando.
Agentes que sofrem desvios podem gerar resultados que parecem bons à primeira vista, enquanto pioram silenciosamente nos bastidores. Anteriormente, nesta série, falei sobre o ciclo de reação como o principal KPI da eficácia no combate à fraude e sobre como a IA agêntica pode tornar esse ciclo muito mais rápido. Desta vez, quero responder à pergunta que realmente importa depois de implementar esses agentes: como saber se eles continuam a funcionar?
A maioria dos painéis responde às perguntas erradas e se limita a indicar se um agente está em execução. Monitorizar agentes de IA significa acompanhar deliberadamente o seu desempenho, e vou explicar-lhe exatamente como fazê-lo.
O que você vai ouvir neste episódio:
- Por que um agente em degradação é realmente mais perigoso do que não ter agente algum.
- Como medir a velocidade do ciclo de reação à fraude em cada etapa individual, em vez de apenas acompanhar um único indicador defasado.
- Por que as métricas de desempenho de agentes de IA que as equipes antifraude devem acompanhar não precisam ser perfeitamente automatizadas para serem úteis.
- A diferença entre o monitoramento de agentes com intervenção humana e o monitoramento autônomo de agentes que tomam decisões em larga escala.
- O que uma taxa de rejeição crescente realmente revela.
- Como detectar um agente autônomo que está falhando silenciosamente antes que os danos reais se acumulem.
- Um painel prático de monitoramento de agentes de IA em três camadas que as equipes antifraude podem criar.
Você deveria ouvir este episódio se:
- Gerencia algum programa de monitoramento de operações antifraude com agentes de IA e quer uma estrutura sólida para identificar a deterioração de um agente antes que ela se reflita em perdas.
- São responsáveis por políticas de prevenção a fraudes na governança de agentes de IA e precisam de uma linguagem que conecte o monitoramento técnico aos relatórios para a liderança.
- Implementaram ferramentas com supervisão humana, como copilotos de investigação ou agentes de recomendação de regras, e querem saber o que realmente devem monitorizar.
- Executam agentes autônomos, como sistemas de rotulagem automática ou de agrupamento de alertas, sem que uma pessoa revise cada decisão, e se preocupam com falhas silenciosas.
- Querem criar um caso de negócio sólido para o investimento em agentes de IA contra fraudes, calculando o ROI com base no ciclo de reação, em vez de considerar apenas as horas de automação poupadas.
Notas do episódio e principais conclusões
O tempo de atividade não é a métrica certa para monitorar
A maioria dos painéis informa apenas se um agente está ativo, e essa é a pergunta menos útil que se pode fazer. Um agente que está se desviando pode parecer perfeitamente saudável enquanto, silenciosamente, piora no desempenho de sua função real. O verdadeiro monitoramento de agentes de IA consiste em acompanhar se o agente continua gerando valor e se comportando conforme o esperado, e não apenas se ainda está em execução.
Divida o ciclo de reação em etapas que você possa realmente medir
Vale a pena acompanhar separadamente a velocidade de deteção, a velocidade de delimitação do âmbito e a velocidade de conceção da correção, em vez de observar uma única média que reflete os resultados com atraso. Nada disto precisa de ser perfeitamente automatizado para ser útil. Um registo manual aproximado, atualizado uma vez por semana, é suficiente para mostrar se os agentes de IA estão a reduzir significativamente o tempo gasto em cada uma destas etapas — e isso é um melhor uso do seu esforço do que esperar até criar primeiro um sistema de medição perfeito.
As métricas no nível do ciclo apresentam defasagem — e esse é exatamente o perigo
Os números do ciclo de reação são médias móveis, o que significa que os sinais de degradação dos agentes de fraude podem ficar ocultos neles durante semanas antes que alguém perceba. Quando uma desaceleração finalmente aparece no tempo total do ciclo, o estrago geralmente já está feito. O ideal é ter sinais que detectem essa deterioração mais cedo.
Agentes com supervisão humana deixam um rastro documental, mas agentes autônomos não
Para agentes que são revistos por uma pessoa antes de qualquer ação ser executada, como copilotos de investigação ou ferramentas de recomendação de regras, a taxa de concordância e a taxa de rejeição são os melhores indicadores antecedentes. Para agentes autónomos que tomam decisões sem qualquer etapa de revisão, como a etiquetagem automática ou o agrupamento de alertas, as falhas são silenciosas por predefinição. A taxa de concordância entre fontes e a mudança na distribuição são os sinais que permitem detetar esse tipo de desvio antes que se agrave e se transforme num problema real.
Estruture seu monitoramento em três camadas
Os alertas em tempo real devem detetar violações dos limites no próprio dia em que ocorrem. Uma análise semanal deve examinar as tendências e o impacto global, e não apenas verificar se foi acionado um alerta. Já um relatório mensal deve relacionar o desempenho do agente com os custos e o retorno sobre o investimento para a liderança. Nada disto exige novas ferramentas, e a responsabilidade cabe à sua equipa de análise de fraude.
Conclusão final
Comecei este episódio com um cenário simples. Um dos seus agentes está a perder desempenho há seis semanas, e ainda não sabe. Se monitorizar a taxa de concordância dos seus agentes com intervenção humana, detetará o problema em poucos dias. Se monitorizar a concordância entre fontes e as alterações na distribuição dos seus agentes autónomos, detetá-lo-á no espaço de uma semana. Ao mesmo tempo, conseguirá demonstrar exatamente quanto valor esses mesmos agentes estão a gerar, não apenas em horas poupadas, mas também em dinheiro poupado graças a uma reação mais rápida à fraude. Essa é a verdadeira diferença entre gerir os seus agentes e deixar que sejam eles a geri-lo.
Recursos e links
Este conteúdo faz parte de uma série. Se você chegou primeiro a esta página, talvez seja melhor voltar e ouvir os episódios anteriores. Já abordamos muitos assuntos que tornarão este episódio bem mais fácil de acompanhar.
Confira A ascensão das operações autônomas de fraude, parte 1
Confira A ascensão das operações autônomas de fraude, parte 2
Confira A ascensão das operações autônomas de fraude, parte 3
Ainda não quer encerrar a conversa sobre o meu assunto favorito — e, espero, o seu também? Assine a newsletter The Saturday Fraud Strategist.
Conecte-se com Chen Zamir | LinkedIn
Apresentador do The Saturday Fraud Strategist
Ajudando fintechs a criar defesas mais inteligentes contra fraudes
Coautor de “The Fraud Fighter’s AI Playbook”
Episode transcript
Chen Zamir
00:00
Here's a question I've been asking fraud leaders lately. If I told you one of your AI agents had been degrading for 6 weeks, would you know? Most would say no. And it's a problem because drifting agents are actually worse than no agents. They generate outputs that look fine from the outside while they degrade. And it's especially problematic when these outputs are being used as inputs for downstream processes like other agents. And so you can imagine how easy it is to encounter cascading events that very quickly get out of control. But the truth is that if you take a look at your average dashboard, it's designed to answer something else entirely than these questions. Is this agent even running? That is the least useful question to ask. I mean, of course, system health is important, but focus only on that and you'll miss the really important things to monitor. Whether your agents are improving your outcomes and not just whether they are live. So, how do you keep track of your agents? There are two things you want to watch for. First, is the agent actually contributing value? And the second, is it behaving as expected? Let's talk about it. In the first video of this series, I mentioned that in my view, the master KPI of fraud effectiveness is the reaction cycle. The time it takes your system to detect a gap and deploy a fix for it. And in the second video of the series, I outline how you can streamline your reaction cycle with Agentic AI to make it significantly faster. But how do you actually measure it? How do you connect AI agent performance metrics to how fast you're stopping fraud? Fraud AI agents can definitely help with that, but not necessarily in an even manner across the board. Here's what I would suggest to track and how. First, detecting coordinated fraud patterns faster. The first step in the fraud reaction cycle is detecting that there's a system gap. Without agents, this is slow by default. Alerts arrive and analysts eventually notices a pattern and then someone needs to pull related cases by hand to validate there's an issue. A new attack can run for days or even weeks before anyone connects a dots easily. But if you follow the steps in the second video of this series and you implemented alert clustering, a new ring can now surface in hours. Every alert triggers an agent that searches against known cases and matches events to existing rigs. So how do you track this? You track it by measuring the time from the first event of a new attack entering your system to when your team actually recognizing it. Now, I know what you're thinking. This is super tricky. So, I let you in on a little secret. Not every metric you measure needs to happen automatically. Just make sure that every week someone in your team goes through the alerts they've actually picked up and note down how long it took since the issue first started. Do it enough times and you'll be able to see a trend, especially if you start doing that before you implement agentic AI, as you should. The second thing you want to do is to scope fraud rings with AI agents. Once a pattern is flagged, the next question to ask is how big is it? How many accounts? What time period? What's the actual loss exposure? This is where Agentic AI moves from spotting a signal to mapping the full population at risk. Without AI, the scoping is manual and analysts will have to pull related cases, cross reference them with device data and build a picture account by account. But with AI agents, the same work can be done in a much shorter time window. How do you track this? Measure the time between when an issue got noticed to when you had it scoped in dollar or account exposure terms. Again, this doesn't have to be super sophisticated. The point isn't to say it takes 5 minutes and 12 seconds to scope an attack instead of 5 minutes and 36 seconds. It's being able to show it takes less than 30 minutes versus the day or two it took before. And that is pretty easy to do in the same manner I mentioned before by recording it manually once a week. By the way, I know that what I just said may sound like sacrilege to some of you. What? Do something manually when we talk about AI monitoring? How can I say that while I preach for more automation? But this is exactly where I see teams get distracted by investing their attention and tokens into productivity hacks instead of focusing on the right things. Don't get me wrong, if you can automate these metrics with or without AI, you should absolutely do so. But don't let it stop you from measuring them if you can't. Finally, the last thing you want to track is how fast you design and test fixes.
Chen Zamir
04:42
Once you understand the issue and what causes it, you need a fix. And AI agent can help with the two steps that used to consume the most time. Proposing a solution and proving it works. When fed a specific data set, agents can identify patterns, learn from labels, and propose a fix like a new rule. Then they can run the back test all before human even reviews it. So the analyst's job shifts from building the rule to stress testing and approving it. Now the reason why I would measure these two phases together is because in some cases like rule writing, it's a bit difficult to know when design ends and testing begins. And in other cases like SOP or policy changes, it might be that there would be no testing phase. So when you think about how to measure it, this one is a bit tricky. Supposedly you need to start measuring it when you have the root cause figured out, but this can prove to be quite a fuzzy definition. So again, you don't have to overengineer it and get a super accurate measure. It's enough to have a ballpark range for how much time it took since you started working on a solution and until it was approved for deployment. You want to see if when using AI agents, you can consistently shave a meaningful chunk of time. And if you don't see through rough measurements, it's likely not doing enough work anyway. Here's the problem with cycle level metrics. They lag. Meaning, you're likely going to measure some sort of a roll in average. And as we just discussed, a rough one at that. So if for example your rules agents start to degrade, it'll be quite hard to catch. Analyst would spend more time on prompting or tasks that it would oneshot before would now take several prompts to achieve. Or a labeling agent that started misclassifying 3 weeks ago will eventually cause your rules to drift and elevate fraud rates, but until you notice it and understand where it's coming from, it might be going on for weeks. And that's why you want to make sure your agents are not only live are not only noticeably making you react faster, but also that they do not degrade with time. And you want to know something is wrong before it shows up in your reaction cycle metrics as by then it's too late. But these signals look different depending on whether the agent has a human-in-the-loop or not. Human-in-the-loop agents are the first type of AI agents you'd monitor. These include investigation co-pilots or rule recommendation agents that have humans reviewing every output before anything gets actioned. And because you have a human in the loop, these agents are easier to monitor because the review itself creates a record. Did the human agree or reject the agents decision?
Chen Zamir
07:26
This creates a natural trail you can record and monitor. Agreement rate is the best leading indicator and one of the most useful agent metrics for human reviewed workflows. If investigators are ruling differently than the agent more often than they used to, then the agent is degrading. For rule recommendation agents, rejection rate is the equivalent. It's another AI agent performance metric that shows whether the agent is still producing useful outputs. It's a bit trickier because rejected rules don't reach production, so they don't necessarily impact your general KPIs. But a rise in rejection rate means the agent is generating faulty proposals that consume human time and resources without moving the needle. And by the way, there can be many reasons for agent degradation. Maybe there are underlying data issues. Maybe fraud has shifted. Maybe moving to a newer model didn't work well. Whatever the reason is, you could identify issues earlier by tracking these metrics. Meaning you could also start solving it faster. Exactly the concept behind the reaction cycle. Autonomous agents such as auto labeling and alert clustering don't have humans reviewing every decision they make. So when they fail, the failure is silent until something downstream breaks. Let me give an example. When an auto labeling agent starts misclassifying legitimate accounts could get labeled as fraud without anyone noticing. Obviously, no one wants that. This is why AI governance has to account for autonomous agents differently than human-in-the-loop flows. But the problem here is scale. The reason humans are not in the loop is because these flows simply make too many decisions or they make decisions in near real time. So not only you cannot track human agreement rates, the danger here is substantial. Any slight degradation can have a severe impact, but you can track other agreement rates coming from other sources or cross- source agreement rates. Let me give an example. Say you want to monitor your labeling agent.
Chen Zamir
09:31
It's not your only source of labels, right? You probably have at the very least the chargebacks you're getting as well. Now, if two sources for the same decision that were aligned 90% of the time drop to 70% of the time, you know something changed. You don't know which source is wrong yet, but you know where and when to look before the damage compounds. Another signal worth tracking is distribution shift. If your labeling agent starts flagging 40% more events as fraud in a segment that hasn't seen elevated fraud rates, that's worth investigating. Again, there can be many reasons for why an autonomous agent degrades. Just like with human in the flow agents, the point is that you now have the metrics and the alerts so you could catch it early. In the previous video of this series, I outlined how managing AI agents, including monitoring them, falls under the responsibility of your fraud analytics function. And from a tooling perspective, it's nothing new. What do we actually have here? Real-time alerts for threshold breaches. For example, when the agreement rate of a human-in-the-loop agent falls down below a certain value or when an autonomous agent's distribution skews more than a set rate. These alerts form the foundation layer of AI governance and should fire automatically on Slack or email or wherever your team catches alerts and get looked at the same day. Then you have weekly reviews of trend lines. Here you're not only making sure everything looks good regardless of whether an alert was fired or not. You also take a look at impact. Measure your reaction cycle speed and make sure that your agents provide actual value. And lastly, you have monthly reports on cost and ROI that you can share with your leadership team. And especially for the reaction cycle, it's pretty straightforward. If you can demonstrate an attack was blocked 4 days faster, you just save 4 days of losses. That's part of the ROI calculation, not just how many work hours you saved with automation. I started this video with a simple scenario. One of your fraud agents has been degrading for six weeks.
Chen Zamir
11:34
What do you know? If you're tracking agreement rates for your human-in-the-loop agents, you'd know in days. And if you're tracking cross agreements and distribution shifts for your autonomous agents, you'd know in a week. At the same time, you can now also track the value agents create. Not only how many hours you saved, but also how many dollars you saved by detecting and reacting to fraud quicker. That's how you connect your reaction cycle to your top KPIs. And this is the difference between you running agents and the agents running you.
View full transcript
