Before joining Sardine, I spent two decades in banking. First, building end-to-end risk systems for consumer banking and wealth management, then bringing together a group of tier-1 banks to design and operationalize the risk management framework for the Zelle network.

Today, I help build modern fraud detection and risk infrastructure at Sardine, and lead Sonar, an industry utility for counterparty risk and shared intelligence across banks, fintechs, and payment networks.

When we introduced the Agentic Oversight Framework (AOF), it resonated across the industry, but it also raised an immediate question from my former peers in risk, compliance, and policy functions:

“How do we take this from a conceptual framework to something regulators will actually sign off on?”

This playbook is my answer.

It bridges the gap between innovation and regulatory scrutiny, translating the AOF into actionable, supervision-ready components tailored for model governance, third-party oversight, and operational risk teams.

For banks looking to safely deploy AI agents - whether for review and or decisioning, customer service, or workflow automation, this playbook offers the operational scaffolding to align with evolving expectations from regulators.

1. Agent classification and risk-tiering

Rather than creating separate policies for each AI agent, develop a classification map that assigns agents to risk tiers. The tier determines the level of oversight, documentation, and validation required

• Tier-1 (critical impact): Agents that directly trigger regulatory, financial, or legal actions - such as filing Suspicious Activity Reports (SARs), blocking payments, or conducting sanctions checks. These require comprehensive model validation consistent with Federal Reserve SR 11-7, including fallback controls and immutable audit logs.
• Tier-2 (moderate impact): Agents that assist decision-making but do not act autonomously. Examples include supporting onboarding, fraud triage, or KYC workflows. Outputs influence human decisions, so explainability and human-in-the-loop reviews are mandatory.
• Tier-3 (low impact): Agents that support internal functions like knowledge searches or report drafting. They do not trigger compliance obligations but must be logged and monitored to avoid unregulated use in critical workflows. Tier-3 agents may be subject to lighter controls but require reclassification if their impact grows.

This tiered approach aligns with OCC and FFIEC risk-based governance.

2. Technical architecture: Audit-ready, defensible AI systems

For examiners, architecture is part of risk management. A compliant AI system isn’t just accurate. It’s auditable, defensible, and secure by design:

• Inference gateway: Sits between the user and the model. It masks or tokenizes personal data, ensuring models do not access unnecessary sensitive information.
• Agent runtime (model container): Fixes the model version and config at runtime. This prevents “silent updates” that would break explainability or drift from validation.
• Explainability layer: Generates chain-of-thought reasoning, confidence scores, and a one-line summary of why the model made the decision.
• Immutable audit log: Every prompt, input, model, and output is recorded and hashed. This supports FFIEC audit readiness and allows regulators to replay exactly what the model saw.
• Quality assurance (QA) layer for Agent decisions: Beyond logging, systems should enable periodic sampling and structured review of agent decisions, just as they would for human decision-makers. This allows compliance, audit, and QA teams to regularly test for policy alignment, spot-check edge cases, and ensure outcomes remain consistent with expectations. It also supports continuous improvement and satisfies supervisory expectations under frameworks like SR 11-7, FFIEC, and EU AI Act Article 15, which emphasize ongoing oversight over one-time validation.
• Fallback paths: Built-in if the model fails or response quality drops. The rules engine triggers default conservative actions or sends the task to a human reviewer.
• Continuous monitoring: Automated dashboards and alerts to detect model drift, outlier behavior, or performance degradation in real time.

By mapping each part of the system to NIST 800-53 and ISO 27001, your infosec team can certify readiness for both internal audit and exams.

3. Model validation: Supervisory-compliant playbook

AI agents that materially inform or execute decisions are treated as models under SR 11-7, regardless of whether they use traditional machine learning or LLMs. Validation must meet supervisory expectations. As such, validation should include:

1. Policy fit: Review that the agent’s logic reflects bank policy (e.g., denies a transaction for the right reason, flags suspicious behavior per BSA/AML rules).
2. Backtesting: Test the agent on historical cases. Check how often it makes correct decisions, false positives, or false negatives. Then quantify the risk.
3. Robustness checks: Randomize or rephrase inputs to ensure the model doesn’t change outputs based on superficial differences.
4. Adversarial testing: Feed bad inputs (prompt injections, malformed data, extra-long text) and observe failure modes. Does it crash, freeze, or hallucinate?
5. Independent review: Final report is signed by someone outside the build team (per SR 11-7 independence requirement). It must be easy to understand, not just Python code.
6. Bias and fairness audits: Periodically assess for disparate impact, especially in lending, fraud, or onboarding. Track fairness metrics by protected class and document mitigation steps.

This process is repeated periodically, after any model update, or whenever drift is detected by continuous monitoring.

4. Data privacy and cybersecurity controls that minimize compliance gaps

Every agent must follow Zero Trust principles when handling data, meaning it should verify identity, limit access to only the data needed for each task, and log every interaction for audit. No agent should assume internal systems or other agents are inherently trustworthy. No model, vendor, or prompt should access more than what’s absolutely needed.

• GLBA: Financial data must be encrypted and accessed only for defined permissible use. (15 USC §§ 6801-6809)
• CCPA: Individuals have the right to be informed, to request corrections, and to opt out. If AI agents generate messages to customers, these rights must be embedded. (Cal. Civ. Code §§ 1798.100–1798.199)
• NY DFS: Requires 72-hour breach notification, incident response plans, and annual compliance certification. (23 NYCRR §§ 500.1–500.22)
• GDPR Article 22: For EU/UK operations, prohibits solely automated decisions with significant effects without human intervention.
• ISO/NIST: Each system component should be tagged to its control set to support audits and security testing.
• Synthetic data: If used for testing, training, or validation, synthetic data must be evaluated for privacy leakage and membership inference risk, particularly when derived from production datasets. Under NIST SP 800-53 Rev. 5 and emerging guidance in ISO/IEC 42001, banks are expected to demonstrate that synthetic datasets cannot be reverse-engineered to reveal nonpublic personal information (NPI).
• Cross-border data transfers: For global banks, ensure AI agent data flows comply with international data transfer laws (e.g., GDPR, UK DPA).

Don’t assume the AI vendor handles this. Your bank remains the data controller and is liable for misuse.

5. Explainability, auditability, and regulator access

If you can’t explain a decision, you can’t defend it. Every output that informs or executes a regulated decision must be logged with:

• Exact inputs (appropriately redacted)
• Model versions and configuration parameters
• Clear, human-readable rationales for decisions.
• Maintain replayable audit trails to support examinations and consumer dispute processes.

These logs must support adverse action notices under ECOA/FCRA and be retrievable for complaint investigation and compliance QA. The more structured this is, the easier to survive a model governance exam.

6. Vendor risk management: Contractual oversight for safer third-party AI

Where third-party platforms or APIs are involved, contracts must explicitly cover:

• Audit rights: Allow internal teams and regulators to inspect logs, version history, and data lineage
• Security certifications: Require SOC 2 Type II, ISO 27001, and documented penetration tests.
• Incident notification: Mandate 72-hour disclosure of security or model incidents.
• Data exit: Guarantee that your data can be retrieved and deleted. Otherwise, you can’t switch vendors without risk.
• Open-source model risks: If using open-source LLMs or frameworks, ensure vetting, patching, and monitoring for vulnerabilities.

These provisions align with regulatory guidelines on third-party risk management.

7. Failure modes: Guardrails and fallbacks

Operational risk teams must assume failure and define controls for each mode.

• Timeouts and token limits: Fallback to rules engines or human triage.
• Low confidence outputs: Automatically escalate to manual review.
• Toxic inputs and prompt injection: Filter known attack patterns.
• Output drift: Continuously compare outputs to baselines. Retrain or revert as necessary.
• Incident response playbook: Define clear triage, communication, regulatory notification, and remediation steps (NY DFS § 500.16).

All failure scenarios should be mapped to existing incident response procedures. Include model-specific entries in your operational playbooks and business continuity planning.

8. Regulatory reporting alignment: Ensuring ongoing audit readiness

If an agent interacts with a regulated process, map its role to each reporting obligation:

• SAR filing: Outputs must be reviewable by compliance. Store logs for 5 years.
• ECOA/FCRA: Provide model rationale and input data for adverse action disclosures.
• UDAAP: Avoid unfair, deceptive, or abusive automated communications.
• GDPR: Provide human appeal rights for automated decisions (EU/UK operations).

This should be included in control testing and examined periodically by legal and compliance.

9. Customer communication and transparency

For decisions involving customers:

• Provide up-front notice that AI is being used
• Offer a channel for human appeal
• Ensure disclosures meet CFPB and GDPR Article 22 standards
• Avoid vague or buried disclaimers.

Here’s a sample disclosure template:

“This decision was made with the assistance of automated systems. If you have questions or wish to request a manual review, please contact [support channel].”

10. Model inventory and lifecycle management

Maintain a single, version-controlled registry of all AI systems in use or in testing. Include:

• Agent name
• Risk tier
• Owner
• Last validation
• Update history
• Current status
• Audit logs location

Use this for quarterly reporting to model risk and enterprise risk functions. Make sure to archive and version-control all changes for audit readiness. If an unused agent is retired formally, ensure there are controls in place to prevent accidental reactivation.

11. Deployment checklist: Before you go live

Require documented sign-offs from:

• Model risk: Validation, policy alignment, fairness audit.
• Information security: Threat model review, endpoint security, monitoring.
• Privacy: Lawful data processing, minimization, cross-border compliance.
• Procurement: Vendor risk assessment, contractual safeguards, exit planning.
• Operations: Human override testing, real-time response, incident response readiness.
• Internal audit: Log verification, audit trail integrity.
• Compliance: Confirmed alignment with applicable laws and regulations.

Each signoff must indicate the specific control area reviewed, and any conditions placed on deployment. If any team blocks launch, that decision should be documented with justification and escalation route.

12. Continuous monitoring and improvement

Once deployed, agents must be continuously evaluated through:

• Real-time monitoring dashboards and alerts to detect drift and anomalies.
• Periodic challenge testing with synthetic and adversarial inputs.
• Quarterly reviews of risk tiering, validation results, and model registry.

Establish a formal change management protocol that logs, reviews, and approves all modifications to agent prompts, model parameters, or underlying tools. This ensures traceability and policy alignment, and satisfies supervisory expectations under frameworks like SR 11-7 and OCC 2023-17.

It is important to incorporate feedback from incidents, audits, and customer appeals into ongoing improvements. These inputs help surface edge cases that may not be captured through synthetic or backtested data, and ensure that control functions remain responsive to real-world behavior.

Build trust into the architecture

AI agents can reduce decision times from hours to minutes. But without proper controls, they can increase regulatory risk. By establishing tiered oversight, embedding technical safeguards, maintaining traceability, and enforcing human accountability, banks can deploy AI agents that are compliant by design and defensible under examination.

If you’d like to see a demo of Sardine AI agents in action, or would like to learn more about how to safely deploy these agents in production, contact us.