AI Agents for healthcare: How to Automate fraud detection (multi-agent with LangGraph)

Healthcare fraud is not a generic anomaly detection problem. You’re dealing with claims abuse, duplicate billing, upcoding, phantom services, identity misuse, and referral manipulation across EHR, claims, eligibility, and payment systems. A multi-agent setup with LangGraph fits because the work is naturally decomposed: one agent triages claims, another validates policy rules, another looks for historical patterns, and a final agent produces an auditable case summary for investigators.

The Business Case

• •

  Reduce manual review volume by 30-50%

  • •In a mid-sized payer or provider network processing 200k-500k claims/month, that typically removes 2,000-8,000 analyst hours per quarter.
  • •The fraud team stays focused on high-signal cases instead of spending time on obvious false positives.
• •

  Cut investigation cycle time from days to hours

  • •A typical SIU or revenue integrity team may take 2-5 business days to triage a suspicious claim cluster.
  • •With agentic pre-screening and evidence assembly, you can get that down to 2-6 hours for first-pass review.
• •

  Lower false positive rates by 15-25%

  • •Rules-only systems are brittle in healthcare because coding patterns vary by specialty, site of care, and payer contract.
  • •A multi-agent workflow can cross-check CPT/HCPCS/CPT modifiers, ICD-10 context, provider history, and member utilization before flagging.
• •

  Recover more avoidable leakage

  • •For organizations with meaningful claims volume, even a 0.1%-0.3% reduction in improper payments can translate into six or seven figures annually.
  • •That includes duplicate claims, unbundling patterns, and out-of-network billing inconsistencies caught earlier.

Architecture

A production setup should separate detection, policy validation, evidence retrieval, and case generation. Don’t build one giant “fraud bot”; build a controlled workflow with explicit handoffs.

• •

  Agent orchestration layer: LangGraph

  • •Use LangGraph to model the investigation flow as a state machine.
  • •Example nodes:
    • •claim_triage_agent
    • •policy_rules_agent
    • •provider_history_agent
    • •case_summary_agent
  • •This gives you deterministic routing, retries, human-in-the-loop checkpoints, and traceable execution.
• •

  LLM application layer: LangChain

  • •Use LangChain for tool calling, prompt templates, structured outputs, and retrieval wrappers.
  • •Keep prompts narrow:
    • •one prompt for claim anomaly explanation
    • •one for policy interpretation
    • •one for investigator summary
  • •Force JSON outputs so downstream systems can score and audit them.
• •

  Evidence store: pgvector + PostgreSQL

  • •Store embeddings for prior fraud cases, denial letters, medical necessity notes, provider profiles, and audit findings.
  • •When a new claim comes in, retrieve similar historical cases by specialty and pattern:
    • •duplicate claim sequences
    • •same-day repeated services
    • •impossible place-of-service combinations
    • •suspicious referral chains
• •

  Control plane: rules engine + case management integration

  • •Keep hard compliance rules outside the LLM:
    • •NCCI edits
    • •LCD/NCD policies
    • •payer-specific edits
    • •credentialing status checks
  • •Push high-confidence cases into ServiceNow, Salesforce Health Cloud, or your internal SIU queue with evidence attached.

A practical deployment usually needs 4-6 engineers, 1 data scientist, 1 compliance lead, and 1 SIU/revenue integrity SME. Expect an initial pilot to take 8-12 weeks if your claims data is accessible and your security reviews don’t stall.

What Can Go Wrong

Regulatory risk: PHI exposure and weak auditability

If the system touches PHI without tight controls, you’re in HIPAA trouble fast. For EU members or multinational operations, GDPR adds data minimization and retention constraints; if you operate as a covered entity with financial partners involved in payment workflows, SOC 2 controls still matter for vendor assurance.

Mitigation:

• •De-identify where possible before LLM processing.
• •Use role-based access control and field-level masking.
• •Log every agent decision path with timestamps and source references.
• •Keep model prompts free of unnecessary PHI.
• •Run BAAs with vendors and validate data residency if GDPR applies.

Reputation risk: false accusations against providers or members

Fraud flags are sensitive in healthcare. A bad model can damage provider relationships or create member trust issues if it incorrectly labels legitimate care as suspicious.

Mitigation:

• •Treat agents as triage assistants only; never auto-adjudicate fraud.
• •Require human review before escalation to SIU or external action.
• •Calibrate thresholds by specialty and geography.
• •Track precision/recall separately for inpatient facility claims, professional claims, pharmacy benefit claims, and DME.

Operational risk: workflow sprawl and alert fatigue

If every edge case becomes an alert, investigators will ignore the queue. That’s how good systems die in operations.

Mitigation:

• •Start with three high-value use cases:
  • •duplicate claims
  • •unbundling/upcoding
  • •impossible service combinations
• •Cap daily alert volume per investigator team.
• •Add feedback loops from closed cases back into retrieval memory.
• •Review drift monthly as coding behavior changes with payer policy updates.

Getting Started

1. •

   Pick one narrow fraud pattern Focus on a pattern with clean labels and measurable impact. Duplicate billing in outpatient radiology or durable medical equipment is usually easier than generalized “fraud detection.”

2. •

   Assemble a small pilot team You need:

   • •1 engineering lead
   • •1 data engineer
   • •1 ML/LLM engineer
   • •1 SIU or revenue integrity analyst
   • •part-time compliance/legal support
     Keep it lean. If the pilot needs more than six people, the scope is too broad.
3. •

   Build the first LangGraph workflow Implement a graph that:

   • •ingests claim metadata
   • •runs deterministic policy checks
   • •retrieves similar historical cases from pgvector
   • •generates an investigator summary with citations
     Add human approval before any downstream action.
4. •

   Measure against operational KPIs Run the pilot for 6-8 weeks on a historical backlog plus live shadow traffic. Track:

   • •precision at top K alerts
   • •analyst time per case
   • •recovery rate of confirmed issues
   • •false positive rate by provider type
     If you can’t show at least a clear reduction in review time or alert noise within one quarter, stop expanding scope.

The right goal is not “AI finds fraud.” The goal is tighter triage with better evidence so your investigators spend their time on actual leakage instead of sorting noise. In healthcare finance workflows that’s where LangGraph earns its place: controlled execution, auditable steps, and enough flexibility to handle messy real-world claims data without turning compliance into an afterthought.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.