How to Integrate Azure OpenAI for banking with CosmosDB for multi-agent systems

Combining Azure OpenAI for banking with CosmosDB gives you a clean pattern for multi-agent systems that need memory, state, and auditability. In practice, that means one agent can classify a customer request, another can retrieve account context, and a third can write the final decision trail into CosmosDB for compliance and replay.

Prerequisites

• •Python 3.10+
• •An Azure subscription with:
  • •Azure OpenAI resource
  • •Azure Cosmos DB account
• •Deployed Azure OpenAI model:
  • •Chat model deployment name, for example gpt-4o-mini
• •Cosmos DB database and container created
• •API keys or managed identity configured
• •Python packages installed:
  • •openai
  • •azure-cosmos
  • •python-dotenv

pip install openai azure-cosmos python-dotenv

Integration Steps

1. •Set up configuration and clients

Use environment variables so your agents can run in local dev, CI, and production without code changes.

import os
from dotenv import load_dotenv
from openai import AzureOpenAI
from azure.cosmos import CosmosClient

load_dotenv()

AZURE_OPENAI_ENDPOINT = os.getenv("AZURE_OPENAI_ENDPOINT")
AZURE_OPENAI_API_KEY = os.getenv("AZURE_OPENAI_API_KEY")
AZURE_OPENAI_DEPLOYMENT = os.getenv("AZURE_OPENAI_DEPLOYMENT")

COSMOS_ENDPOINT = os.getenv("COSMOS_ENDPOINT")
COSMOS_KEY = os.getenv("COSMOS_KEY")
COSMOS_DATABASE = os.getenv("COSMOS_DATABASE", "agentdb")
COSMOS_CONTAINER = os.getenv("COSMOS_CONTAINER", "agentmemory")

openai_client = AzureOpenAI(
    azure_endpoint=AZURE_OPENAI_ENDPOINT,
    api_key=AZURE_OPENAI_API_KEY,
    api_version="2024-06-01"
)

cosmos_client = CosmosClient(COSMOS_ENDPOINT, credential=COSMOS_KEY)
database = cosmos_client.get_database_client(COSMOS_DATABASE)
container = database.get_container_client(COSMOS_CONTAINER)

2. •Create a banking-aware agent prompt

For banking workflows, keep the model constrained. You want structured output that downstream agents can store and route.

def classify_banking_request(customer_message: str) -> dict:
    response = openai_client.chat.completions.create(
        model=AZURE_OPENAI_DEPLOYMENT,
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a banking operations assistant. "
                    "Return JSON with fields: intent, risk_level, requires_human_review."
                )
            },
            {"role": "user", "content": customer_message}
        ],
        temperature=0
    )

    return response.choices[0].message.content

This gives you a deterministic classification layer that other agents can consume. In a multi-agent setup, one agent can handle triage while another handles retrieval or escalation.

3. •Persist agent state in CosmosDB

Store each interaction as a document. That gives you durable memory across agents and an audit trail for regulated workflows.

from datetime import datetime, timezone
import uuid

def save_agent_event(session_id: str, agent_name: str, payload: dict):
    document = {
        "id": str(uuid.uuid4()),
        "session_id": session_id,
        "agent_name": agent_name,
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "payload": payload
    }
    container.upsert_item(document)
    return document["id"]

A common pattern is to use session_id as the partitioning key if your container is designed around conversation memory. That keeps reads fast when multiple agents need the same case context.

4. •Retrieve shared memory for downstream agents

Agents should not operate on isolated prompts only. Pull prior decisions from CosmosDB so each step has context from the same case.

def get_session_history(session_id: str):
    query = """
    SELECT * FROM c
    WHERE c.session_id = @session_id
    ORDER BY c.timestamp ASC
    """
    params = [{"name": "@session_id", "value": session_id}]

    items = list(
        container.query_items(
            query=query,
            parameters=params,
            enable_cross_partition_query=True
        )
    )
    return items

This is where multi-agent systems become useful in banking. A dispute-handling agent can read the fraud-triage agent’s output before deciding whether to escalate.

5. •Orchestrate the full flow

Here’s a simple end-to-end flow: classify the request, persist the result, then fetch memory for the next agent.

def process_banking_case(session_id: str, customer_message: str):
    classification = classify_banking_request(customer_message)

    save_agent_event(
        session_id=session_id,
        agent_name="triage_agent",
        payload={
            "customer_message": customer_message,
            "classification_raw": classification
        }
    )

    history = get_session_history(session_id)

    return {
        "classification": classification,
        "history_count": len(history),
        "latest_event": history[-1] if history else None
    }


result = process_banking_case(
    session_id="case-1001",
    customer_message="I see an unauthorized card transaction of $480."
)

print(result)

Testing the Integration

Run a small smoke test that verifies both services are reachable and the data path works end to end.

test_session_id = "test-session-001"

event_id = save_agent_event(
    test_session_id,
    "test_agent",
    {"status": "integration_ok", "source": "smoke_test"}
)

items = get_session_history(test_session_id)

print(f"Saved event id: {event_id}")
print(f"Retrieved items: {len(items)}")
print(f"First item status: {items[0]['payload']['status']}")

Expected output:

Saved event id: 3f2b6f2a-8d7c-4f6a-bc9e-2a3a5d7e1d91
Retrieved items: 1
First item status: integration_ok

If this passes, your Azure OpenAI banking workflow can generate structured outputs and your CosmosDB layer can persist and replay them across agents.

Real-World Use Cases

• •

  Fraud triage pipelines

  • •One agent classifies suspicious activity with Azure OpenAI.
  • •Another agent pulls prior cases from CosmosDB.
  • •A final agent drafts analyst notes and stores them back into the case record.
• •

  Loan servicing assistants

  • •Use Azure OpenAI to interpret borrower messages.
  • •Store application state, missing documents, and next actions in CosmosDB.
  • •Let separate agents handle intake, verification, and escalation without losing context.
• •

  Policy servicing for insurance-linked banking products

  • •Agents can answer coverage-related questions tied to premium financing or bundled financial products.
  • •CosmosDB keeps per-customer conversation memory and decision history for compliance review.

Keep learning

• •The complete AI Agents Roadmap — my full 8-step breakdown
• •Free: The AI Agent Starter Kit — PDF checklist + starter code
• •Work with me — I build AI for banks and insurance companies

By Cyprian Aarons, AI Consultant at Topiax.