1) What Is Multi-Modal Leakage?

Multi-modal leakage is the security failure mode where an AI system that accepts images, audio, or mixed media unintentionally reveals sensitive data or performs unsafe actions because it treats untrusted media as authoritative instructions or privileged context. This is not only “OCR mistakes” or “speech-to-text errors.” The modern risk is bigger: the model’s interpretation can flow into tool access (files, email, calendar, internal knowledge bases, ticketing systems, cloud consoles), turning a perception pipeline into an exfiltration pipeline.

The core issue is structural: many systems do not enforce a hard boundary between:

• Untrusted inputs (images, audio, documents, links)
• Model interpretation (OCR/ASR transcripts, captions, “what it sees”)
• High-impact actions (tool calls, data access, exports, sends, deletes, admin operations)

2) Why This Became a Top Enterprise Risk

Multi-modal AI moved from “cool demo” to “embedded workflow.” Enterprises now use image/audio AI for: invoice processing, KYC, HR onboarding, meeting transcription, SOC triage screenshots, customer support attachments, and executive assistants. That means your AI is routinely exposed to external media from email, chat, web uploads, and tickets — exactly where adversaries operate.

Security frameworks increasingly treat prompt injection and sensitive data disclosure as first-class risks. OWASP’s GenAI guidance highlights prompt injection as a primary class of vulnerability, including indirect injection through external content. NIST’s AI RMF documentation discusses indirect prompt injection risks and the need to manage them through governance, monitoring, and secure design.

3) Attack Surface Map: Where Leakage Happens

Leakage tends to occur at four choke points:

• Pre-processing transforms that change what the model “sees” (e.g., resizing reveals hidden patterns to OCR pipelines).
• Extraction that turns media into text, then treats that text as “instructions.”
• Retrieval where the model pulls sensitive internal data to answer an untrusted prompt.
• Tool execution where the model’s output is executed without strict authorization gates.

4) Realistic Threat Scenarios (Defender View)

Scenario A: “Harmless” Screenshot → Unsafe Action

An employee uploads a screenshot from an external source (ticket, email, chat). The vision pipeline extracts text that contains manipulative instructions. The assistant then tries to perform actions such as searching internal systems, summarizing confidential data, or drafting outbound messages with leaked context. The user believes they requested “summarize this image,” but the system interpreted hidden or embedded content as higher priority instructions.

Scenario B: Audio Clip → “Transcription” → Credential Spill

A call recording is uploaded for transcription. The ASR text includes social-engineering language (“read back the token,” “confirm the one-time code,” “repeat the internal URL”), causing staff to paste secrets into the system or prompting the assistant to retrieve internal resources to “help.” If transcripts are automatically pushed into knowledge bases, the secret becomes durable and searchable.

Scenario C: Document + Images → Retrieval-Augmented Exfiltration

Attackers embed instruction-like phrases in images inside PDFs or slides. The OCR output influences what the RAG layer retrieves. Result: the model fetches unrelated sensitive documents (“to be helpful”), then includes snippets in the answer or an external email draft.

Scenario D: Multi-Modal Agent with Tool Access → “Confusable Deputy”

The most dangerous category is an agent that can call tools (email, drive, calendar, cloud). If the tool layer trusts the model’s text, the model becomes a confusable deputy—following attacker-provided content embedded in media. This is why OWASP’s prompt injection prevention guidance emphasizes least privilege, strict validation, and human confirmation for sensitive steps.

5) Detection Signals & Telemetry

You cannot “patch” this class of risk with one filter. You detect it by instrumenting the whole chain. The goal is to make multi-modal leakage observable.

5.1 Logging That Must Exist

• Media hash + provenance: source system, sender/domain, upload path, device, user identity.
• Transforms applied: resize method, OCR/ASR version, confidence scores.
• Extracted text artifacts: OCR/ASR transcript stored separately as untrusted evidence (with redaction controls).
• RAG retrieval logs: which documents were retrieved, why, and what policy allowed it.
• Tool-call ledger: every tool attempt, parameters, allow/deny decision, and user confirmation events.
• Egress events: external send attempts (email/chat/webhook) including destination risk scoring.

5.2 High-Confidence Alert Patterns

• Instructional phrasing appearing inside OCR/ASR output (e.g., “ignore above,” “system,” “developer,” “tool,” “send,” “export,” “retrieve”).
• Mismatch anomalies: user asked for “summarize image,” but model attempts tool calls (drive search, email send, calendar access).
• Over-retrieval: unusually large context fetches for a simple task.
• Cross-domain egress: newly observed external domains receiving content following media ingestion.
• Repeated failures: multiple denied tool calls in a single session (indicates coercion attempts).

6) Security Controls That Actually Work

The strongest controls focus on system design, not “prompting the model to behave.” OWASP’s LLM prompt injection guidance emphasizes least privilege, validation, and segmentation.

6.1 Treat OCR/ASR as Untrusted Data

• Label all extracted text as UNTRUSTED and keep it separate from system instructions.
• Use explicit “data channel” wrappers internally (policy metadata) so tool logic never treats it as authority.
• Apply redaction on extracted text before it touches LLM context (tokens, secrets, personal data patterns).

6.2 Tool-Call Governance (Non-Negotiable)

• Allowlist tools per workflow (transcription should not have “send email,” “export,” “download files”).
• Parameter validation (destinations, query scopes, max rows, max attachments, safe file types).
• Human confirmation for sensitive actions (external send, privilege changes, data export).
• Least privilege identities for the AI service account (read-only by default, scoped access, short-lived tokens).

6.3 Input Hygiene for Multi-Modal Media

• Enforce media constraints: maximum dimensions, formats, duration limits, safe codecs.
• Run malware scanning and content disarm & reconstruction (CDR) where feasible (documents especially).
• Preview what the model will process: store and review downscaled or transformed versions when pipelines resize content.

6.4 Retrieval Hardening (RAG)

• Apply policy-based retrieval: user entitlement + data classification labels.
• Use query constraints: prevent untrusted OCR/ASR from directly forming retrieval queries without sanitization.
• Set context limits and sensitive-field masking on retrieved passages.

6.5 Privacy & Compliance Controls

• Adopt a risk management process aligned with NIST AI RMF (governance, mapping, measuring, managing).
• Data minimization: store only what’s required, with retention limits for transcripts and extracted text.
• DLP on egress for AI outputs; block secrets leaving to external domains.

7) Reference Architecture: Secure Multi-Modal Agent

8) Incident Response Playbook

When you suspect multi-modal leakage, treat it like a combined data exposure + semantic injection incident. Your IR objective is to stop tool misuse, identify exposure scope, and prevent recurrence.

8.1 Triage (First 30 Minutes)

• Disable high-risk tools for the impacted workflow (external send/export/connectors).
• Quarantine the suspicious media (hash-based block) across ingestion points.
• Pull audit logs: tool-call ledger, retrieval logs, egress events, user sessions.
• Identify whether sensitive data left boundary (DLP hits, outbound destinations).

8.2 Containment (Same Day)

• Rotate credentials/tokens used by AI service accounts; enforce short-lived tokens.
• Apply allowlist-only tool access; require human approval for sensitive actions.
• Increase monitoring thresholds: alert on denied tool calls and anomalous retrieval volume.

8.3 Eradication & Recovery

• Fix the root cause: missing policy checks, excessive privileges, unsandboxed transforms, insufficient redaction.
• Replay the incident with safe test fixtures to ensure controls trigger correctly.
• Update training: “media is untrusted,” tool-call confirmations, and reporting process.

9) 30-60-90 Day Hardening Plan

First 30 Days (Stop the Bleeding)

• Turn on tool-call auditing + centralized logs.
• Implement allowlists and deny-by-default for tools per workflow.
• Add human confirmation for external send/export actions.
• Tag OCR/ASR output as UNTRUSTED and block it from forming tool parameters directly.

60 Days (Make It Durable)

• Policy engine for retrieval + classification-based access enforcement.
• DLP redaction on extracted text and model outputs.
• Sandbox media transforms and document a secure pipeline threat model.

90 Days (Make It Enterprise-Grade)

• Continuous testing: adversarial media tests (internal red team) focused on defense validation.
• Formal governance aligned to NIST AI RMF; regular risk reviews and metrics.
• Playbooks integrated into SOC runbooks with alert tuning and response automation.

10) FAQ

Is this the same as “prompt injection”?

It’s closely related. Multi-modal leakage often involves indirect or embedded instruction influence through images/audio. OWASP treats prompt injection and sensitive information disclosure as top-tier risks in GenAI systems.

Can we solve this with better prompting?

No. You reduce risk primarily with architecture controls: least privilege, tool-call gating, validation, and monitoring. Prompts are not a security boundary.

What’s the #1 mistake enterprises make?

Allowing multi-modal assistants to call sensitive tools without strict authorization checks and confirmation steps. The moment your assistant can “act,” you must treat it like production automation with a security perimeter.

Do we need to disable vision/audio features?

Not necessarily. You can keep them if you implement a secure pipeline with sandboxing, provenance, policy-based access, and least privilege. If you cannot accept residual risk, avoid connecting the system to high-impact tools.

11) References (Defender Reading)

• OWASP GenAI — LLM01 Prompt Injection: Reference
• OWASP — LLM Prompt Injection Prevention Cheat Sheet: Reference
• OWASP Top 10 for LLM Applications (PDF): Reference
• NIST AI RMF (AI 600-1 PDF): Reference
• Multimodal prompt injection research overview (ArXiv HTML view): Reference