Threat Modeling AI Applications: Start Before the First Line of Code

The cheapest security bug to fix is the one that never gets built. That is the whole case for threat modeling, and it applies to AI systems more than anywhere else — because AI failure modes are architectural, not just implementation bugs.

What's new in the AI threat model

• New assets: system prompts, model weights, training data, RAG corpora, conversation history
• New boundaries: user input to model, retrieved content to context window, model output to downstream tools
• New threats: prompt injection, data poisoning, model extraction, sensitive data disclosure through outputs
• New actors: the model itself as a confused deputy acting with your privileges

The process we run

We start from a data flow diagram with every model call, retrieval step, and tool invocation drawn explicitly, then apply STRIDE per element with an AI-specific threat library layered on top. Trust boundary analysis gets special attention: in most AI incidents we review, the root cause is content crossing a boundary — from a web page, a document, a user — and being treated as trusted instructions.

The output is not a compliance artifact. It is a ranked risk register with concrete mitigations, mapped to your architecture, that your team can implement sprint by sprint. Design-time security is still the highest-leverage work in the SDLC — AI has only raised the stakes.