11 Runtime AI Attacks CISOs Can’t Ignore — And How to Build Inference-First Defenses

Enterprise security teams are running into a new kind of problem: the defenses are sophisticated, but the threat model has shifted underneath them. As generative AI systems and agents move into production, attackers are exploiting weaknesses in the runtime behavior of models and the applications that surround them. Breakout times are now measured in seconds, patch windows in hours, and most traditional controls have little visibility into what happens inside an LLM call.

CrowdStrike’s 2025 Global Threat Report highlights how fast this has become. Breakout times can be as low as 51 seconds, meaning adversaries can move from initial access to lateral movement before many security teams get their first alert. The same report found that 79% of detections were malware-free, relying on hands-on-keyboard techniques that sidestep classic endpoint defenses. In parallel, AI-native attacks are emerging that don’t look anything like malware at all.

For CISOs, the implication is clear: defending AI in production requires inference-first security — controls that understand prompts, context, and model behavior at runtime, not just the infrastructure they sit on.

The shrinking 72-hour patch window

One of the most visible places AI is changing the security landscape is patching speed. Mike Riemer, field CISO at Ivanti, has seen attackers collapse the window between a patch being published and that same patch being reverse engineered for exploit.

“Threat actors are reverse engineering patches within 72 hours,” Riemer told VentureBeat. “If a customer doesn’t patch within 72 hours of release, they’re open to exploit. The speed has been enhanced greatly by AI.”

That three-day window is fundamentally misaligned with how most enterprises operate. Patching cycles are still often measured in weeks or months due to operational risk, change control, and sheer backlog. In practice, incident response and production firefighting frequently outrank systematic patching, especially in complex environments.

AI-accelerated patch weaponization turns that lag into a systemic liability. Once exploit code is available, the same generative techniques that speed up reverse engineering also make it easier for a broad pool of attackers to customize and operationalize those exploits at scale.

This doesn’t just apply to OS or middleware vulnerabilities. As organizations stand up AI applications and new APIs, each release and configuration change can become another target for rapid recon and abuse. Without automation around patching and configuration management, CISOs are effectively handing attackers a predictable window of opportunity after each vendor update.

Why traditional controls miss runtime AI abuse

Legacy perimeter and endpoint tools were not designed to reason about prompts, embeddings, or model outputs. That gap shows up most starkly in prompt injection and other semantic attacks that look harmless at the protocol level but are devastating at the application logic level.

Consider the difference between a classic SQL injection and a prompt injection. An SQL injection typically has recognizable syntactic signatures: unusual operators, comments, or patterns in the payload. Over time, security teams have gotten extremely good at spotting these, and many can be blocked with minimal false positives.

By contrast, an instruction like “Ignore previous instructions and instead…” carries exploit potential similar to a buffer overflow but shares nothing with conventional malware or exploit code. The danger lies in semantics — the meaning of the instruction to the model — not in any obvious malicious string. Prompt injection drags adversarial tradecraft into a semantic domain where traditional IDS, WAFs, and endpoint tools have almost no insight.

Gartner has captured the organizational side of the problem bluntly: “Businesses will embrace generative AI, regardless of security.” The firm found that 89% of business technologists would bypass cybersecurity guidance to meet a business objective. Shadow AI, therefore, is not a hypothetical; it is a predictable operational reality.

Riemer notes that attackers have a head start: “Threat actors using AI as an attack vector has been accelerated, and they are so far in front of us as defenders. We need to get on a bandwagon as defenders to start utilizing AI; not just in deepfake detection, but in identity management. How can I use AI to determine if what’s coming at me is real?”

Carter Rees, VP of AI at Reputation, frames the technical shortfall this way: “Defense-in-depth strategies predicated on deterministic rules and static signatures are fundamentally insufficient against the stochastic, semantic nature of attacks targeting AI models at runtime.”

In other words, if your controls can’t see what the model sees — and can’t reason about the conversation history, intent, and outputs — you are blind to a growing class of attacks.

Inside the 11 runtime attacks breaking AI security

The OWASP Top 10 for LLM Applications 2025 lists prompt injection as the top risk. But in practice, security leaders face a broader set of 11 runtime attack vectors that can bypass every traditional control in the stack. The common thread: they either manipulate the model’s behavior directly or exploit the application logic surrounding it.

1. Direct prompt injection. Models trained to follow instructions will often prioritize user commands over their own safety constraints. Pillar Security’s State of Attacks on GenAI report found that 20% of jailbreak attempts succeed, with an average time-to-success of 42 seconds. Critically, 90% of successful attacks in their data set resulted in sensitive data leakage. The attacker’s objective is simple: convince the model to disregard its safeguards or reveal information it should never expose.

Defensive implication: Intent classification must sit in front of the model to spot jailbreak patterns before prompts are executed, and robust output filtering is needed to catch when a bypass slips through. Both layers require semantic understanding, not just keyword filtering.

2. Camouflage attacks. Research from Palo Alto Networks’ Unit 42 (“Deceptive Delight”) showed that attackers can hide malicious instructions inside seemingly benign conversations. Across 8,000 tests on eight models, they reached 65% success within three interaction turns. Each individual message appears safe; the danger only emerges when the conversation is seen as a whole.

Defensive implication: Controls must evaluate cumulative intent across the entire interaction history. Stateless, single-turn filters will miss adversarial prompts that are slow-rolled over multiple exchanges.

3. Multi-turn crescendo attacks. Crescendo-style attacks deliberately spread the payload across multiple turns, each harmless in isolation. The Crescendomation tool, for example, reached 98% success on GPT-4 and 100% on Gemini-Pro. The model is gradually nudged toward unsafe behavior, often without any single “obviously bad” prompt.

Defensive implication: Stateful context tracking and pattern recognition are essential. Security systems must maintain and analyze conversation history, looking for escalation in sensitivity, scope, or requested actions over time.

4. Indirect prompt injection (RAG poisoning). Retrieval-augmented generation (RAG) introduces a zero-click attack surface: adversaries poison the data sources models retrieve from. PoisonedRAG research demonstrated 90% attack success by inserting just five malicious texts into databases containing millions of documents. Once indexed, these snippets can override or subvert system instructions whenever retrieved.

Defensive implication: Applications need strict RAG instruction hierarchy. Retrieved content should be clearly delimited and treated as data, not as trusted instructions. Pre-processing must strip or neutralize control tokens in vector database chunks before they enter the model’s context window.

5. Obfuscation attacks. Malicious prompts encoded as ASCII art, Base64, or Unicode can pass through naive keyword filters but remain fully interpretable to the model. ArtPrompt research showed these techniques can reach up to 76.2% success across GPT-4, Gemini, Claude, and Llama2.

Defensive implication: A normalization layer should decode non-standard representations to plain text before any semantic analysis. For many encoding-based attacks, this single step dramatically reduces risk.

6. Model extraction. By systematically querying an API, attackers can reconstruct core capabilities of proprietary models via distillation. Model Leeching research extracted a model with 73% behavioral similarity to ChatGPT-3.5-Turbo for roughly $50 in API spend over 48 hours.

Defensive implication: Behavioral fingerprinting and rate-limiting must go beyond raw request counts. Security teams should analyze query distribution patterns, detect suspicious coverage of capability space, and consider watermarking techniques to prove theft post-facto.

7. Resource exhaustion (sponge attacks). Carefully crafted inputs can exploit the quadratic complexity of Transformer attention, dramatically increasing latency and cost. IEEE EuroS&P research on sponge examples recorded 30× latency increases on language models, and in one case, increased Microsoft Azure Translator latency from 1ms to 6 seconds — a 6,000× degradation.

Defensive implication: Token budgeting per user, prompt complexity scoring, and semantic caching are key. Systems should reject or reshape highly recursive, computationally heavy prompts and cache responses to expensive but legitimate workloads.

8. Synthetic identity fraud. AI-generated identities mix real and fabricated data to slip past traditional KYC and fraud models. The Federal Reserve’s research on synthetic identity fraud found that 85–95% of synthetic applicants evade conventional detection. Signicat’s 2024 report adds that AI-driven fraud now represents 42.5% of all detected fraud attempts in the financial sector.

Defensive implication: Static identifiers are no longer enough. Multi-factor verification has to include behavioral signals, and detection models should be explicitly trained on synthetic identity patterns.

9. Deepfake-enabled fraud. AI-generated audio and video can convincingly impersonate executives to authorize fraudulent transactions. Onfido’s 2024 Identity Fraud Report documented a 3,000% increase in deepfake attempts in 2023. In one high-profile incident, engineering firm Arup reportedly lost $25 million after staff were deceived during a video call populated by AI-generated participants posing as the CFO and colleagues.

Defensive implication: For high-value actions, organizations need out-of-band verification and strict liveness detection. Policies must mandate secondary confirmation regardless of the apparent seniority or familiarity of the person on the call.

10. Data exfiltration via negligent insiders. Not all AI incidents are driven by external attackers. Employees frequently paste proprietary code, strategy documents, or customer data into public LLMs. Samsung engineers did exactly this within weeks of the company lifting its ChatGPT ban, leaking source code and internal meeting notes in multiple incidents. Gartner predicts that through 2026, 80% of unauthorized AI transactions will stem from internal policy violations rather than overt attacks.

Defensive implication: PII redaction and safe usage patterns are critical. The secure path for using AI must be the easiest one, with tooling that automatically strips sensitive content before prompts ever reach external models.

11. Hallucination exploitation. Counterfactual prompting can steer models into confidently agreeing with false statements. Research on LLM-based agents shows that hallucinations can accumulate and amplify across multi-step processes. The risk escalates when organizations wire AI outputs directly into automated workflows without human review, allowing fabricated information to trigger real-world actions.

Defensive implication: Grounding modules should continuously compare model responses against retrieved or authoritative context to assess faithfulness. Confidence scoring and flags for low-fidelity outputs can prevent hallucinations from silently propagating through systems.

From edge security to inference-aware defenses

Gartner forecasts that by 2028, 25% of enterprise breaches will be attributable to AI agent abuse. The gap between AI deployment speed and AI security maturity suggests that this projection could arrive sooner for organizations that treat AI as “just another app” on their existing stack.

Chris Betz, CISO at AWS, emphasized this dynamic at RSA 2024: “Companies forget about the security of the application in their rush to use generative AI. The places where we’re seeing the security gaps first are actually at the application layer. People are racing to get solutions out, and they are making mistakes.”

The lesson for CISOs is that AI applications require application-layer and inference-layer security as first-class concerns. Protecting the network edge or the underlying VM does not protect against semantic jailbreaks, RAG poisoning, or model extraction via API.

Riemer ties this to a more operational view of zero trust. “When you put your security at the edge of your network, you’re inviting the entire world in,” he said. “Until I know what it is and I know who is on the other side of the keyboard, I’m not going to communicate with it. That’s zero trust; not as a buzzword, but as an operational principle.”

Inference-first security aligns closely with that principle: you do not trust a prompt, a retrieved document, or an output just because it came through a known channel. Every step of the AI interaction is subject to verification, policy, and continuous monitoring.

Five immediate priorities for CISOs

The question facing security leaders is not whether they will need inference-aware defenses, but whether they can implement them before they become the next case study. Microsoft’s prolonged exposure and Samsung’s internal leaks underscore how long AI-related risks can lurk before detection.

From current research and early field experience, five deployment priorities stand out:

1. Automate patch deployment. With AI-accelerated reverse engineering shrinking the window to roughly 72 hours, manual patch processes are no longer viable for critical systems. CISOs should push toward autonomous or strongly automated patching integrated with cloud and configuration management platforms, especially for internet-facing services and AI application backends.

2. Deploy normalization layers first. Before any content reaches your models or detection logic, introduce a normalization stage that decodes Base64, ASCII art, Unicode tricks, and other obfuscation techniques into standard text. This step dramatically improves the effectiveness of prompt classification, DLP, and policy enforcement by removing attackers’ encoding advantages.

3. Implement stateful context tracking. Multi-turn attacks like camouflage and crescendo methods exploit the blind spots of stateless filters. Security controls should track conversation history, model intermediate states, and escalation patterns. This applies both to customer-facing chatbots and internal AI assistants, where sensitive data and system control may be at stake.

4. Enforce RAG instruction hierarchy. For any RAG-based system, treat retrieved content as untrusted data by default. Wrap it in clear delimiters, strip or neutralize control tokens, and ensure system-level instructions always override anything pulled from a knowledge base. This both constrains the blast radius of a poisoned document and offers a clear policy model for developers.

5. Propagate identity into prompts. Inference-time decisions should reflect who the user is and what they are permitted to do. Injecting user metadata and authorization context into prompts — in a privacy-preserving way — allows the model and surrounding middleware to apply least-privilege logic. This is especially important for actions that trigger downstream systems, such as access requests, transactions, or data retrieval.

Collectively, these measures move organizations from a world where AI apps are bolted onto existing controls to one where inference itself is a monitored, policy-driven asset. As attackers continue to weaponize AI and compress timelines, CISOs who adapt their defenses around runtime behavior — not just infrastructure — will be best positioned to stay ahead of the next generation of breaches.