heroui logo

High Confidence Malicious GenAI Assistant Response

Elastic Detection Rules

View Source
Summary
Detects high-confidence malicious GenAI assistant responses by evaluating a generated message with security_ai classifiers. The rule monitors logs from cloud AI services (AWS Bedrock and OpenAI ChatGPT Enterprise) for assistant messages where the security_ai word_obfuscation, encoding, or hash_string probabilities exceed 0.98, or where the refusal model (and its top snippet probability) indicates a high-likelihood unsafe result. A secondary ES|QL step compiles the assistant content (prompt_body) and, if needed, excerpts, then submits it to a completion endpoint (.google-gemini-3.8-flash-completion) to produce a verdict (TP/FP/SUSPICIOUS) with a confidence. The rule only alerts when the completion verdict is TP with confidence >= 0.9. It preserves rich contextual data (security_ai.*, gen_ai.*, aws_bedrock.*, openai_chatgpt_enterprise.*, etc.) for investigation. The documentation includes triage guidance, false-positive considerations, and remediation steps, focusing on detection of data leakage, hidden prompts, exfiltration, or obfuscated payloads in GenAI outputs.
Categories
  • Cloud
  • AWS
  • Web
Data Sources
  • Cloud Service
ATT&CK Techniques
  • T0057
  • T0054
Created: 2026-10-21