Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming
arXiv · HuggingFace · ▲2
Abstract (verbatim)
The fast growth of open-source AI infrastructure, from model serving engines and agent platforms to the Model Context Protocol (MCP) ecosystem and the language models themselves, has outpaced the security tooling available to defend it. We present AI-Infra-Guard, an open-source framework that organizes AI red teaming around a single observation: the attack surface of an AI agent is stratified across layers (infrastructure, protocol/tool, agent behavior, and model), and no single detection paradigm fits all of them. The framework therefore matches a paradigm to each layer, from deterministic rule matching over 75+ AI components and 1{,}400+ vulnerability rules, through LLM-driven agentic auditing of MCP servers and agent-skill packages and multi-turn black-box agent red teaming, to a jailbreak harness with 26+ attack operators over sixteen datasets. To our knowledge it is the only open-source framework to span all of these, including supply-chain auditing of the agent skills that increasingly extend AI agents. We release AI-Infra-Guard as open source so that layer-paradigm matching can serve as a practical foundation for agent security and a shared base for the community to build on.
Background
Background Analysis
1. Technical Context
Recent years have seen rapid growth in open-source AI infrastructure (e.g., model servers, agent platforms, Model Context Protocol ecosystems), creating a new class of network-exposed software. These systems are deployed by individuals or small teams on untrusted networks to build conversational agents, automation workflows, and tool-calling capabilities. However, their security needs differ drastically from traditional software: they must defend against unauthorized access to expensive compute, API credential leaks, prompt injection attacks that hijack agent goals, tool abuse leading to "confused deputy" vulnerabilities, and misalignment of models under adversarial prompting.
2. Previous Limitations
Traditional security tools fail to address these challenges due to three core issues: (1) AI components are too new for existing vulnerability databases (e.g., CVE) to catalog; (2) AI software uses irregular versioning (e.g., rolling updates, development tags), breaking semantic version comparison logic; (3) Most critically, AI’s threat model has shifted—high-risk vulnerabilities are no longer just injection or scripting flaws but require layered detection methods (e.g., infrastructure exposure, protocol bugs, behavioral logic weaknesses, model alignment failures), which single techniques cannot cover.
3. Proposed Solution
The paper introduces "AI-Infra-Guard," a framework based on the principle of layered detection paradigms: different attack surfaces (infrastructure, protocol/tool, agent behavior, model) require tailored detection methods. For example, infrastructure uses rule-based matching and version normalization; protocols employ LLM-driven dynamic auditing; behavior is tested via dialog-based red teaming; and models are evaluated with multi-turn jailbreak harnesses. This ensures each layer gets the most appropriate detection method instead of forcing a one-size-fits-all approach.
4. Key Differentiation
Unlike prior work, AI-Infra-Guard treats AI security as a unified cross-layer problem, implemented through an open-source framework. It covers the full stack from infrastructure to model alignment and introduces innovations like "Prompt-as-Rule," allowing community-driven expansion of rule libraries and attack operators. This design not only solves current tool limitations but also provides a scalable foundation for future AI security research.



