New methodology brings unprecedented depth, scale, and adversarial rigor to validating real-world AI security controls
NSS Labs announced the release of its AI Protection Systems (AIPS) test methodology, establishing one of the most comprehensive and technically rigorous frameworks developed to date for evaluating the security of enterprise AI deployments.
As AI accelerates the speed at which vulnerabilities are discovered and exploited, enterprise security teams are facing a new reality: threats are moving faster than organizations can safely respond. Attack techniques are becoming more adaptive, more scalable, and more difficult to detect, while real-world deployment constraints—testing cycles, operational dependencies, and risk management requirements—continue to limit how quickly enterprises can implement change.
AI Protection Systems will be evaluated across eight major dimensions using hundreds of thousands of attack variations.
This growing imbalance is placing unprecedented pressure on security controls to prove they work. In this environment, assumptions about protection are no longer sufficient; enterprises require continuous, independent validation of how effectively their AI security controls perform under real-world conditions.
Also Read: CIO Influence Interview with Gihan Munasinghe, CTO of One Identity
The NSS Labs AIPS methodology is designed specifically to address this challenge through a multi-dimensional, adversarial testing approach that reflects the complexity, scale, and unpredictability of modern AI-driven threat environments.
The methodology evaluates AI Protection Systems across eight major dimensions, including prompt injection resistance, data exfiltration prevention, system resilience, policy enforcement accuracy, agentic AI and tool invocation security, observability and auditability, performance and scalability, and cross-model integration. Within these areas, the test program executes hundreds of thousands of attack variations—including obfuscation techniques, role-based manipulation, context and instruction hijacking, RAG poisoning, exploit generation attempts, and API/tool misuse scenarios—designed to simulate how real adversaries attempt to bypass AI security controls.
Recognizing the rapidly evolving nature of the AI Protection Systems market, the NSS Labs methodology is designed to reflect real-world product capabilities rather than impose a one-size-fits-all standard. No single vendor is expected to support every capability defined in the framework. Instead, testing is aligned to each product’s publicly documented features, with results highlighting supported capabilities and areas of differentiation. The goal is not to declare a single winner, but to provide clarity on how solutions perform and where they fit within a layered AI security strategy. This approach supports a defense-in-depth model by helping organizations identify complementary technologies that minimize gaps in protection.
“AI security is fundamentally different from anything we’ve tested before,” said Vikram Phatak, CEO of NSS Labs. “The attack surface is dynamic, context-driven, and constantly evolving. This methodology reflects that reality by combining adversarial testing, policy validation, and system-level resilience into a single, unified framework that mirrors how AI systems are actually used—and abused—in enterprise environments.”
A key differentiator of the NSS Labs approach is its adversarial testing model, where vendors are not given full visibility into test cases in advance. This ensures that results reflect real-world conditions rather than optimized configurations, providing enterprises with a more accurate view of how AI protection technologies perform under unpredictable threat scenarios.
Keysight Technologies, previously announced as the lead partner for the AIPS initiative, continues to support the development and execution of this testing program. Keysight’s expertise in building scalable, real-world test environments contributes to the ability to model complex attack scenarios and measure system performance under realistic operating conditions.
Catch more CIO Insights: CIO as Orchestrator of Cross-Functional Digital Strategy
[To share your insights with us, please write to psen@itechseries.com ]
