Risk-based assessment for LLM and the entire system in two stages
NRI Secure Technologies a leading global provider of cybersecurity services, launched a new security assessment service, “AI Red Team,” targeting systems and services using generative AI.
Read More: CIO Influence Interview with Archie Agarwal, Founder and CEO at ThreatModeler
“If the risk becomes apparent, will it have a negative impact on the system or end-users?”
Vulnerabilities and Risks of AI
In recent years, the use of generative AI, especially Large Language Models (LLMs), has continued to grow in many fields. While expectations for LLMs have increased, LLMs have also highlighted the existence of vulnerabilities, such as prompt injection and prompt leaking, as well as hallucination, sensitive information disclosure, inappropriate content generation, and bias risk (see Figure). Companies utilizing LLM technologies need to be aware of these issues specific to generative AI and apply appropriate countermeasures. For this reason, the importance of security assessment specific to generative AI is now being called for, and various countries are beginning to mention the need for assessment by independent outside experts.
Overview and Features of this Service
In this service, NRI Secure’s experts conduct simulated attacks on actual systems to evaluate, from a security perspective, AI-specific vulnerabilities in LLM-based services and problems in the overall system, including peripheral functions linked to the AI.
AI does not function as a service by itself; rather, it constitutes a service by linking with its peripheral functions. It is necessary not only to identify the risk of LLM alone, but also to evaluate it from a risk-based approach, which is to say, “If the risk becomes apparent, will it have a negative impact on the system or end-users?”
Therefore, this service provides a two-stage assessment: Identifying risks in the LLM alone, and evaluating the entire system, including the LLM. The results of the assessment are summarized into a report, detailing the problems found and recommended mitigation measures.
The two main features of this service are:
1. It performs efficient, comprehensive, and high-quality assessments using our proprietary automated tests and expert investigations
NRI Secure has developed its own assessment application that can be automatically tested by employing DAST6 for LLM. Using this application, vulnerabilities can be detected efficiently and comprehensively. Furthermore expert engineers in LLM security perform manual assessment to identify use-case-specific issues that cannot be covered by automated testing, and also investigate detected vulnerabilities in depth.
2. It assesses actual risk across the entire system and reduces countermeasure costs
Generative AI has the nature of determining its output probabilistically. Additionally, because it is difficult to completely understand internal operations, there are limits to how much of and how many vulnerabilities can be uncovered through a partial system evaluation. NRI Secure combines its long-accumulated expertise in security assessment to comprehensively assess the entire system and determine whether AI-caused vulnerabilities are apparent or not. This service also supports “OWASP Top10 for LLM,”7 which is difficult to deal with only by evaluating AI-specific problems.
If the AI itself appears to have vulnerabilities, the system will then evaluate the actual degree of risk from the perspective of the entire system, and can propose alternative countermeasures to avoid having to deal with vulnerabilities in the AI itself, which would be difficult to implement. As a result, the cost of countermeasures can be expected to be reduced.
Read More: CIO Influence Interview with Jim Kwiatkowski, CEO at LTX, a Broadridge Company
[To participate in our interview series, please write to us at sghosh@martechseries.com]
