Article 15 EU AI Act requires high-risk AI systems to achieve an appropriate level of accuracy, robustness, and cybersecurity, and to perform consistently in those respects throughout their lifecycle. It is the clause where AI regulation stops being a documentation exercise and becomes a testing one.
It is also the clause most teams underestimate. Declaring accuracy metrics is straightforward. Proving that your system resists an attacker who wants to alter its outputs is a different discipline entirely.
The timing has shifted in your favor. Following the AI Omnibus, which entered into force on 27 July 2026, Annex III high-risk obligations now apply from 2 December 2027, with product-embedded systems following on 2 August 2028. That is a defined runway rather than a missed deadline.
In this article, we’ll dive into what Article 15 asks for, how adversarial testing produces the evidence, and what a defensible test package looks like — including a table of the work itself, in the order it makes sense to do it.
What Does Article 15 of the EU AI Act Require?
Article 15 of Regulation (EU) 2024/1689 sets four connected requirements for high-risk AI systems: appropriate accuracy, robustness, cybersecurity, and consistent performance across the lifecycle. Each one converts into something you have to measure and record.
Let’s look at what each requirement means in practice:
- Accuracy, declared. The levels of accuracy and the relevant accuracy metrics must be stated in the instructions for use. You choose the metric, but you have to publish it and stand behind it.
- Resilience to errors and faults. The system must be as resilient as possible to errors, faults, or inconsistencies arising within it or in its operating environment — including from interaction with people or other systems. Technical redundancy, backup, and fail-safe arrangements are named options.
- Feedback loop control. Systems that keep learning after deployment must be built so that biased outputs do not contaminate future inputs, with mitigation where feedback loops exist.
- Resistance to attack. The system must be resilient against unauthorized third parties attempting to alter its use, outputs, or performance by exploiting vulnerabilities.
That fourth point is the security requirement, and it is broader than conventional application security. It covers attacks on the model and its data, not only on the infrastructure around them.
What Is Adversarial Testing for AI Systems?
Adversarial testing for AI systems is the practice of deliberately attacking a model and its surrounding application to find inputs, data, or conditions that make it behave in ways it should not. It is how you generate evidence for the resilience requirements in Article 15.
Conventional penetration testing looks for flaws in code, configuration, and access control. Those flaws still matter, and an AI application inherits every one of them. But an AI system adds a second attack surface: the model’s behavior itself.
Here are the categories that belong in a serious AI test scope:
- Prompt injection. Instructions hidden in user input or retrieved content that override the system’s intended behavior.
- Data and model poisoning. Manipulation of training, fine-tuning, or embedding data to plant behavior that only surfaces under specific conditions.
- Model evasion. Crafted inputs that cause a classifier or detector to reach the wrong conclusion while looking normal to a human reviewer.
- Sensitive information disclosure. Extraction of training data, system prompts, or other users’ content through the model’s outputs.
- Improper output handling. Downstream systems trusting model output as safe, turning a text-generation flaw into code execution or data loss.
- Excessive agency. Systems granted more permissions, tools, or autonomy than their failure modes justify.
Those categories track the OWASP GenAI Security Project’s Top 10 for LLM applications, which published its 2026 edition on 4 August 2026. Working to a recognized taxonomy matters here: it makes your test scope explicable to an assessor who was not in the room.
What Is Prompt Injection?
Prompt injection is an attack where crafted input causes an AI system to follow instructions from an attacker instead of its operator. It has been the top-ranked risk in the OWASP Top 10 for LLM applications since the list was created, and it remains unsolved rather than merely difficult.
Direct prompt injection comes from the user typing to the system. Indirect prompt injection is more dangerous: the instructions arrive inside content the system retrieves on its own — a web page, a document, an email, a support ticket.
An AI assistant that reads a customer email and can also send email is one indirect injection away from becoming a data exfiltration tool. That is not a hypothetical class of bug; it is the reason agentic systems need their permissions tested as carefully as their prompts.
Does the EU AI Act Require Penetration Testing?
No. The AI Act does not name penetration testing, red teaming, or any specific test method anywhere in Article 15. It states outcomes and leaves the means to you.
We think it is worth being clear about that, because plenty of marketing suggests otherwise. What Article 15 requires is that the system is resilient against unauthorized third parties exploiting its vulnerabilities, and that you can show it.
Testing is how organizations normally demonstrate that kind of claim. An untested assertion of resilience is not evidence, and a market surveillance authority reviewing a serious incident is going to ask what you did to find the problem before it found you.
Here’s how the requirement and the practice line up:
| Article 15(1) — appropriate accuracy, robustness, cybersecurity | Benchmark testing against declared metrics; performance testing under degraded conditions |
| Article 15(4) — resilience to errors, faults, inconsistencies | Fault injection, edge-case testing, fail-safe and fallback validation |
| Article 15(4) — feedback loop control | Drift monitoring, output sampling, bias testing across retraining cycles |
| Article 15(5) — resistance to unauthorized alteration | Adversarial testing, red team engagement, application security testing |
Article 15 also sits alongside Article 9 risk management and Article 72 post-market monitoring. Testing that happens once before launch satisfies none of them properly — the requirement is lifecycle performance, which implies a cadence.
What Evidence Do You Need to Show Article 15 Compliance?
The evidence you need is a documented, repeatable test program with results, remediation, and retest records — not a single report. Conformity assessment looks for a process that runs, not a snapshot that passed.
Here’s what a defensible package generally contains:
- A declared accuracy specification. The metrics you chose, why you chose them, the thresholds you set, and the results, carried into the instructions for use.
- A scoped threat model. What an attacker would want from this system, which trust boundaries exist, and which of them you tested. Our threat modeling work usually starts here.
- Adversarial test results. Findings mapped to a recognized taxonomy, with severity, reproduction steps, and business impact rather than raw tool output.
- Remediation and retest evidence. What you fixed, what you accepted and why, and proof the fix holds. Accepted risk documented honestly is stronger than a clean report nobody believes.
- Monitoring in production. Logging and detection that would catch the same attack after launch. Log management and managed detection and response turn a point-in-time test into ongoing assurance.
- A retest cadence tied to change. Model updates, fine-tuning runs, and new integrations each reopen the question. Tie testing to release events, not to the calendar alone.
One honest caveat about certification. CyberGlobal delivers readiness, testing, and implementation support. We are not a notified body and cannot issue a conformity certificate or a CE marking decision under the AI Act — those routes run through designated bodies and, where self-assessment applies, through your own declaration.
What Does My Company Actually Have to Do About Article 15?
Article 15 only binds you if you have a high-risk system. So the first thing to do is establish whether you do, and the second is to build the testing program while you still have runway.
Before any of this, the baseline applies to every company: inventory every AI system, classify each one, define your role, check for prohibited practices, establish governance, write a use and approval policy, deliver AI literacy training, and set up vendor governance. Our guide to whether the EU AI Act applies to your company covers that groundwork.
Here’s the Article 15 work itself, in the order it makes sense to do it:
| Action | What it evidences | AI Act reference | Typical effort |
| Confirm the system is in scope. Classify against Annex I and Annex III before spending anything on testing. | Applicability | Art. 6, Annex I, III | 1–3 days |
| Set and declare accuracy metrics. Choose the metric, set the threshold, justify both, carry them into the instructions for use. | Art. 15(1), 15(3) | Art. 15, 13 | 3–8 days |
| Threat model the system. Trust boundaries, attacker goals, and which of them you will test. | Scope justification | Art. 9, 15(5) | 2–5 days |
| Test resilience to errors and faults. Fault injection, edge cases, fallback and fail-safe validation. | Art. 15(4) | Art. 15 | 3–10 days |
| Run adversarial testing. Prompt injection, poisoning, evasion, disclosure, output handling, excessive agency. | Art. 15(5) | Art. 15 | 5–15 days |
| Address feedback loops. Drift monitoring and output sampling for systems that keep learning. | Art. 15(4) | Art. 15, 72 | 3–8 days |
| Remediate and retest. Fix, accept with justification, and prove the fix holds. | Corrective action | Art. 15, 20 | Varies by findings |
| Instrument production monitoring. Detection that would catch the same attack after launch. | Ongoing performance | Art. 12, 19, 72 | 5–15 days |
| Set a retest cadence. Tied to model updates and new integrations, not only to the calendar. | Lifecycle performance | Art. 15(1), 72 | 1–2 days to define |
The sequence matters more than the individual line items. Testing before you have declared accuracy metrics produces findings nobody can act on, because there is no threshold to fail against.
How Much Does Article 15 Testing Cost?
Cost is driven by the system, not by the regulation. A single-purpose classifier with a narrow interface is a different exercise from an agentic system with tool access and a retrieval pipeline.
Here’s what moves the number:
- Attack surface. A model behind an internal API costs less to test than one exposed to the public with document ingestion.
- Agency and integrations. Every tool the system can call, and every system it can write to, extends the scope.
- Retrieval and data flow. Systems that ingest external content need indirect injection testing, which is slower than direct testing.
- Retest volume. Frequent model updates mean a cadence, not a one-off engagement.
- Evidence depth. Testing to find problems is cheaper than testing to produce a defensible conformity package.
We scope against the architecture rather than a price list, and we will tell you when a lighter engagement would answer your question just as well.
Frequently Asked Questions
What does Article 15 of the EU AI Act require?
Article 15 requires high-risk AI systems to achieve an appropriate level of accuracy, robustness, and cybersecurity, and to perform consistently in those respects throughout their lifecycle. Accuracy metrics must be declared in the instructions for use, systems must be resilient to errors and faults, feedback loops must be controlled, and the system must resist unauthorized third parties attempting to alter its use, outputs, or performance.
What is adversarial testing for AI systems?
Adversarial testing is the practice of deliberately attacking a model and its surrounding application to find inputs, data, or conditions that make it behave in ways it should not. It covers prompt injection, data and model poisoning, model evasion, sensitive information disclosure, improper output handling, and excessive agency. It produces the evidence that supports Article 15 resilience claims.
What is prompt injection?
Prompt injection is an attack where crafted input causes an AI system to follow instructions from an attacker instead of its operator. Direct prompt injection comes from user input, while indirect prompt injection arrives inside content the system retrieves itself, such as a web page, document, or email. It has been the top-ranked risk in the OWASP Top 10 for LLM applications since the list was created.
Does the EU AI Act require penetration testing?
No. The AI Act does not name penetration testing or any specific test method in Article 15. It requires that high-risk systems are resilient against unauthorized third parties exploiting their vulnerabilities, and that this can be demonstrated. Testing is how organizations normally evidence that outcome, but the choice of method is yours.
When do Article 15 obligations start to apply?
Requirements for Annex III high-risk AI systems, including Article 15, apply from 2 December 2027. For AI embedded in regulated products such as lifts, toys, and machinery, they apply from 2 August 2028. These dates were extended by the AI Omnibus, which entered into force on 27 July 2026.
What does my company actually have to do about Article 15?
First, confirm you have a high-risk system, because Article 15 only binds you if you do. Then set and declare accuracy metrics, threat model the system, test resilience to errors and faults, run adversarial testing, address feedback loops, remediate and retest, instrument production monitoring, and set a retest cadence tied to model updates. The sequence matters, because testing before you have declared accuracy thresholds produces findings nobody can act on.
Strengthen Your AI Security Testing With CyberGlobal
Article 15 asks you to prove something about a system whose behavior is probabilistic and whose attack surface keeps moving. That is a genuinely hard problem, and anyone who tells you otherwise has not tried it.
But with the right partner by your side, it becomes a scoped program with findings you can act on and evidence you can hand over.
At CyberGlobal, our offensive security teams have spent years running penetration testing and red team engagements against systems where failure was expensive. We have brought that discipline to AI systems rather than starting a new practice from scratch — if you want to see how we think about scoping, our red team engagement readiness guide is a good place to start.
We will tell you what we could not test and why, every time. A findings report that admits its limits is worth more than one that implies coverage it never had.
But behind our advanced technology, there are real people — engineers who will walk your team through every finding until it makes sense.
Reach out to CyberGlobal and let us be your ally against today and tomorrow’s cybersecurity challenges.