Course: Course 2B — Securing & Attacking Harnesses and LLMs
Module: B0 — Legal, Ethics, and Disclosure for AI Security Testing
Duration: ~35 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
Welcome to Course 2B. Before a single prompt is injected, before a single weight is read, before the first jailbreak — there is this module. B zero. Legal, Ethics, and Disclosure for AI Security Testing. It is non-optional, and it is the only module in the course where the anti-patterns carry criminal penalties, civil liability, and — unique to AI — a realistic chance of enabling the very harm you were hired to prevent.
[SLIDE 2 — The inversion from 2A]
If you came from Course 2A, you had an S zero zero on legal and rules of engagement. That module covered the law of using a security harness against traditional targets — networks, apps, contracts. This module inverts the lens. In 2A the target is infrastructure under one owner. In 2B the target is an AI system, and that changes the law in two ways.
First, the traditional computer-crime statutes still apply, but they stretch in non-obvious directions. A prompt injection that makes an agent read a file it was not supposed to read is, arguably, a CFAA event — except the access is being performed by the model, on behalf of an input you provided.
Second — and this is the new part — a layer of AI-specific law now exists that has no pentest analogue. The EU AI Act. US executive orders and OMB memoranda. DMCA anti-circumvention. State bot and deepfake laws. Traditional pentesters never had to learn this. You do.
[SLIDE 3 — Two legal layers, stacked]
Here is the mental model. There are two legal layers, stacked. The traditional layer governs the infrastructure around the model — the server, the database, the API gateway. This is the law from 2A: CFAA, UK Computer Misuse Act, EU Directive 2013-40 plus GDPR. You know it.
The AI-specific layer governs the model and its outputs — the weights, the system prompt, the jailbreak. This is the EU AI Act, the Defend Trade Secrets Act, DMCA section 1201, the US executive order and OMB rules. A finding often implicates both layers. A prompt injection reaches infrastructure and manipulates the model. A weight read is infrastructure access and trade-secret theft at the same time.
[SLIDE 4 — B0.1 Authorization law]
Sub-section one. Authorization law for AI systems. The statutes, the authorization chain, and the engagement contract.
[SLIDE 5 — The authorization chain has one more link]
Authorization is a chain — from asset owner down to the specific test action. For an AI system, the chain has one more link than 2A's, because the asset owner splits in two. The provider — OpenAI, Anthropic, whoever built the model — owns the weights and sets the terms of service. The deployer — your client — runs the agent and owns the data. They are often different entities.
Here is the load-bearing point. The deployer can authorize you to test their agent. They cannot authorize you to violate the provider's terms. That gap — the provider link in the chain — is where most AI red-team legal mistakes happen. A client says "yes, jailbreak our agent." The agent calls an OpenAI model. OpenAI's acceptable use policy prohibits jailbreaking. The client cannot waive OpenAI's terms. If you proceed on the deployer's authorization alone, you have a gap in your chain.
[SLIDE 6 — Three contested legal questions]
Three legal questions that traditional pentesting never had to answer.
One. Is a prompt injection access under the CFAA? Honestly, unsettled. No case law as of 2026. There is a plausible CFAA theory for the attacker who injects the payload. There is a weak theory for an authorized red-teamer testing their own employer. The practical rule: test only systems you own or are explicitly authorized to red-team. The authorization chain is your protection, not the novelty of the legal theory.
Two. Are model weights a trade secret? Yes, when the provider has taken reasonable steps to keep them secret. The Defend Trade Secrets Act and the EU Trade Secrets Directive apply. And here is the asymmetry that matters: unlike a SQL injection, where you can run COUNT-star to prove the vuln without taking data, copying the weight file IS the harm. The artifact itself is the protected asset. So the control is minimum-proof discipline: prove the path — you can reach N bytes — capture a hash, never save the file.
Three. Does DMCA section 1201 apply? Section 1201 makes circumventing a technological access control a crime, independent of any CFAA question. A model behind an API key, a rate limiter, a watermark, or an output filter is behind such a control. Bypassing it to reach the model may be a 1201 violation. The Library of Congress issues triennial exemptions. There is a narrow security-testing exemption, but it does not clearly cover AI models. So the rule: if your test bypasses a model access control you do not own, get an explicit written waiver, or do not run it.
[SLIDE 7 — The AI-specific legal layer]
The AI-specific layer. The EU AI Act, Regulation 2024-1689, is the world's first comprehensive AI law. It prohibits certain uses outright — article five — and imposes obligations on general-purpose AI providers. Here is the part that changes red-teaming: if your test reveals that a deployed system violates an article-five prohibition, you have uncovered a legal non-compliance, not just a security bug. The reporting obligations differ. For systemic-risk GPAI providers, article fifty-five imposes red-teaming duties. Your client may be contracting you to satisfy their own AI Act obligation.
US executive order 14110 and OMB memorandum M-24-10 impose AI incident-reporting duties on federal agencies and their vendors. A finding from your red-team may itself be a reportable event. Verify current administration policy before relying on specific provisions.
[SLIDE 8 — B0.2 Responsible disclosure]
Sub-section two. Responsible disclosure of AI vulnerabilities. CVD for AI, and the dual-use dilemma.
[SLIDE 9 — The dual-use dilemma]
This is the ethical problem that has no analogue in 2A. A successful jailbreak is simultaneously a security finding — a safety control failed, the CVD tradition says report and eventually publish — and a misuse recipe — a copy-paste-usable prompt that bypasses refusal training. The same artifact is both. Traditional security has dual-use problems. But the gap between security research and offensive capability is narrower for a jailbreak, because a jailbreak prompt is literally usable by a non-technical attacker. A buffer overflow exploit requires skill to weaponize. A jailbreak prompt does not.
[SLIDE 10 — Four principles resolve the dilemma]
Four principles resolve the tension. One: report to the provider first, always, before anything else. Two: publish existence and severity, not the recipe, by default. Three: time-bound the embargo, but make it longer than ninety days. The ninety-day Project Zero default is calibrated to software patches measured in weeks. AI mitigations — retraining, RLHF rounds — are measured in months. One hundred eighty days is more realistic for a model-level finding. Four — and this is where AI diverges from the traditional norm: decline to publish pure-misuse content with no defensive lesson. If a finding is "here is how to make the model produce disallowed content" and there is no new technique, no architectural insight, the responsible choice is often provider-only disclosure.
[SLIDE 11 — Evidence keep vs destroy]
Evidence. Same structure as 2A, with AI-specific classes. You keep the model version, the exact prompt, the success rate over N attempts. You destroy personal data in retrieved context, model weights and system prompts, and working jailbreak prompts beyond the engagement. The retention policy has four classes: Public, Provider-Only, Restricted, and Destroy-on-Report. And never — never — exfiltrate weights to prove extractability. A hash of a small shard proves reachability without retaining the trade secret.
[SLIDE 12 — B0.3 AI-specific risks]
Sub-section three. Five AI-specific legal and ethical risks, and the control for each.
[SLIDE 13 — Five risks]
Risk one: weight exfiltration. Control: minimum-proof discipline enforced in the harness. Risk two: dual-use publication. Control: the four principles codified in the engagement rules before testing begins. Risk three: third-party model harm. A red-team that fuzzes a shared production endpoint degrades the service for every other tenant. Control: test against a dedicated or preview tier, or a local model; cap volume. Risk four: provider terms-of-service conflict. Control: a provider-authorization check as a precondition. Risk five: test-data contamination — your test traffic becomes training data. Control: opt out of training; test on frozen checkpoints.
[SLIDE 14 — The load-bearing principle]
The load-bearing principle, restated. The operator is liable. "The model did it" is not a defense now, and is unlikely to become one. The deployer's authorization does not extend to provider-controlled surfaces. The legal control plane — the scope file, the provider-authorization gate, the evidence classifier — must exist in code before the first technique is pointed at a target. Everything from B1 onward assumes this control plane exists. B1 builds the threat model. B2 through B12 attack and defend within it. The scope file and the gate are the legal layer made engineering.
[SLIDE 15 — Lab and what's next]
The lab has you extract real provider terms-of-service clauses, build the JSON scope file separating deployer from provider surfaces, write the coordinated-disclosure timeline and the dual-use rubric, and implement the provider-authorization-check function that the harness runs before every provider-surface action. Next: B1, the threat model of agentic systems. The loop, the tools, the memory, the provider, the identity, the sandbox, the inter-agent edges. B1 takes the scope file from this module and turns it into the surface map that every subsequent module attacks. Let's build it.
# Teaching Script — Module B0: Legal, Ethics, and Disclosure for AI Security Testing **Course**: Course 2B — Securing & Attacking Harnesses and LLMs **Module**: B0 — Legal, Ethics, and Disclosure for AI Security Testing **Duration**: ~35 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] Welcome to Course 2B. Before a single prompt is injected, before a single weight is read, before the first jailbreak — there is this module. B zero. Legal, Ethics, and Disclosure for AI Security Testing. It is non-optional, and it is the only module in the course where the anti-patterns carry criminal penalties, civil liability, and — unique to AI — a realistic chance of enabling the very harm you were hired to prevent. [SLIDE 2 — The inversion from 2A] If you came from Course 2A, you had an S zero zero on legal and rules of engagement. That module covered the law of using a security harness against traditional targets — networks, apps, contracts. This module inverts the lens. In 2A the target is infrastructure under one owner. In 2B the target is an AI system, and that changes the law in two ways. First, the traditional computer-crime statutes still apply, but they stretch in non-obvious directions. A prompt injection that makes an agent read a file it was not supposed to read is, arguably, a CFAA event — except the access is being performed by the model, on behalf of an input you provided. Second — and this is the new part — a layer of AI-specific law now exists that has no pentest analogue. The EU AI Act. US executive orders and OMB memoranda. DMCA anti-circumvention. State bot and deepfake laws. Traditional pentesters never had to learn this. You do. [SLIDE 3 — Two legal layers, stacked] Here is the mental model. There are two legal layers, stacked. The traditional layer governs the infrastructure around the model — the server, the database, the API gateway. This is the law from 2A: CFAA, UK Computer Misuse Act, EU Directive 2013-40 plus GDPR. You know it. The AI-specific layer governs the model and its outputs — the weights, the system prompt, the jailbreak. This is the EU AI Act, the Defend Trade Secrets Act, DMCA section 1201, the US executive order and OMB rules. A finding often implicates both layers. A prompt injection reaches infrastructure and manipulates the model. A weight read is infrastructure access and trade-secret theft at the same time. [SLIDE 4 — B0.1 Authorization law] Sub-section one. Authorization law for AI systems. The statutes, the authorization chain, and the engagement contract. [SLIDE 5 — The authorization chain has one more link] Authorization is a chain — from asset owner down to the specific test action. For an AI system, the chain has one more link than 2A's, because the asset owner splits in two. The provider — OpenAI, Anthropic, whoever built the model — owns the weights and sets the terms of service. The deployer — your client — runs the agent and owns the data. They are often different entities. Here is the load-bearing point. The deployer can authorize you to test their agent. They cannot authorize you to violate the provider's terms. That gap — the provider link in the chain — is where most AI red-team legal mistakes happen. A client says "yes, jailbreak our agent." The agent calls an OpenAI model. OpenAI's acceptable use policy prohibits jailbreaking. The client cannot waive OpenAI's terms. If you proceed on the deployer's authorization alone, you have a gap in your chain. [SLIDE 6 — Three contested legal questions] Three legal questions that traditional pentesting never had to answer. One. Is a prompt injection access under the CFAA? Honestly, unsettled. No case law as of 2026. There is a plausible CFAA theory for the attacker who injects the payload. There is a weak theory for an authorized red-teamer testing their own employer. The practical rule: test only systems you own or are explicitly authorized to red-team. The authorization chain is your protection, not the novelty of the legal theory. Two. Are model weights a trade secret? Yes, when the provider has taken reasonable steps to keep them secret. The Defend Trade Secrets Act and the EU Trade Secrets Directive apply. And here is the asymmetry that matters: unlike a SQL injection, where you can run COUNT-star to prove the vuln without taking data, copying the weight file IS the harm. The artifact itself is the protected asset. So the control is minimum-proof discipline: prove the path — you can reach N bytes — capture a hash, never save the file. Three. Does DMCA section 1201 apply? Section 1201 makes circumventing a technological access control a crime, independent of any CFAA question. A model behind an API key, a rate limiter, a watermark, or an output filter is behind such a control. Bypassing it to reach the model may be a 1201 violation. The Library of Congress issues triennial exemptions. There is a narrow security-testing exemption, but it does not clearly cover AI models. So the rule: if your test bypasses a model access control you do not own, get an explicit written waiver, or do not run it. [SLIDE 7 — The AI-specific legal layer] The AI-specific layer. The EU AI Act, Regulation 2024-1689, is the world's first comprehensive AI law. It prohibits certain uses outright — article five — and imposes obligations on general-purpose AI providers. Here is the part that changes red-teaming: if your test reveals that a deployed system violates an article-five prohibition, you have uncovered a legal non-compliance, not just a security bug. The reporting obligations differ. For systemic-risk GPAI providers, article fifty-five imposes red-teaming duties. Your client may be contracting you to satisfy their own AI Act obligation. US executive order 14110 and OMB memorandum M-24-10 impose AI incident-reporting duties on federal agencies and their vendors. A finding from your red-team may itself be a reportable event. Verify current administration policy before relying on specific provisions. [SLIDE 8 — B0.2 Responsible disclosure] Sub-section two. Responsible disclosure of AI vulnerabilities. CVD for AI, and the dual-use dilemma. [SLIDE 9 — The dual-use dilemma] This is the ethical problem that has no analogue in 2A. A successful jailbreak is simultaneously a security finding — a safety control failed, the CVD tradition says report and eventually publish — and a misuse recipe — a copy-paste-usable prompt that bypasses refusal training. The same artifact is both. Traditional security has dual-use problems. But the gap between security research and offensive capability is narrower for a jailbreak, because a jailbreak prompt is literally usable by a non-technical attacker. A buffer overflow exploit requires skill to weaponize. A jailbreak prompt does not. [SLIDE 10 — Four principles resolve the dilemma] Four principles resolve the tension. One: report to the provider first, always, before anything else. Two: publish existence and severity, not the recipe, by default. Three: time-bound the embargo, but make it longer than ninety days. The ninety-day Project Zero default is calibrated to software patches measured in weeks. AI mitigations — retraining, RLHF rounds — are measured in months. One hundred eighty days is more realistic for a model-level finding. Four — and this is where AI diverges from the traditional norm: decline to publish pure-misuse content with no defensive lesson. If a finding is "here is how to make the model produce disallowed content" and there is no new technique, no architectural insight, the responsible choice is often provider-only disclosure. [SLIDE 11 — Evidence keep vs destroy] Evidence. Same structure as 2A, with AI-specific classes. You keep the model version, the exact prompt, the success rate over N attempts. You destroy personal data in retrieved context, model weights and system prompts, and working jailbreak prompts beyond the engagement. The retention policy has four classes: Public, Provider-Only, Restricted, and Destroy-on-Report. And never — never — exfiltrate weights to prove extractability. A hash of a small shard proves reachability without retaining the trade secret. [SLIDE 12 — B0.3 AI-specific risks] Sub-section three. Five AI-specific legal and ethical risks, and the control for each. [SLIDE 13 — Five risks] Risk one: weight exfiltration. Control: minimum-proof discipline enforced in the harness. Risk two: dual-use publication. Control: the four principles codified in the engagement rules before testing begins. Risk three: third-party model harm. A red-team that fuzzes a shared production endpoint degrades the service for every other tenant. Control: test against a dedicated or preview tier, or a local model; cap volume. Risk four: provider terms-of-service conflict. Control: a provider-authorization check as a precondition. Risk five: test-data contamination — your test traffic becomes training data. Control: opt out of training; test on frozen checkpoints. [SLIDE 14 — The load-bearing principle] The load-bearing principle, restated. The operator is liable. "The model did it" is not a defense now, and is unlikely to become one. The deployer's authorization does not extend to provider-controlled surfaces. The legal control plane — the scope file, the provider-authorization gate, the evidence classifier — must exist in code before the first technique is pointed at a target. Everything from B1 onward assumes this control plane exists. B1 builds the threat model. B2 through B12 attack and defend within it. The scope file and the gate are the legal layer made engineering. [SLIDE 15 — Lab and what's next] The lab has you extract real provider terms-of-service clauses, build the JSON scope file separating deployer from provider surfaces, write the coordinated-disclosure timeline and the dual-use rubric, and implement the provider-authorization-check function that the harness runs before every provider-surface action. Next: B1, the threat model of agentic systems. The loop, the tools, the memory, the provider, the identity, the sandbox, the inter-agent edges. B1 takes the scope file from this module and turns it into the surface map that every subsequent module attacks. Let's build it.