Legal, Ethics & Disclosure for AI Security Testing

Module B0 · Course 2B — Securing & Attacking Harnesses and LLMs

60 minutes · The only module where the anti-patterns carry criminal penalties — plus a layer of law with no pentest analogue

You are attacking AI systems. Every statute that governs attacking a database governs attacking the agent that reaches it — plus an AI-specific layer the deployer cannot waive.

Pillar 0 — Foundations

The inversion from 2A

In 2A the target is infrastructure under ONE owner. In 2B the target is an AI system — and the "asset owner" splits in two.

Course 2A — security harness

Target: networks, apps, contracts. Owner: the deployer. Law: CFAA / CMA / EU 2013-40. The law you already know.

Course 2B — the AI itself

Target: the model, the prompt, the memory. Owner splits: provider (built it) + deployer (runs it). Law: traditional plus an AI-specific layer.

Two legal layers, stacked

Layer	Governs	Key instruments
Traditional	The infrastructure: server, DB, API gateway	CFAA § 1030 · UK CMA · EU 2013/40 + GDPR · NIS2
AI-specific	The model & its outputs: weights, prompt, jailbreak	EU AI Act 2024/1689 · DTSA + EU Trade Secrets · DMCA § 1201 · US EO 14110 / OMB M-24-10

A finding often implicates BOTH layers. A prompt injection reaches infrastructure (CFAA) and manipulates the model (AI Act). A weight read is infrastructure access and trade-secret theft.

B0.1 — Authorization law for AI systems

The statutes, the chain, the contract

The authorization chain has one more link

Model PROVIDER  ──ToS / waiver──▶  DEPLOYER
   (owns weights)                       (runs agent)
        │                                  │
        ▼                                  ▼
   Engagement Sponsor ──▶ RoE/Scope ──▶ Scope File
   (CISO / AI-sec lead)                (machine-checkable)
                                            │
                                            ▼
                                   Harness scope GATE
                                            │
                                            ▼
                                  Individual test action

The provider link is the gap. The deployer can authorize testing of THEIR agent — but cannot authorize violating the provider's terms. This is where most AI red-team legal mistakes happen.

Three contested legal questions

1. Is a prompt injection "access" under the CFAA? Unsettled — no case law as of 2026. A plausible theory for the attacker who injects; weak for an authorized red-teamer. Rule: test only what you own or are authorized to red-team.

2. Are model weights a trade secret? Yes (DTSA § 1836, EU TSD 2016/943) when kept secret. Copying the file IS the harm — no COUNT(*) equivalent. Control: prove reachability with a hash, never the file.

3. Does DMCA § 1201 apply? A model behind an API key/rate-limiter/watermark is behind an access control. Bypassing it is a separate crime from any CFAA question. The triennial security-testing exemption does NOT clearly cover AI. Get a § 1201 waiver or don't run it.

The AI-specific legal layer (no pentest analogue)

Regime	What it does to a red-team
EU AI Act (2024/1689)	A finding may be a legal non-compliance, not just a bug (Art. 5 prohibitions). GPAI systemic-risk providers must red-team (Art. 55).
US EO 14110 / OMB M-24-10	A finding may be a reportable incident. Verify current administration policy.
State bot/deepfake laws	CA AB 2013 / SB 942, TX — touch synthetic-content payloads.

B0.2 — Responsible disclosure of AI vulnerabilities

CVD for AI + the dual-use dilemma

The dual-use dilemma

A successful jailbreak is simultaneously:

A security finding — a safety control failed.
The CVD tradition says: report it, eventually publish.

A misuse recipe — copy-paste-usable, bypasses refusal.
Publishing hands a weapon to non-technical attackers.

The same artifact is both. Traditional security has dual-use problems, but the gap between "research" and "offensive capability" is narrower for a jailbreak — it is literally usable by a non-technical attacker.

Four principles resolve the dilemma

Report to the provider first — always, before anything else.
Publish existence + severity, not the recipe — by default.
Longer embargo — ~180 days for model-level (RLHF is months, not weeks); harness-level stays 45–90.
Decline to publish pure-misuse with no defensive lesson — provider-only disclosure. This is where AI CVD diverges from "publish everything."

Evidence: keep vs. destroy (AI-specific)

Class	Example	Rule
Public	Technique description (no recipe)	Retain indefinitely
Provider-Only	The recipe, system prompt	Engagement + provider tail, NDA
Restricted	PII / weights in retrieved context	Destroy on report
Destroy-on-Report	Working jailbreak prompts	Destroy, never cross-client

Never exfiltrate weights to prove extractability. A hash of a small shard proves reachability without retaining the trade secret. The harness enforces the green path.

B0.3 — AI-specific legal & ethical risks

Five risks, five controls

Five AI-specific risks

Risk	Control
Weight exfiltration	Minimum-proof: path + hash + byte count, never the file
Dual-use publication	4 principles codified in the RoE before testing
Third-party model harm (sweeps degrade shared infra)	Dedicated/preview tier or local model; cap volume
Provider ToS conflict	`provider_authorization` check as a precondition
Test-data contamination	Opt out of training; test on frozen checkpoints

The load-bearing principle

The operator is liable. "The model did it" is not a defense now, and is unlikely to become one. The deployer's authorization does not extend to provider-controlled surfaces. The legal control plane — scope file, provider-authorization gate, evidence classifier — must exist in code before the first technique is pointed at a target.

Everything from B1 onward assumes this control plane exists. B1 builds the threat model; B2–B12 attack and defend within it. The scope file and the gate are the legal layer made engineering.

Lab & what's next

Lab (07): extract provider-ToS clauses, build the JSON scope file separating deployer vs. provider surfaces, write the CVD + dual-use rubric, implement provider_authorization_check().

Next — B1: Threat Model of Agentic Systems. Loop, tools, memory, provider, identity, sandbox, inter-agent edges. B1 takes the scope file from B0 and turns it into the surface map every subsequent module attacks.