Data rights

The idea in one line: a Jev-like decision model sits in the riskier legal bucket. The only US appeals ruling on AI training rejected fair use for a non-generative tool that competed with its source. So build on outcome data you are allowed to use, label with models whose terms allow it, and keep provenance on every row.

Rights are a moat and a cliff at the same time. As a moat: investors and customers now ask for proof of data rights next to the accuracy numbers, as the course’s landscape report found, and the prompt behind this course names “rights to the data” as one of two bars to clear. As a cliff: training on pirated books cost Anthropic a $1.5 billion settlement, about $3,000 per work for 482,460 works (Pearl Cohen, 2026-07-30).

The 10x lever is timing. Choosing clean sources and a clean labeller on day one costs almost nothing. Finding out after launch that your labels came from a provider whose terms forbid training a competing model means relabelling and retraining. The course’s stack report calls that “the most expensive hidden cost of all”. Everything else in this lesson is 10% polish next to that one early decision.

Picture borrowing a book from a library. You need permission to walk in and take it off the shelf. You need permission to photocopy it. If a tutor annotated it, using the tutor’s notes is a third matter. And the names written in the margin raise a fourth. Data works the same way:

Each permission can fail on its own. A public bug tracker can pass access and training, and still put you in breach of privacy law if you keep reporters’ email addresses.

Fair use is the US doctrine that allows some unlicensed uses of copyrighted work. Courts weigh four factors: the purpose and character of the use (commercial or not, and whether it is transformative, that is, used for a different purpose from the original), the nature of the work (factual or creative), the amount taken, and the effect on the market for the original (Haynes Boone, 2026-10-02).

Bartz v. Anthropic. On 2025-06-23 Judge Alsup held that training on books was fair use (“exceedingly transformative”), and that scanning purchased print books was fair use. Downloading pirated copies for a central library was not (NBC News, 2025-06-24). The case settled for $1.5 billion plus interest, covering 482,460 works. Judge Araceli Martínez-Olguín gave final approval on 2026-07-20. The release covers only past conduct up to 2025-08-25, not claims about outputs, and Anthropic must destroy its LibGen and PiLiMi files (Pearl Cohen). Fees were cut to about $101.5 million (Authors Alliance, 2026-07-21). The lesson is blunt: how you got the data can lose a case that how you used it would have won.

Kadrey v. Meta. On 2025-06-25 Judge Chhabria granted Meta summary judgment on training as fair use “at least for now”, because the authors brought no real evidence of market harm. He called “market dilution”, the idea that a flood of AI output harms the market for human work, “far more promising” (Authors Alliance, 2025-06-26). A claim about torrent “seeding” stayed alive, and the plaintiffs moved to add a contributory claim on 2025-12-11 (Norton Rose Fulbright, 2026-03).

Thomson Reuters v. Ross is the case that matters most for this course. Ross built a legal search tool trained on Westlaw’s headnotes, the short summaries of legal points that Westlaw’s editors write. Judge Bibas rejected fair use on 2025-02-11. On 2026-09-29 the Third Circuit affirmed (No. 25-2153), the first federal appellate decision on AI-training fair use (Haynes Boone, 2026-10-02; Ballard Spahr, 2026-10-02). The court went factor by factor:

1. Purpose: against Ross. The use was highly commercial and “minimally transformative, at best”.
2. Nature of the work: slightly for Ross, because headnotes are more factual than creative.
3. Amount: against Ross, reversing the district court. Ross took more than it needed.
4. Market effect: against Ross. The court found harm to Westlaw and recognised a derivative market for licensing headnotes as AI training data. It treated that as a real market even where the owner had licensed to no one.

Two lines from the ruling stick. “Unlike necessity, ease is not a justification for copying”: Ross used the headnotes because they were easier to use than the court opinions themselves. And a footnote distinguished systems that generate new expression from Ross’s tool, which only returned existing passages. On that basis the panel set aside Bartz, Kadrey and the Justice Department’s statement of interest in the OpenAI case.

NYT v. OpenAI (the OpenAI copyright multi-district case before Judge Stein). OpenAI’s motion to dismiss the output-infringement claim was denied on 2025-10-27. OpenAI was ordered to produce 20 million output logs on 2026-01-05, with further log sets on 2026-03-09 (Norton Rose Fulbright). Summary-judgment motions were due on 2026-09-04, days after a US statement of interest called training “extraordinarily transformative” and the dilution theory “deeply flawed” (ChatGPT Is Eating the World, 2026-09-04). It was undecided as of 2026-10-28. The US Copyright Office’s report on generative AI training exists as a pre-publication version (2025-05-09), with a final version promised “without any substantive changes” (copyright.gov).

The two generative wins rested on transformation: a chatbot that writes new text serves a different purpose from the books it read. The one appellate ruling concerned a non-generative tool that served the same purpose as its source, and lost. A Jev-like model is non-generative. It returns a judgement, not new expression. The footnote that shielded generative AI from Ross does not shield you.

The course’s dataset report reads the rulings this way. It is the report’s inference, not a holding, so treat it as a map, not a guarantee:

The fourth factor deserves a second look. Ross recognised a market for licensing data as AI training material even where the owner had licensed to no one. For a founder, that means “nobody sells these labels anyway” is a weak defence if the owner plausibly could sell them.

• UK, Getty v. Stability AI ([2025] EWHC 2863 (Ch), 2025-11-04). Getty’s secondary copyright claim failed. A narrow, historic trade-mark infringement was found (judiciary.uk). Getty has permission to appeal on whether a model must store a copy of a work to be an “infringing copy” (Burges Salmon, 2025-12-19).
• Germany, GEMA v. OpenAI (Munich Regional Court, 42 O 14139/24, 2025-11-11). Song lyrics memorised inside a model count as a reproduction, and the TDM exceptions did not cover them. The case is under appeal (MediaLaws, 2026-06-03).
• Germany, Kneschke v. LAION (Hamburg Higher Regional Court, December 2025). LAION’s copies, made to check image descriptions while building a dataset, were covered by Germany’s research and general TDM exceptions. A plain-language reservation in website terms was not machine-readable by 2021 technology, so it was not an effective opt-out. Machine-readability is judged at the time of use (Open Future, 2025-12-18). A further appeal to the Federal Court of Justice was reported as permitted, though only by secondary sources.

The EU has two separate rulebooks you might meet: the AI Act, which regulates models, and the Copyright (DSM) Directive, which governs mining text and data.

General-purpose AI (GPAI) duties. Providers of GPAI models must keep technical documentation, adopt a copyright policy, and publish a summary of their training content. These duties apply from 2025-08-02. The Commission’s enforcement powers start on 2026-08-02. Models placed on the market before 2025-08-02 have until 2027-08-02. Fines reach 3% of global turnover or €15 million (FLI summary, 2025-07-30; Commission FAQ).

Who counts as a GPAI provider. Indicatively, a model trained with more than 10²³ FLOP that can generate language, images or video. Narrow models (transcription, weather, upscaling) are excluded. If you modify someone else’s model, you become its provider only if your modification uses more than a third of the original’s training compute, or a third of 10²³ FLOP when the original’s is unknown. Open-source models keep the copyright-policy and summary duties (FLI summary, a secondary summary of the Commission’s guidelines).

A worked example: are you anywhere near the line? Training compute is roughly 6 × parameters × tokens (the course’s architecture report). Continuing to pre-train a 150M encoder on 20 billion domain tokens costs about 6 × 1.5e8 × 2e10 = 1.8e19 FLOP, a figure from the stack report’s $5,000 plan. That is about 1/5,500 of 10²³, and the model is not generative anyway. A fine-tune of an open model at around 10¹⁹ FLOP is about 3,000 times below a third of 10²³. For a bootstrapper the Act’s GPAI duties almost certainly don’t apply. Your duties come from copyright, contracts and privacy law instead.

The training-content summary, for those who are providers, follows a template (2025-07-24, updated 2026-03-26). It asks for the top 10% of scraped domains (for SMEs, the top 5% or 1,000 domains, whichever is lower), updated every six months (Commission). Compliance so far is thin. Of 24 published summaries checked on 2026-09-25, 21 had the domains section and none named a single site, and DeepSeek’s three left it out (ActuIA, 2026-09-25).

The Digital Omnibus reached provisional agreement on 2026-05-07, with a Parliament vote on 2026-06-16. It delays the Act’s high-risk deadlines to 2027-12-02 (Annex III) and 2028-08-02 (Annex I). Sidley’s summary describes no GPAI changes (Sidley, 2026-06-22). If your decision model is used in one of the Act’s high-risk areas, separate obligations apply to that use. The course’s reports did not analyse which decision products fall there.

The TDM opt-out binds your crawler regardless. Article 4 of the DSM Directive (2019/790) allows text and data mining of lawfully accessible works unless the rightsholder has reserved that right. For online content, German courts require the reservation to be machine-readable (§44b(3) of the German copyright act, applied in Kneschke). Article 3 separately covers research organisations. This applies to your mining whether or not the AI Act applies to your model.

robots.txt is the file at a site’s root that states which crawlers may fetch what. Its standard, RFC 9309 (September 2022), is explicit: “these rules are not a form of access authorization” (RFC 9309). It is a request, not a lock. Requests are multiplying fast. In one year, robots.txt changes made 5%+ of all tokens in the C4 web corpus unusable, and 28%+ of its most actively maintained critical sources. 45% of C4 is restricted by terms-of-service clauses, often inconsistently with robots.txt (Longpre et al., 2024-07-20).

Cloudflare Content Signals (2025-09-24) add an AI-specific line to robots.txt (Cloudflare, 2025-09-24):

User-Agent: *
Content-Signal: search=yes, ai-train=no
Allow: /

search covers building a search index and returning links and short excerpts, but not AI-generated summaries. ai-input covers feeding content into AI models in real time, as in retrieval. ai-train covers training or fine-tuning AI models. Cloudflare’s managed robots.txt runs on over 3.8 million domains and defaults to search=yes, ai-train=no, with no ai-input signal. Cloudflare frames the line as an express reservation under DSM Article 4. For your EU crawling, that makes it the machine-readable opt-out Kneschke talks about.

The standards are fragmenting. The IETF’s AIPREF working group is standardising a vocabulary for AI preferences, attached through robots.txt and HTTP headers, with milestones due 2026-08-31 (IETF datatracker). Open Future reported it stalled, with only two defined terms, while Content Signals, RSL 1.0, TDMRep and others split the space (Open Future). Spawning’s ai.txt is another proposal, which the course’s report did not re-verify.

The practice that follows: snapshot robots.txt, any content-signal line and the terms of service at crawl time, and keep the snapshot with the data. Honour ai-train=no, GPTBot- or CCBot-style disallows, and bot walls. This snippet takes the snapshot with only the standard library. It was tested against a live site in October 2026.

# Snapshot robots.txt and its AI content signals before crawling (stdlib only).
import datetime, hashlib, json, urllib.request, urllib.robotparser

def rights_snapshot(site: str, agent: str = "my-dataset-builder") -> dict:
    """Record what a site's robots.txt said about crawling and AI use, now."""
    url = site.rstrip("/") + "/robots.txt"
    req = urllib.request.Request(url, headers={"User-Agent": agent})
    text = urllib.request.urlopen(req, timeout=30).read().decode("utf-8", "replace")
    rp = urllib.robotparser.RobotFileParser()
    rp.parse(text.splitlines())
    signals = [ln.split(":", 1)[1].strip() for ln in text.splitlines()
               if ln.lower().startswith("content-signal:")]
    return {"url": url, "sha256": hashlib.sha256(text.encode()).hexdigest(),
            "fetched": datetime.datetime.now(datetime.UTC).isoformat(),
            "may_fetch_root": rp.can_fetch(agent, site),
            "content_signals": signals, "robots_txt": text}

snap = rights_snapshot("https://example.org")
print(json.dumps({k: v for k, v in snap.items() if k != "robots_txt"}, indent=2))

It collects every Content-Signal line in the file. A real crawler should also note which user-agent group each line sits in, and store the site’s terms page next to the snapshot.

Open content licences. Creative Commons licences “do not supersede existing limitations and exceptions”. If your training is covered by fair use or a TDM exception, you need not rely on the licence. If you do rely on it, you must follow its conditions (Creative Commons, 2023-08-18). Redistributing a dataset of CC BY-SA text (Stack Exchange, many wikis) carries attribution and share-alike duties. So ship IDs and URLs plus the code to rebuild the dataset, the way The Stack v2 ships Software Heritage IDs.

Code. The Stack v2 kept only permissively licensed or unlicensed files, detected licences with ScanCode, honoured opt-outs, and requires users to follow each file’s licence (The Stack v2 card). Whether GPL code as training data makes a model a derivative work is untested in court, as far as the course’s report could verify. The GitHub Copilot litigation site’s latest dated entry is 2022-11-10 (githubcopilotlitigation.com).

Licence metadata is unreliable. An audit of 1,800+ datasets found licences omitted on 70%+ of popular hosting-site listings, and miscategorised on 50%+ (Longpre et al., 2023-10-25). Check the source, not the hub’s tag.

Base-model licences can surprise you too. The course’s stack report found that the flagship examples of Part 3 ship non-commercial weights. TimesFM-3.0 uses a “TimesFM Non-Commercial License”, and TabPFN-2.5, 3 and 3.5 are licensed for “non-commercial and non-production use”. TimesFM-2.5 (Apache-2.0) and TabPFN v2 (Apache-2.0 plus attribution) are the last permissive versions.

Model-output terms, meaning who may train on what a model writes, are where a decision-model business is most exposed. Your labeller is a model, and its terms follow your labels into your product. As of October 2026:

The course’s two reports read the Gemini terms’ date differently (2026-03-23 and 2026-04-28). The clause is the same in both.

The Qwen3.8 lesson. Qwen’s 2025 releases and Qwen3.5 were Apache-2.0, a permissive licence that restricts copying the weights, not using what the model writes. Then Qwen3.8 moved to a community licence with a carve-out aimed at exactly the business model of a decision API. Nothing about the model’s quality told you. Only the licence file did. Whether a vertical decision API counts as “Model as a Service” (the licence talks of third parties exercising “meaningful control over the inputs”) is unclear without guidance from Qwen. Pin your labeller’s version, and re-read the licence on every upgrade.

Two nuances hold across the table. First, a model’s weight licence governs self-hosting, while a hosted API, even one serving the same weights, has its own platform terms. Second, “competing” is undefined in all these terms, and a general decision API is where it bites hardest.

Privacy, briefly. Issue trackers are full of people. The Bugzilla response the course’s dataset report pulled included real names and email addresses in its cc_detail field. Request only the fields you need, hash user IDs, and scrub free text (sourcing). GDPR applies to EU data subjects whatever copyright says. Its specifics for tracker data were beyond the depth of the course’s reports.

The course’s dataset report distils all of this into ten points for a bootstrapper:

1. Inventory every source. Record the URL or domain, licence, the ToS clause on AI and scraping, robots.txt and content signals at crawl time (a stored snapshot), the date, and the share of rows the source supplies.
2. Prefer clean sources: your own users’ data, with consent; public-domain and government data; permissive licences; project APIs whose terms allow your use; and public outcome records.
3. Never use pirated corpora (Bartz: about $3,000 per work), bypass bot walls, logins or paywalls, or ignore machine-readable opt-outs when you crawl in or for the EU.
4. Don’t copy a competitor’s judgements. Training on labels from a commercial database and then selling the same decision is the Ross pattern. License them, or derive your own labels from outcomes.
5. Label with clean models. For anything you will sell, prefer self-hosted MIT or Apache weights (DeepSeek V4, gpt-oss, Qwen3 2025), or get explicit permission. Record the labeller on every row.
6. Strip personal data at ingestion: emails and names in tracker metadata.
7. Keep provenance per row, so you can honour takedowns and retrain without the removed rows.
8. Redistribute IDs, not text, for third-party content, and publish the code to rebuild.
9. Write the datasheet, with sources, licences and opt-out handling. It doubles as your EU copyright policy if you ever need one.
10. Re-check every year. NYT v. OpenAI, the Getty and GEMA appeals and the LAION appeal can each move the line.

Points 1, 5 and 7 meet in one record per source, stored next to the data:

source_id        bugzilla-fedora
url              https://bugzilla.redhat.com/rest/bug
access           public REST API, identified user agent
robots_snapshot  sha256 of robots.txt + date fetched
content_signal   none found
terms            tracker terms, read on the crawl date
licence          as stated by the source, checked at the source
personal_data    cc_detail dropped; user ids hashed
labeller         self-hosted MIT/Apache model, version pinned
rows_supplied    share of the dataset from this source

• Haynes Boone: Third Circuit rejects fair use for Ross’s copying of Westlaw headnotes (2026-10-02) — the factor-by-factor reasoning in the ruling that matters most for decision models.
• Pearl Cohen: Court approves the $1.5 billion Anthropic settlement (2026-07-30) — what piracy cost, and what the release does and does not cover.
• FLI: Overview of the guidelines for GPAI models (2025-07-30) — who counts as a provider under the EU AI Act, in plain terms.
• Longpre et al., Consent in crisis (2024-07-20) — how fast the web is closing to AI crawlers.
• Cloudflare: Content Signals Policy (2025-09-24) — the most widely deployed AI opt-out signal, and what each term means.
• Qwen Community License 1.0 (Qwen3.8-Flash-Next) — read a model-as-a-service carve-out in the original.