{"eid":"gutenberg","name":"Project Gutenberg","domain":"gutenberg.org","resolved_by":"domain","topics":["debrid"],"n_records":1,"best_vorder":1,"review":null,"aliases":["Project Gutenberg","gutenberg.org"],"records":[{"rid":"debrid/project-gutenberg","card":{"rid":"debrid/project-gutenberg","eid":"gutenberg","topic":"debrid","slug":"project-gutenberg","provider":"Project Gutenberg","url":"https://www.gutenberg.org/","domain":"gutenberg.org","category":"public-domain ebook library (bulk mirrorable)","tier":"free","verdict":"qualifies-with-caveat","vorder":1,"grp":"usable","self_hosted":0,"confidence":"documented","binding":"per-ip-shaping","identity_kind":"anonymous","identity_basis":"stated","needs_card":0,"needs_phone":0,"needs_id":0,"needs_institution":0,"country":null,"crypto":null,"interfaces":"rest,rsync,cli","egress_metered":0,"per_object_bytes":null,"retention_days":null,"verified_at":"2026-10-23T03:24:34.450Z","verified_basis":"live","checked_on":"2026-10-23","stale_on":"2026-11-23","n_evidence":5,"usd_month":null,"usd_per_tb_month":null,"summary":"79,657 free eBooks, 'No fees, no registration, completely free'. The published collection metadata gives the real size: main collection 3,443,023 files / 1,0...","duplicate_of":null,"n_unknown":0,"free_unit":"TB"},"record":{"topic":"debrid","slug":"project-gutenberg","provider":"Project Gutenberg","url":"https://www.gutenberg.org/","category":"public-domain ebook library (bulk mirrorable)","tier":"free","free_tier":{"stated":"79,657 free eBooks, 'No fees, no registration, completely free'. The published collection metadata gives the real size: main collection 3,443,023 files / 1,064,736,252,276 bytes, generated collection (EPUB, MOBI, HTML) 2,681,516 files / 1,375,569,708,792 bytes, i.e. 6,124,539 files and 2,440,305,961,068 bytes (2.44 TB), metadata updated 2026-10-19.","unit":"TB","metered_on":"downloaded-bytes","source_url":"https://www.gutenberg.org/dirs/metadata.json","read_on":"2026-10-23","notes":"The 79,657 counts titles; metadata.json counts files. Both were read on the same day and they describe different things, so neither is the other's check."},"per_object_cap":{"stated":"not applicable: one file per book per format. No per-file cap is published; the largest objects in the tree are audio ZIPs, and the file list is the only inventory.","bytes":null,"source_url":"https://www.gutenberg.org/dirs/metadata.json","notes":"A text file measured here was 772,386 bytes (Pride and Prejudice, plain text)."},"retention":{"stated":"permanent: the mirroring guide describes a fixed collection that 'changes daily, as new items are added or updated', with no expiry clock.","days":null,"source_url":"https://www.gutenberg.org/help/mirroring.html","notes":""},"egress":{"terms":"Free and unmetered from the main site and from every mirror. The main site is not a bulk endpoint: 'This website is intended for human users only. Any perceived use of automated tools to access this website will result in a temporary or permanent block of your IP address.' Bulk is approved through rsync mirrors, which the mirroring guide documents.","metered":false,"free_allowance":"no byte allowance published","price_per_tb":null,"source_url":"https://www.gutenberg.org/policy/robot_access.html"},"interface":["rest","rsync","cli"],"interface_detail":"plain HTTPS per file (https://www.gutenberg.org/cache/epub/<id>/pg<id>.txt), an OPDS feed at /ebooks/search.opds/, a JSON OPDS2 feed 'currently available for testing', GUTINDEX.ALL and metadata.json for inventory, and rsync mirrors listed in MIRRORS.ALL.","endpoint":"https://www.gutenberg.org/cache/epub/<id>/pg<id>.txt","tooling":"wget/curl, rsync against any mirror, the OPDS feed for readers. The robot policy's own example is 'wget -w 2 -m -H \"https://www.gutenberg.org/robot/harvest?filetypes[]=html\"' - two seconds between requests, from the main site, for a bounded harvest.","chaining":"The canonical public-domain text corpus to pair with LibriVox audio and Internet Archive scans of the same titles; a 2.44 TB rsync mirror can be filtered to the zip files only, which drops the audio weight.","limits":[{"kind":"per-ip-shaping","value":"automated access from the main site is blocked by IP; 'Such blocks automatically expire after a few days'","source":"https://www.gutenberg.org/policy/robot_access.html"},{"kind":"request-rate","value":"the vendor's own harvest example uses wget -w 2 (two seconds between requests); mirrors are the approved bulk path","source":"https://www.gutenberg.org/policy/robot_access.html"}],"limits_note":"","binding":"per-ip-shaping","binding_note":"At 1 TiB/month the main site blocks the caller long before any byte cap; the mirror network is the only path that carries that volume, and the mirror list is what makes the quota effectively unlimited.","identity_cost":{"kind":"anonymous","detail":"The home page states 'No fees, no registration, completely free'. No account exists to create.","country":null},"reliability_note":"Main-site IP blocks are automatic and last days; the generated collection is rebuilt in batches, so a mirror refresh sees traffic spikes rather than a steady delta.","confidence":"documented","confidence_note":"The collection size is the vendor's own metadata.json, fetched here; one complete file was downloaded and hashed. What would raise it to measured for the bulk path is an rsync of a filtered subset from a mirror with a byte count, which was not run.","evidence":[{"url":"https://www.gutenberg.org/","date":"2026-10-23","proved":"'Project Gutenberg is a library of 79,657 free eBooks' and '100% Free - No fees, no registration'"},{"url":"https://www.gutenberg.org/dirs/metadata.json","date":"2026-10-23","proved":"collection size: main 3,443,023 files / 1,064,736,252,276 bytes, generated 2,681,516 files / 1,375,569,708,792 bytes, metadata_updated 2026-10-19T02:49:46+00:00"},{"url":"https://www.gutenberg.org/policy/robot_access.html","date":"2026-10-23","proved":"'intended for human users only', automatic IP blocks that 'expire after a few days', and the sanctioned wget -w 2 harvest recipe"},{"url":"https://www.gutenberg.org/help/mirroring.html","date":"2026-10-23","proved":"rsync mirroring is offered; 'around 6 million files, over 60 languages'; main collection changes daily"},{"url":"probe:https://www.gutenberg.org/cache/epub/1342/pg1342.txt","date":"2026-10-23","proved":"200, 772,386 bytes, sha256 3f6bb9d6f78e0293b56acd4714dd68cb7d6d1d293402031ce9d5a216bcaf9d75, text/plain, 2,239.5 ms - the file downloaded here is the book"}],"verdict":"qualifies-with-caveat","verified_at":"2026-10-23T03:24:34.450Z","price_per_tb_month":"USD 0","notes":"Caveat is the main site, not the catalogue. The abandoned route is 'scrape gutenberg.org with parallel connections': the robot policy names it as a blockable offence and the mirror path costs nothing.","extras":{"total_bytes":2440305961068,"total_files":6124539,"titles":79657,"metadata_updated":"2026-10-19T02:49:46+00:00","mirror_list":"https://www.gutenberg.org/MIRRORS.ALL","measured_file_sha256":"3f6bb9d6f78e0293b56acd4714dd68cb7d6d1d293402031ce9d5a216bcaf9d75"}},"claims":[{"path":"$.free_tier","kind":"has-free-tier","num":1.0,"unit":null,"per":null,"currency":null,"usd":null,"fx_date":null,"parse":"ok"},{"path":"$.free_tier.stated","kind":"free-quota","num":2.44,"unit":"TB","per":null,"currency":null,"usd":null,"fx_date":null,"parse":"ok"},{"path":"$.limits[0].value","kind":"limit:per-ip-shaping","num":null,"unit":null,"per":null,"currency":null,"usd":null,"fx_date":null,"parse":"unparsed"},{"path":"$.limits[1].value","kind":"limit:request-rate","num":null,"unit":null,"per":null,"currency":null,"usd":null,"fx_date":null,"parse":"unparsed"},{"path":"$.price_per_tb_month","kind":"price","num":0.0,"unit":null,"per":null,"currency":"USD","usd":0.0,"fx_date":null,"parse":"ok"}],"unknown":[],"history":[],"drift":[]}]}