Skip to content

All tests / Messy content

The final boss

What this tests. A thousand pages in one go: 900 scraps of broken markup, 46 real pages, 44 in the wrong or a lying encoding, and ten giants of 13 to 28 MB. Parse every one without falling over, and count what you built.

The rule. Clean pages one at a time tell you how fast a parser is. They don't tell you what happens when a crawl comes back with everything at once. Feed all 1,000 pages to your parser as bytes, in order, and it should finish every one: no crash, no hang, no empty document. Then check what it built. The answer key gives each page's element count, its encoding and its title, and the SHA-256 of its bytes so you know you have the right ones.

What's in it

265.1 MB from seed 20261119, the same bytes on every machine. By count it's mostly scraps; by size it's mostly the giants, and that's where memory goes.

CategoryPagesMBWhat's there
Fragments9001.4A few hundred bytes each: unclosed and misnested tags, stray </table>s, NUL bytes, &amp with no semicolon, &#xD800;, tags nested 5,000 deep, a comment or <script> left open at the end.
Real pages4633.0Twelve pages as their sites served them (Wikipedia, MDN, BBC News, Hacker News and others), from fixed Internet Archive snapshots.
Encodings4436.6Real pages re-encoded, four of each: Shift_JIS; windows-1251; GBK; EUC-KR; UTF-16LE, BOM; UTF-16BE, BOM; meta says UTF-8, bytes windows-1251; UTF-8 BOM, meta says windows-1251; EUC-KR from the transport, meta says ISO-8859-1; UTF-8, undeclared; UTF-8 with invalid bytes.
Giants10194.213 to 28 MB each: real pages run together; one huge table; one text node of references.

Getting the pages

You build them yourself, and there are two reasons this site doesn't just hand them over. Ninety-four of them are other people's pages, and the two biggest giants, 26 and 28 MB, are past the 25 MB a file can be here. So the script fetches the real pages from the Internet Archive, builds the rest from the seed with Parsec's generator (pinned at 4d6fe82), and checks every page against /messy/final-boss/sha256sums.txt. It needs git, curl, a Rust toolchain, CMake, and C and C++20 compilers:

curl -fsSLO https://test.netweir.com/messy/final-boss/build.sh
bash build.sh            # pages in ./final-boss/0000.html to 0999.html

Page N is built from the seed and N alone, so parsec-html-bench boss-page N gives you any single one. The fingerprint of the whole batch is de2480edafa2baa9.

How to run it

Give your parser each page's bytes, not text you decoded first: finding the encoding is half the test. One page has a charset that only the HTTP header knows; the key marks it in http_charset, so pass it the way your parser takes a Content-Type. Time each page from bytes to a built document, and run each parser in a process of its own so its peak memory is its own.

Netweir's bench/boss.py does exactly that for Netweir, selectolax, lxml, Beautiful Soup, parsel and html5lib, and Parsec's bench does it for Parsec, Lexbor and html5ever.

Data version 1.0.0. Everything here is made up: products, brands, people, towns. tests.json · robots.txt