Skip to content
GitHub Install

All tests / Access

Soft blocks

What this tests. Twenty item pages. Four answer 200 with a "verify you are human" page instead of the item. Count them as missing.

The rule. 4 of these 20 links answer 200 with a page titled Please verify you are human instead of the item. Real sites do this to scrapers they don't like, and a scraper that only checks status codes saves the block page as a record. Here it's a fixture: the same 4 pages are blocks for everyone, every time, and nothing on this site detects bots. Spot them and count them as missing.

For calibrating a detector, here's a clean page and a block page, labelled.

  1. Quillmere Low Notebook
  2. Thistlecombe Compact Pen Set
  3. Vexmoor Classic Pen Set
  4. Hollinsgate Quiet Letter Rack
  5. Hollinsgate Everyday Notebook
  6. Upwold Quiet Notebook
  7. Orrin Lane Quiet Desk Tray
  8. Hollinsgate Woven Notebook
  9. Pellwick Folding Desk Tray
  10. Tarrowby Travel Notebook
  11. Quillmere Studio Letter Rack
  12. Saltgrove Square Planner
  13. Pellwick Compact Notebook
  14. Saltgrove Tall Desk Tray
  15. Brindlecote Brushed Pen Set
  16. Hollinsgate Woven Pen Set
  17. Quillmere Heavy-duty Notebook
  18. Upwold Twin Desk Tray
  19. Fennick & Daw Slim Pen Set
  20. Pellwick Soft Earbuds

Data version 1.0.0. Everything here is made up: products, brands, people, towns. tests.json · robots.txt · Source