User-agent: * Allow: / # The rider-comparison pages take two free-text slugs (?r=&as=), so every pair # of finishers is a distinct URL — an unbounded space over fields of up to # ~57,000 people. Crawlers walk it: 239,449 hits, 20.2% of ALL AI-crawler # traffic to this site, across 8 bots, on pages that are server-rendered # against the database on every request and were never meant to be indexed # (they already canonicalise to the bare /compare). Disallow: /*/compare? Disallow: /*/compare/ Sitemap: https://nicolassitter.com/sitemap.xml