What happened
Vercel, which hosts this site, reported 23 million requests in a twelve-hour window on 10 October. The site’s own hourly counter shows when they arrived:
| Hour (UTC), 10 Oct | Class | Requests |
|---|---|---|
| 04:00 | human | 31,075 |
| 05:00 | human | 4,031,187 |
| 06:00 | human | 6,499,921 |
| 07:00 | human | 6,859,065 |
| 08:00 | human | 11,530,030 |
| 09:00 | human | 8,457,348 |
| 10:00 | human | 1,979,673 |
| 11:00 | human | 118,820 |
A normal hour is about 19,000 requests. The counter classifies traffic by user agent, and this traffic identified itself as an ordinary browser, so it was filed under “human”.
About 95% of the requests were served from Vercel’s cache. The cost came from the volume itself and from the remaining 5%, which each triggered a server render and a database query.
Where it came from
The busiest addresses were all in Tencent Cloud’s Singapore region (43.172.x.x and 43.173.x.x). Each one sent about 17,000 requests. I counted around a hundred of them before the list ran off the screen. Each server ran a headless Chrome, which is a real browser without a window, controlled by a script.
I don’t know who operated them. A cloud network shows where the servers were rented, not who rented them. The site’s logs keep full request details only for known AI crawlers, so I can’t say which pages they wanted. The largest set of pages on the site is the race results: 1.36 million finishers across 85 races, each with their own page.
The same pattern appears in the counter on 16 September: thirteen hours and about 9.5 million requests. Nobody noticed at the time.
The site already offers machine access
This site is built to be read by programs. It publishes a JSON feed of all articles, a per-article JSON endpoint, llms.txt files, an MCP server and WebMCP tools in the page. Since the MCP server launched in September, it has recorded 61 tool calls in total (live counts).
The scraper used none of these. It loaded the HTML pages in a browser that presented itself as a person.
One note for whoever ran it: the aggregate race statistics are available through the API and the MCP server. Individual results are only on the pages, on purpose.
Why the number was 23 million
The scraper did not open 23 million pages. Based on the measurements below, it opened roughly 50,000. The rest came from link prefetching.
Next.js, the framework this site uses, prefetches links when they appear on screen. The browser fetches the linked page in the background so that a click loads instantly. The site’s header has dropdown menus (Research, Sports Data, Projects) that are part of every page, including when they are closed. They contain 131 links, and each prefetch took about four requests.
Every page view cost about 500 requests before counting the page’s own content. A browser that opens 50,000 pages, each in a fresh session, therefore produces about 25 million requests.
I measured this on a local production build, with a fresh browser for each page, before and after turning prefetching off in the header and the race tables:
Requests per page view went from about 500 to about 25. Links still work the same way. The next page now loads when it is clicked rather than in advance.
In July I published The Cost of AI Crawling, which found that heavy crawling made this site more reliable. That finding still holds. It did not look at cost per request.
What I changed
- Blocked the cloud networks the traffic came from.
- Added a browser check on content pages for automated clients. Verified crawlers such as Googlebot and the AI search bots pass without it. The APIs, feeds and
llms.txtare not affected. - Turned off link prefetching in the header and the race tables.
- Made the three pages that queried the database on every request cacheable.
- Added an hourly alert. Tested against the 10 October data, it would have sent an email at 06:20 UTC with the busiest IP addresses.
- Added a spending limit that pauses this site only.
On the last point: Vercel has a built-in spending cap, but when it is reached it pauses every project on the account, including sites that had nothing to do with the problem. Vercel also sends a webhook at 50%, 75% and 100% of the cap. A small endpoint that receives it can pause a single project instead.
A last note
Whoever ran this read a lot of the site in a few hours. If they found something in the data that I missed, I would like to hear about it.
Sources: the Vercel dashboard, the site’s hourly request counter, and page-by-page measurements on a local production build. The 23 million is Vercel’s count for a twelve-hour window. The hourly counter’s figures above cover a slightly different window and run higher.