The Internet Is No Longer a Safe Haven

Summary

A couple of days ago, the small server hosting this website was temporarily knocked out by scraping bots. This wasn’t the first time, nor is it the first time I’m seriously considering employing more aggressive countermeasures such as Anubis (see for example the June 2025 summary post). But every time something like this happens, a portion of the software hobbyist in me dies. We should add this to the list of things AI scrapers destroy next to our environment, the creative enthusiasm of the individuals who made things that are being scraped, and our critical thinking skills. When I tried accessing Brain Baking, I was met with an unusual delay that prompted me to login and see what’s going on. A simple top revealed both Gitea and the Fail2ban server gobbling up almost all CPU resources. Uh oh. Quickly killing Gitea didn’t reduce the work of Fail2ban as the Nginx access logs were being flooded with entries such as: 47.79.216.157 - - [27/Oct/2025:13:05:34 +0100] "GET /wgroeneveld/brainbaking/src/commit/4359ae68930de084df09e1cfa05ffd4520fb7e40/content/links.md?display=source HTTP/1.1" 502 568 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36" 47.79.217.151 - - [27/Oct/2025:13:05:34 +0100] "GET /wgroeneveld/brainbaking/rss/commit/5911666cf0b30236cdc7590abb4e171534faf972/content/museum.md HTTP/1.1" 502 568 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36" 47.79.217.32 - - [27/Oct/2025:13:05:35 +0100] "GET /wgroeneveld/brainbaking/src/commit/7b46fd682f36af81d4852b8ee2ee9970c638cac6/layouts HTTP/1.1" 502 568 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36" 47.79.218.157 - - [27/Oct/2025:13:05:35 +0100] "GET /wgroeneveld/brainbaking/src/commit/4359ae68930de084df09e1cfa05ffd4520fb7e40/content/404.md HTTP/1.1" 502 568 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS...