Lieferando.de (Takeaway) has captured 5.7% of restaurant related domain names

Summary

I recently decided to compile a very large list of domain names for the German country code top-level domain .de. I did this with the help of the Common Crawl project which provides a free, open repository of web crawl data and web graphs for the last years.A few command line commands and a small bash shell script was all it took to compile a nice list of roughly 9 million .de domain names.💡If you want to know more about how exactly I compiled the domain name list, just let me know and I might write about it in another blog post.The resulting list is by no means complete and also not up-to-date, but it can be a nice starting point for further analysis and investigations. One of which I am going to write about in this post.Enter the restaurant worldI was interested in domain names that belonged to restaurants and eating places in general. A simple but quite effective way to achieve this was to filter the domain names by German words that indicate such eating places: Restaurant Gasthaus Gasthof Gaststätte Wirtshaus Gastwirtschaft Schänke Speisewirtschaft Speiselokal Speisehaus Speiserestaurant Speisegaststätte Gastlokal Kneipe Pizzaria Since I was running this analysis on my windows desktop, grep was not directly available. So I used the findstr PowerScript command instead to do this filtering:findstr /I "Restaurant Gasthaus Gasthof Gaststaette Wirtshaus ..." huge-domain-list.txt This yielded a still impressive list of about 31.000 domain names related to German restaurants.Who's still alive?Since I knew that the original list is outdated by definition, I had to check which domain names were still active. My first attempt to do this was also with a PowerShell script. This worked in principle and I was somewhat impressed by the capabilities that PowerShell provided, but at the same time the unfamiliar syntax turned me off and most importantly the speed was not great. So I created a small Golang program to get this job done, which worked great due Golang's concurrency f...