Blocking an ASN (or similar) from my sites

Lobsters Hottest Tools

Summary

A technical guide on blocking entire ASNs and cloud provider IP ranges from websites to mitigate unwanted bot and AI scraper traffic, using iptables and consolidation tools like net-consolidator and cidr-aggregator.

<p><a href="https://lobste.rs/s/bgtvnw/blocking_asn_similar_from_my_sites">Comments</a></p>
Original Article
View Cached Full Text

Cached at: 06/09/26, 02:46 PM

# Blocking an ASN (or similar) from my sites Source: [https://dracos.co.uk/wrote/blocking-an-asn/](https://dracos.co.uk/wrote/blocking-an-asn/) 24th May 2026 I run a[number](https://theatricalia.com/)[of](https://postofficeinquiry.dracos.co.uk/)[websites](https://traintimes.org.uk/), some of which could even be said to be popular\. I want humans to visit these websites, enjoy these websites, make their change of trains at New Street in good time, investigate[miscarriages of justice](https://www.postofficescandal.uk/post/project-brisbane-a-report-the-post-office-is-desperate-to-keep-secret/), or find out the play they watched in London when hitch\-hiking round Europe in the ’80s\. But I find it harder to do this when my server is swamped with artificial traffic from bots, AI clankers, and whatever else nonsense there is nowadays \([Weird Gloop](https://weirdgloop.org/blog/clankers)has a good summary; I don’t have wikis but for other reasons am basically in the same boat\)\. This is especially egregious when they are incredibly poorly written and constantly fetch basically identical pages which may cause issues not just on my own server but with upstream sources\. Clearly the people behind all this simply do not think or care \(and as an aside, this is why I cannot extricate positives from this technology without the negatives alongside, and how it has been/is being introduced\)\. Some you can block by individual IP address, some by user agent, some by location, and sometimes \(the purpose of this post\) you just feel like every hit you get from an entire company is artificial traffic ultimately derived from selfish individuals; reporting the abuse won’t do anything, so you just want to block any traffic from that company\. \(This doesn’t help with residential proxies, of course, but every little helps\.\) So, how do I block any Amazon, or Tencent, or DigitalOcean, or \.\.\., IP address from accessing my site? ## Getting a list ### Cloud providers Google and Amazon publish JSON of their current cloud IP ranges\. So for getting a list of ranges from Google you can use: ``` curl -s -O https://www.gstatic.com/ipranges/cloud.json jq -r '.prefixes | .[] | (.ipv4Prefix // .ipv6Prefix)' cloud.json ``` Or for Amazon: ``` curl -s -O https://ip-ranges.amazonaws.com/ip-ranges.json jq -r '.prefixes | .[].ip_prefix' ip-ranges.json jq -r '.ipv6_prefixes | .[].ipv6_prefix' ip-ranges.json ``` ### ASN lists Other providers don’t publish such lists, but they do have to tell the internet which IP addresses they are responsible for and that they provide routing for them\. This is done using[Autonomous System Numbers](https://en.wikipedia.org/wiki/Autonomous_system_%28Internet%29), which are used in BGP \(Border Gateway Protocol\) routing\. A routing registry, such as[RADb](https://www.radb.net/), lets you look up all the routes given an ASN\. So once you have discovered that, I dunno, AS136907 is Huawei, you can ask RADb for all the ranges: ``` whois -h whois.radb.net -- "-i origin AS136907" | grep 'route:' | cut -d' ' -f 11 ``` ## Checking it twice Now you have some lists, you can then add these to your firewall however you wish\. In my case, I use iptables, and stick most in a total block list, and some in an incoming drop list \(so I can still make outgoing connections\)\. Having done the above, though, reloading my firewall was now pretty slow; fine, but annoying if I wanted to quickly block something else\. A couple of ASNs had an awful lot of IP ranges in them, and I wondered if I could consolidate these at all\. Some searching found me two consolidators,[one in Python](https://github.com/TKCERT/net-consolidator)and[one in Rust](https://github.com/Gowee/cidr-aggregator)\. Both cut down my list of ranges to block substantially, which was great; the Python one was very slow and heavy on memory, and the Rust one was very quick but I didn’t really want to install rust etc on my server\. ### Yak shaving into cross\-compilation I had heard rust could cross\-compile binaries on one platform to run on another, which seemed ideal – some more searching found me[these helpful instructions](https://medium.com/better-programming/cross-compiling-rust-from-mac-to-linux-7fad5a454ab1)which worked perfectly for me \(and boil down to 1\. add x86\_64\-unknown\-linux\-gnu target; 2\. install provided linker; 3\. build\)\. This gave me a`cid\-aggregator`binary that I could pipe my IP ranges to, before adding to my firewall\. 30,000 ranges reduced to 730 odd in the end\. ## Finding out who’s naughty or nice Having such wide blocks isn’t without its own issues, even if it has cut quite a bit of bot traffic\. The two main problems I have had since are: - Let’s Encrypt, which provides the SSL certificates for my domains, uses random IP addresses at multiple providers \(which it won’t reveal\) to perform HTTP validation of domains\. I have at least once blocked my own renewals due to this; I’ll try and switch to DNS validation at some point, but in the meantime I can temporarily drop the blocks while the renewals take place; - Bluesky similarly uses Amazon servers for checking for custom domain handles, so they occasionally break, with a similar solution or switching to DNS verification at some point in future\. ## Reading If you’ve made it this far, I have just finished reading The God Engines by John Scalzi, am currently reading The Tiger and the Wolf by Adrian Tchaikovsky, and have recently bought Soviet Scientific Institutes by Eric Lusito, and a book of time travel romance stories called Someone in Time because it has an Alix Harrow in and I have also recently finished reading The Everlasting by her and it was great\.

Similar Articles

Agent Browser Shield

Product Hunt

Agent Browser Shield is a product that blocks prompt injection attacks and reduces token costs for AI browser agents.

Reuse Less Software

Lobsters Hottest

The article discusses a website using Anubis, a proof-of-work system, to protect against mass scraping by AI companies, highlighting the ongoing battle between content providers and AI data collection.