Problems of Netfilter/IPtables
In Linux, banning an IP address can be done very easily with netfilter/iptables framework:
If you want to ban a whole IP address block, you can also do it as easily:
However, what if you have 1,000 independent IP addresses with no common CIDR prefix that you want to ban? You would have 1,000 iptables rules! Clearly this does not scale.
$ sudo iptables -A INPUT -s 220.127.116.11 -p TCP -j DROP
$ sudo iptables -A INPUT -s 18.104.22.168 -p TCP -j DROP
. . . .
What are IP Sets?
That is when IP sets come in handy. IP sets are a kernel feature which allows multiple (independent) IP addresses, MAC addresses or even port numbers to be encoded and stored efficiently within bitmap/hash kernel data structures. Once an IP set is created, you can create an iptables rule which matches against the set.
You should immediately see the benefit of using IP sets, which is that you can match against multiple IP addresses in an IP set by using a single iptables rule! You can construct IP sets using combinations of multiple IP addresses and port numbers, and can dynamically update iptables rules with IP sets without any performance impact.
Install IPset Tool on Linux
To create and manage IP sets, you need to use a userspace tool called ipset.
To install ipset on Debian, Ubuntu or Linux Mint:
To install ipset on Fedora or CentOS/RHEL 7:
Ban IP Addresses using IPset Command
Let me walk you through on how to use ipset command using simple examples.
First, let’s create a new IP set named banthis (name can be arbitrary):
The second argument (hash:net) in the above is required, and represents the type of a set being created. There are multiple types of IP sets. An IP set of hash:net type uses a hash to store multiple CIDR blocks. If you want to store individual IP addresses in a set, you can use hash:ip type instead.
Once you have created an IP set, you can check up on the set with:
This shows a list of available IP sets, along with detailed information of each set including set membership. By default, each IP set can contain up to 65536 elements (CIDR blocks in this case). You can increase this limit by appending “maxelem N” option.
Now let’s add IP address blocks to the set:
$ sudo ipset add banthis 22.214.171.124/24
$ sudo ipset add banthis 126.96.36.199/24
$ sudo ipset add banthis 188.8.131.52/24
You will see that the set membership has been changed.
Now it is time to create an iptables rule using this IP set. The key here is to use “-m set –match-set <name>” option.
Let’s create an iptables rule which prevents all those IP blocks in the set from accessing a web server at port 80. This can be achieved by:
If you want, you can save a specific IP set to a file, and then later restore it from the file:
$ sudo ipset destroy banthis
$ sudo ipset restore -f banthis.txt
In the above, I tried removing an existing IP set using destroy option to see if I can restore the IP set.
Automate IP Address Banning
By now you should see how powerful the concept of IP sets is. Still maintaining a up-to-date IP blacklist can be a cumbersome and time-consuming process. In fact, there are free or paid services out there which maintain these IP blacklists for you. As a bonus, let’s see how we can automatically translate available IP blacklists into IP sets.
Let me grab free IP lists from iblocklist.com which publish various IP block lists for free or for a fee. Free versions are available in P2P format.
Here I am going to use an open-source python tool called iblocklist2ipset which converts P2P versions of iblocklist into IP sets.
First, you need to have pip installed (see this guideline to install pip).
Then install iblocklist2ipset as follows.
On some distros like Fedora, you may need to run:
Now go to iblocklist.com, and grab any P2P list URL (e.g., “level1” list).
Then paste the URL into the following command.
–ipset banthis “http://list.iblocklist.com/?list=ydxerpxkpcfqjaybcssw&fileformat=p2p&archiveformat=gz” \
After you run the above command, you will get a file named bandthis.txt created. If you check its content, you will see something like:
create banthis hash:net family inet hashsize 131072 maxelem 237302 add banthis 184.108.40.206/24 add banthis 220.127.116.11/24 add banthis 18.104.22.168/32 add banthis 22.214.171.124/32 add banthis 126.96.36.199/32 add banthis 188.8.131.52/32 add banthis 184.108.40.206/14
You can simply load this file with ipset command:
Now check the automatically created IP set with:
As of this writing, the “level1” block list contains more than 237,000 IP address blocks. You will see that that many IP address blocks have been added to the IP set.
Finally, go ahead and create a single iptables rule to block them all!
In this tutorial, I demonstrated how you can block unwanted IP addresses using a powerful tool called ipset. Combine that with a third-party tool like iblocklist2ipset, and you can easily streamline the process of maintaining your IP block list. For those of you who are curious about the speed improvement of ipset, the figure below shows the benchmark result comparing iptables without and with ipset (credit to daemonkeeper.net).
Tell me how much you like it. 🙂