Faster IPSET loading

By | May 19, 2013

IPSETs are a very efficient way to manage a large list of IP addresses for your iptables firewall. Rather than have an individual rule for each and every address or network that needs to be dropped or rejected, with ipsets you can have a single iptables rule that tests an entire list of addresses. The rule would look something like this:

-I INPUT -m set --match-set blacklist src -j DROP

where the -m set tells iptables to look for an ipset with a name given by the --match-set option. In this case, the ipset’s name is blacklist. (Note: this will give you an error if you haven’t already created an ipset by that name). You can create an ipset by scripting something like this:

ipset --create blacklist iphash

After the ipset has been created, the next task is to load the IP addresses from the blacklist into it. Bonekracker has a nice post on the Gentoo forums with information on how to do this from a text file with blacklisted IPs losartan potassium 100mg.

When I tried to use his approach for a 3,000+ entry blacklist, derived from my recent botnet attack, I found that it took a very long time (more than a minute) to load the blacklist into the ipset. A primitive approach to profiling showed that the slowdown was in the following code:

while [ $((--i)) -ge 0 ]; do
ipset --add temp_ipset ${networks[i]}
done

where ipset --add ipset_name ip is the code that adds IP addresses, one at a time, to the ipset. Unfortunately, this is very slow.

A faster approach is to use the ipset restore command. In order to use it, the text file needs to be formatted in a special way by adding add ipset_name in front of each IP address in the list, like so:

add temp_ipset 223.205.23.116
add temp_ipset 223.206.41.73
add temp_ipset 223.207.124.81
add temp_ipset 223.207.179.208

Then you can replace the while loop above with the following code, and voila, a 10x to 100x speedup.

$ipset restore < $ip_list

With this change, loading the ipset is nearly instantaneous and I have no qualms about loading my 3,696 entry bad boy list into an ipset on each bootup, effectively locking the barn door now that the horse is gone.

Hope this helps you make use of ipsets for your firewall.

2 thoughts on “Faster IPSET loading

  1. Josh

    Hi Jeff,

    I am just starting to learn and play with ipset and I was wondering if is there a way to test if all the IP’s from the blacklist have been loaded. I have a script which is supposed to load IPs from China, but I still see in my logs that IPs from China are trying to login on my server. I searched the logged IP’s in my blacklist and they are there, so it should have been blocked.

    Reply
    1. Jeff Davis Post author

      HiJosh,

      You can readily sheck individual addresses:

      sudo ipset test blacklist

      Hope this helps
      Jeff

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *