iptables
iptables
EC2 Menu
EC2 Menu
Use iptables to block ip addresses or ports
Blocking IP addresses with ipset and iptables
This gives ability to blacklisted IP addresses, removing lists of “deny from” commands in the .htacess file. The tables are for single addresses, not CIDR ranges. An EC2 instance Security Groups has limits on how many IP addresses you can block, and are difficult to manage and have performance issues.
The ipset/iptables system is at kernel level, so it is fast.
Blocking IPV4 Addresses
This makes use of github.com/kravietz/blacklist-scripts
Make a snapshot backup of your instance volume first.
For Linux 2023, install these packages: (curl should already be installed)
dnf -y install ipset iptables dnf -y install curl
Place this script with root ownership, ec2-user group, chmod 775 in /home/ec2-user:
Where you see URLS=”$URLS https://YOURDOMAIN/block.txt” you can uncomment and use your own domain blacklist if desired.
cd /home/ec2-user
vi blacklist.sh
#!/bin/sh
# IP blacklisting script for Linux servers
# Pawel Krawczyk 2014-2015
# documentation https://github.com/kravietz/blacklist-scripts
# iptables logging limit
LIMIT="10/minute"
# try to load config file
# it should contain one blacklist URL per line
config_file="/etc/ip-blacklist.conf"
if [ -f "${config_file}" ]; then
source ${config_file}
else
# if no config file is available, load default set of blacklists
# URLs for further blocklists are appended using the classical
# shell syntax: "$URLS new_url"
# Emerging Threats lists offensive IPs such as botnet command servers
URLS="http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt"
# Blocklist.de collects reports from fail2ban probes, listing password brute-forces, scanners and other offenders
URLS="$URLS https://www.blocklist.de/downloads/export-ips_all.txt"
# YOUR OWN BLACKLIST (uncomment as needed) where block.txt is your own list, but not with CIDR ranges - you can keep those as needed in /var/www/html/.htaccess
# URLS="$URLS https://YOURDOMAIN/block.txt"
# badips.com, from score 2 up
# URLS="$URLS http://www.badips.com/get/list/ssh/2"
# iblocklist.com is also supported
# URLS="$URLS http://list.iblocklist.com/?list=srzondksmjuwsvmgdbhi&fileformat=p2p&archiveformat=gz&username=USERNAMEx$&pin=PIN"
fi
link_set () {
if [ "$3" = "log" ]; then
iptables -A "$1" -m set --match-set "$2" src,dst -m limit --limit "$LIMIT" -j LOG --log-prefix "BLOCK $2 "
fi
iptables -A "$1" -m set --match-set "$2" src -j DROP
iptables -A "$1" -m set --match-set "$2" dst -j DROP
}
# This is how it will look like on the server
# Chain blocklists (2 references)
# pkts bytes target prot opt in out source destination
# 0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 match-set manual-blacklist src,dst limit: avg 10/min burst 5 LOG flags 0 level 4 prefix "BLOCK manual-blacklist "
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set manual-blacklist src,dst
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set rules.emergingthreats src
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set rules.emergingthreats dst
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set www.blocklist.de src
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set www.blocklist.de dst
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set www.badips.com src
# 0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 match-set www.badips.com dst
blocklist_chain_name=blocklists
# check for dependencies - ipset and curl
if [ -z "$(which ipset 2>/dev/null)" ]; then
echo "Cannot find ipset"
echo "Run "apt-get install ipset" (Debian/Ubuntu) or "yum install ipset" (RedHat/CentOS/Fedora) or "opkg install ipset" (OpenWRT/LEDE)"
exit 1
fi
if [ -z "$(which curl 2>/dev/null)" ]; then
echo "Cannot find curl"
echo "Run "apt-get install curl" (Debian/Ubuntu) or "yum install curl" (RedHat/CentOS/Fedora) or "opkg install curl" (OpenWRT/LEDE)"
exit 1
fi
# check if we are on OpenWRT
if [ "$(which uci 2>/dev/null)" ]; then
# we're on OpenWRT
wan_iface=pppoe-wan
IN_OPT="-i $wan_iface"
INPUT=input_rule
FORWARD=forwarding_rule
COMPRESS_OPT=""
else
COMPRESS_OPT="--compressed"
INPUT=INPUT
FORWARD=FORWARD
fi
# create main blocklists chain
if ! iptables -nL | grep -q "Chain ${blocklist_chain_name}"; then
iptables -N ${blocklist_chain_name}
fi
# inject references to blocklist in the beginning of input and forward chains
if ! iptables -nL ${INPUT} | grep -q ${blocklist_chain_name}; then
iptables -I ${INPUT} 1 ${IN_OPT} -j ${blocklist_chain_name}
fi
if ! iptables -nL ${FORWARD} | grep -q ${blocklist_chain_name}; then
iptables -I ${FORWARD} 1 ${IN_OPT} -j ${blocklist_chain_name}
fi
# flush the chain referencing blacklists, they will be restored in a second
iptables -F ${blocklist_chain_name}
# create the "manual" blacklist set
# this can be populated manually using ipset command:
# ipset add manual-blacklist a.b.c.d
set_name="manual-blacklist"
if ! ipset list | grep -q "Name: ${set_name}"; then
ipset create "${set_name}" hash:net
fi
link_set "${blocklist_chain_name}" "${set_name}" "$1"
# download and process the dynamic blacklists
for url in $URLS
do
# initialize temp files
unsorted_blocklist=$(mktemp)
sorted_blocklist=$(mktemp)
new_set_file=$(mktemp)
headers=$(mktemp)
# download the blocklist
set_name=$(echo "$url" | awk -F/ '{print substr($3,0,21);}') # set name is derived from source URL hostname
curl -L -v -s ${COMPRESS_OPT} -k "$url" >"${unsorted_blocklist}" 2>"${headers}"
# this is required for blocklist.de that sends compressed content regardless of asked or not
if [ -z "$COMPRESS_OPT" ]; then
if grep -qi 'content-encoding: gzip' "${headers}"; then
mv "${unsorted_blocklist}" "${unsorted_blocklist}.gz"
gzip -d "${unsorted_blocklist}.gz"
fi
fi
# autodetect iblocklist.com format as it needs additional conversion
if echo "${url}" | grep -q 'iblocklist.com'; then
if [ -f /etc/range2cidr.awk ]; then
mv "${unsorted_blocklist}" "${unsorted_blocklist}.gz"
gzip -d "${unsorted_blocklist}.gz"
awk_tmp=$(mktemp)
awk -f /etc/range2cidr.awk <"${unsorted_blocklist}" >"${awk_tmp}"
mv "${awk_tmp}" "${unsorted_blocklist}"
else
echo "/etc/range2cidr.awk script not found, cannot process ${unsorted_blocklist}, skipping"
continue
fi
fi
sort -u <"${unsorted_blocklist}" | egrep "^[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}(/[0-9]{1,2})?$" >"${sorted_blocklist}"
# calculate performance parameters for the new set
if [ "${RANDOM}" ]; then
# bash
tmp_set_name="tmp_${RANDOM}"
else
# non-bash
tmp_set_name="tmp_$$"
fi
new_list_size=$(wc -l "${sorted_blocklist}" | awk '{print $1;}' )
hash_size=$(expr $new_list_size / 2)
if ! ipset -q list ${set_name} >/dev/null ; then
ipset create ${set_name} hash:net family inet
fi
# start writing new set file
echo "create ${tmp_set_name} hash:net family inet hashsize ${hash_size} maxelem ${new_list_size}" >>"${new_set_file}"
# convert list of IPs to ipset statements
while read line; do
echo "add ${tmp_set_name} ${line}" >>"${new_set_file}"
done <"$sorted_blocklist" # replace old set with the new, temp one - this guarantees an atomic update echo "swap ${tmp_set_name} ${set_name}" >>"${new_set_file}"
# clear old set (now under temp name)
echo "destroy ${tmp_set_name}" >>"${new_set_file}"
# actually execute the set update
ipset -! -q restore < "${new_set_file}"
link_set "${blocklist_chain_name}" "${set_name}" "$1"
# clean up temp files
rm "${unsorted_blocklist}" "${sorted_blocklist}" "${new_set_file}" "${headers}"
done
exit
[save and exit, or simply copy the. blacklist.sh from github]
I had to comment out the line below as it no longer functions:
www.badips.com/get/list/ssh/2 as it is no longer online.
Now execute these commands:
cd /home/ec2-user chmod 775 blacklist.sh chown root blacklist.sh chgrp ec2-user blackllist.sh sh -x ./blacklist.sh
When this completes, assuming it does not exit in error or freeze, which would mean some problem analysis, you can view the results, and at various times you can look at how many IP addresses were blocked:
ipset list iptables -L -vn
To remove and redo the lists, simply sync;reboot your instance and run the script again. This will ensure all IP’s are dropped and then reconfigured. (A crontab shell script could automate.)
This is mainly here for reference, when I was working with Dovecot. It may come in handy, and can be used for any external attempts to use a port nunber.
We may also add some scripting to deal with attempts to connect to postfix. The idea is to examine /var/log/mail.log to see what kinds of patterns we can search for, then add those patterns to the script.
touch /home/ec2-user/email-drop.txt
touch /home/ec2-user/email-drop.dat
chmod 777 /home/ec2-user/email-drop.txt /home/ec2-user/email-drop.dat
chgrp ec2-user /home/ec2-user/email-drop.txt /home/ec2-user/email-drop.dat
[These data files enable an ongoing collection of IP addresses to block]
vi /home/ec2-user/email-abuse.sh
#!/bin/sh
echo UNKNOWN EMAIL CONNECTION ATTEMPTS
echo ""
cd /var/log
grep unknown mail.log|awk '{print $NF}'|grep -v YOUR_EC2_IP_ADDRESS|grep -v YOUR_OWN_STATIC_IP_ADDRESS|grep unknown|sort -u|awk -F[ '{print $2}'|awk -F] '{print $1}'
echo ""
grep unknown /var/log/mail.log|awk -F[ '{print $3}'|awk -F] '{print $1}'
grep "connect from" mail.log|grep -v submission|awk -F[ '{print $3}'|awk -F] '{print $1}'
exit
[save and exit, then chgrp ec2-user and chmod 777]
vi /home/ec2-user/email-drop.sh
#!/bin/sh
:> /home/ec2-user/email-drop.dat
/home/ec2-user/email-abuse.sh | grep "." | sort -u -n > /home/ec2-user/email-drop.dat
cat /home/ec2-user/email-drop.txt >> /home/ec2-user/email-drop.dat
cat /home/ec2-user/email-drop.dat | sort -u -n > /home/ec2-user/email-drop.txt
iptables -F INPUT
for i in `cat /home/ec2-user/email-drop.txt`
do
iptables -A INPUT -p tcp --destination-port 587 -s $i -j DROP
done
iptables -S INPUT
exit
[save and exit, chgrp ec2-user and chmod 777]
We kick off the acceptance of port 587 with: iptables -I INPUT 1 -p tcp --dport 587 -j ACCEPT Then we used something like this: (I'm showing no more than 4 open connections to port 587 from one IP address. You can change the value, but keep an eye on things by testing with the openssl command: openssl s_client -crlf -quiet -starttls smtp -connect DOMAIN.COM:587 iptables -A INPUT -p tcp --syn --dport 587 -m connlimit --connlimit-above 4 -j DROP We can restrict open connections to the web pages from individual ip addresses. e.g.: iptables -A INPUT -p tcp --syn --dport 443 -m connlimit --connlimit-above 4 -j DROP iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 2 -j DROP We can also block CIDR ranges of addresses instead of putting them in .htaccess. For example, vi cidr.sh with content like this: #!/bin/sh ipset create cidr-set hash:net ipset add cidr-set 104.238.64.0/18 ipset add cidr-set 173.201.0.0/16 iptables -A INPUT -m set --match-set cidr-set src -j DROP [save and exit, and appropriate permissions] When you run these commands, check with iptables -L -vn You will see output including things like this: 0 0 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:587 flags:0x17/0x02 #conn src/32 > 4 803 50704 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443 flags:0x17/0x02 #conn src/32 > 4 22 1232 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80 flags:0x17/0x02 #conn src/32 > 2
Note:
I now only run the blacklist.sh script and these lines:
iptables -A INPUT -p tcp --syn --dport 443 -m connlimit --connlimit-above 5 -j DROP iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 2 -j DROP
If using port 587 for any email inbound and passed by the EC2 Security Group, these settings are needed:
iptables -I INPUT 1 -p tcp --dport 587 -j ACCEPT iptables -A INPUT -p tcp --syn --dport 587 -m connlimit --connlimit-above 4 -j DROP [as a safety measure in crontab for each minute:] * * * * * /usr/sbin/iptables -I INPUT 1 -p tcp --dport 587 -j ACCEPT >/dev/null 2>&1
Without these lines, hackers flood the port and stop it working.
