Senior DevOps Technical Writer
SSH is the de facto method of connecting to a cloud server. It is durable, and it is extensible — as new encryption standards are developed, they can be used to generate new SSH keys, ensuring that the core protocol remains secure. However, no protocol or software stack is totally foolproof, and SSH being so widely deployed across the internet means that it represents a very predictable attack surface or attack vector through which people can try to gain access.
Any service that is exposed to the network is a potential target in this way. If you review the logs for your SSH service running on any widely trafficked server, you will often see repeated, systematic login attempts that represent brute force attacks by users and bots alike. Although you can make some optimizations to your SSH service to reduce the chance of these attacks succeeding to near-zero, such as disabling password authentication in favor of SSH keys, they can still pose a minor, ongoing liability.
Large-scale production deployments for whom this liability is completely unacceptable will usually implement a VPN such as WireGuard in front of their SSH service, so that it is impossible to connect directly to the default SSH port 22 from the outside internet without additional software abstraction or gateways. These VPN solutions are widely trusted, but will add complexity, and can break some automations or other small software hooks.
Prior to or in addition to committing to a full VPN setup, you can implement a tool called Fail2ban. Fail2ban can significantly mitigate brute force attacks by creating rules that automatically alter your firewall configuration to ban specific IPs after a certain number of unsuccessful login attempts. This will allow your server to harden itself against these access attempts without intervention from you.
In this guide, you’ll see how to install and use Fail2ban on a Ubuntu 20.04 server.
To complete this guide, you will need:
An Ubuntu 20.04 server and a non-root user with sudo privileges. You can learn more about how to set up a user with these privileges in our Initial Server Setup with Ubuntu 20.04 guide.
Optionally, a second server that you can connect to your first server from, which you will use to test getting deliberately banned.
Fail2ban is available in Ubuntu’s software repositories. Begin by running the following commands as a non-root user to update your package listings and install Fail2ban:
- sudo apt update
- sudo apt install fail2ban
Fail2ban will automatically set up a background service after being installed. However, it is disabled by default, because some of its default settings may cause undesired effects. You can verify this by using the systemctl
command:
- systemctl status fail2ban.service
Output○ fail2ban.service - Fail2Ban Service
Loaded: loaded (/lib/systemd/system/fail2ban.service; disabled; vendor preset: enabled
Active: inactive (dead)
Docs: man:fail2ban(1)
You could enable Fail2ban right away, but first, you’ll review some of its features.
The fail2ban service keeps its configuration files in the /etc/fail2ban
directory. There is a file with defaults called jail.conf
. Go to that directory and print the first 20 lines of that file using head -20
:
- cd /etc/fail2ban
- head -20 jail.conf
Output#
# WARNING: heavily refactored in 0.9.0 release. Please review and
# customize settings for your setup.
#
# Changes: in most of the cases you should not modify this
# file, but provide customizations in jail.local file,
# or separate .conf files under jail.d/ directory, e.g.:
#
# HOW TO ACTIVATE JAILS:
#
# YOU SHOULD NOT MODIFY THIS FILE.
#
# It will probably be overwritten or improved in a distribution update.
#
# Provide customizations in a jail.local file or a jail.d/customisation.local.
# For example to change the default bantime for all jails and to enable the
# ssh-iptables jail the following (uncommented) would appear in the .local file.
# See man 5 jail.conf for details.
#
# [DEFAULT]
As you’ll see, the first several lines of this file are commented out – they begin with #
characters indicating that they are to be read as documentation rather than as settings. As you’ll also see, these comments are directing you not to modify this file directly. Instead, you have two options: either create individual profiles for Fail2ban in multiple files within the jail.d/
directory, or create and collect all of your local settings in a jail.local
file. The jail.conf
file will be periodically updated as Fail2ban itself is updated, and will be used as a source of default settings for which you have not created any overrides.
In this tutorial, you’ll create jail.local
. You can do that by copying jail.conf
:
- sudo cp jail.conf jail.local
Now you can begin making configuration changes. Open the file in nano
or your favorite text editor:
- sudo nano jail.local
While you are scrolling through the file, this tutorial will review some options that you may want to update. The settings located under the [DEFAULT]
section near the top of the file will be applied to all of the services supported by Fail2ban. Elsewhere in the file, there are headers for [sshd]
and for other services, which contain service-specific settings that will apply over top of the defaults.
[DEFAULT]
. . .
bantime = 10m
. . .
The bantime
parameter sets the length of time that a client will be banned when they have failed to authenticate correctly. This is measured in seconds. By default, this is set to 10 minutes.
[DEFAULT]
. . .
findtime = 10m
maxretry = 5
. . .
The next two parameters are findtime
and maxretry
. These work together to establish the conditions under which a client is found to be an illegitimate user that should be banned.
The maxretry
variable sets the number of tries a client has to authenticate within a window of time defined by findtime
, before being banned. With the default settings, the fail2ban service will ban a client that unsuccessfully attempts to log in 5 times within a 10 minute window.
[DEFAULT]
. . .
destemail = root@localhost
sender = root@<fq-hostname>
mta = sendmail
. . .
If you need to receive email alerts when Fail2ban takes action, you should evaluate the destemail
, sendername
, and mta
settings. The destemail
parameter sets the email address that should receive ban messages. The sendername
sets the value of the “From” field in the email. The mta
parameter configures what mail service will be used to send mail. By default, this is sendmail
, but you may want to use Postfix or another mail solution.
[DEFAULT]
. . .
action = $(action_)s
. . .
This parameter configures the action that Fail2ban takes when it wants to institute a ban. The value action_
is defined in the file shortly before this parameter. The default action is to update your firewall configuration to reject traffic from the offending host until the ban time elapses.
There are other action_
scripts provided by default which you can replace $(action_)
with above:
…
# ban & send an e-mail with whois report to the destemail.
action_mw = %(action_)s
%(mta)s-whois[sender="%(sender)s", dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"]
# ban & send an e-mail with whois report and relevant log lines
# to the destemail.
action_mwl = %(action_)s
%(mta)s-whois-lines[sender="%(sender)s", dest="%(destemail)s", logpath="%(logpath)s", chain="%(chain)s"]
# See the IMPORTANT note in action.d/xarf-login-attack for when to use this action
#
# ban & send a xarf e-mail to abuse contact of IP address and include relevant log lines
# to the destemail.
action_xarf = %(action_)s
xarf-login-attack[service=%(__name__)s, sender="%(sender)s", logpath="%(logpath)s", port="%(port)s"]
# ban IP on CloudFlare & send an e-mail with whois report and relevant log lines
# to the destemail.
action_cf_mwl = cloudflare[cfuser="%(cfemail)s", cftoken="%(cfapikey)s"]
%(mta)s-whois-lines[sender="%(sender)s", dest="%(destemail)s", logpath="%(logpath)s", chain="%(chain)s"]
…
For example, action_mw
takes action and sends an email, action_mwl
takes action, sends an email, and includes logging, and action_cf_mwl
does all of the above in addition to sending an update to the Cloudflare API associated with your account to ban the offender there, too.
Next is the portion of the configuration file that deals with individual services. These are specified by section headers, like [sshd]
.
Each of these sections needs to be enabled individually by adding an enabled = true
line under the header, with their other settings.
[jail_to_enable]
. . .
enabled = true
. . .
By default, the SSH service is enabled and all others are disabled.
.
Some other settings that are set here are the filter
that will be used to decide whether a line in a log indicates a failed authentication and the logpath
which tells fail2ban where the logs for that particular service are located.
The filter
value is actually a reference to a file located in the /etc/fail2ban/filter.d
directory, with its .conf
extension removed. These files contain regular expressions (a common shorthand for text parsing) that determine whether a line in the log is a failed authentication attempt. We won’t be covering these files in-depth in this guide, because they are fairly complex and the predefined settings match appropriate lines well.
However, you can see what kind of filters are available by looking into that directory:
- ls /etc/fail2ban/filter.d
If you see a file that looks related to a service you are using, you should open it with a text editor. Most of the files are fairly well commented and you should be able to at least tell what type of condition the script was designed to guard against. Most of these filters have appropriate (disabled) sections in the jail.conf
file that we can enable in the jail.local
file if desired.
For instance, imagine that you are serving a website using Nginx and realize that a password-protected portion of your site is getting slammed with login attempts. You can tell fail2ban to use the nginx-http-auth.conf
file to check for this condition within the /var/log/nginx/error.log
file.
This is actually already set up in a section called [nginx-http-auth]
in your /etc/fail2ban/jail.conf
file. You would just need to add the enabled
parameter:
. . .
[nginx-http-auth]
enabled = true
. . .
When you are finished editing, save and close the file. At this point, you can enable your Fail2ban service so that it will run automatically from now on. First, run systemctl enable
:
- sudo systemctl enable fail2ban
Then, start it manually for the first time with systemctl start
:
- sudo systemctl start fail2ban
You can verify that it’s running with systemctl status
:
- sudo systemctl status fail2ban
Output● fail2ban.service - Fail2Ban Service
Loaded: loaded (/lib/systemd/system/fail2ban.service; enabled; vendor preset: enab>
Active: active (running) since Tue 2022-06-28 19:29:15 UTC; 3s ago
Docs: man:fail2ban(1)
Main PID: 39396 (fail2ban-server)
Tasks: 5 (limit: 1119)
Memory: 12.9M
CPU: 278ms
CGroup: /system.slice/fail2ban.service
└─39396 /usr/bin/python3 /usr/bin/fail2ban-server -xf start
Jun 28 19:29:15 fail2ban20 systemd[1]: Started Fail2Ban Service.
Jun 28 19:29:15 fail2ban20 fail2ban-server[39396]: Server ready
In the next step, you’ll demonstrate Fail2ban in action.
From another server, one that won’t need to log into your Fail2ban server in the future, you can test the rules by getting that second server banned. After logging into your second server, try to SSH into the Fail2ban server. You can try to connect using a nonexistent name:
- ssh blah@your_server
Enter random characters into the password prompt. Repeat this a few times. At some point, the error you’re receiving should change from Permission denied
to Connection refused
. This signals that your second server has been banned from the Fail2ban server.
On your Fail2ban server, you can see the new rule by checking your iptables
output. iptables
is a command for interacting with low-level port and firewall rules on your server. If you followed DigitalOcean’s guide to initial server setup, you will be using ufw
to manage firewall rules at a higher level. Running iptables -S
will show you all of the firewall rules that ufw
already created:
- sudo iptables -S
Output-P INPUT DROP
-P FORWARD DROP
-P OUTPUT ACCEPT
-N f2b-sshd
-N ufw-after-forward
-N ufw-after-input
-N ufw-after-logging-forward
-N ufw-after-logging-input
-N ufw-after-logging-output
-N ufw-after-output
-N ufw-before-forward
-N ufw-before-input
-N ufw-before-logging-forward
-N ufw-before-logging-input
-N ufw-before-logging-output
…
If you pipe the output of iptables -S
to grep
to search within those rules for the string f2b
, you can see the rules that have been added by fail2ban:
- sudo iptables -S | grep f2b
Output-N f2b-sshd
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A f2b-sshd -s 134.209.165.184/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -j RETURN
The line containing REJECT --reject-with icmp-port-unreachable
will have been added by Fail2ban and should reflect the IP address of your second server.
You should now be able to configure some banning policies for your services. Fail2ban is a useful way to protect any kind of service that uses authentication. If you want to learn more about how fail2ban works, you can check out our tutorial on how fail2ban rules and files work.
For information about how to use fail2ban to protect other services, you can read about How To Protect an Nginx Server with Fail2Ban on Ubuntu 14.04 and How To Protect an Apache Server with Fail2Ban on Ubuntu 14.04.
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
To prevent Fail2Ban from accidentally banning yourself (especially on SSH), you can use the following methods:
1. Whitelist Your IP Address
You can whitelist your IP address in the Fail2Ban configuration so that it will never get banned, regardless of failed login attempts.
ignoreip
option under the[sshd]
section:Replace
<your_ip>
with your actual IP address or range of addresses you want to whitelist.You can add multiple IPs or CIDR ranges by separating them with spaces, like so:
Thanks! I followed the tutorial. Testing from another server did not ban the IP.
Then I read on another tutorial page: “Although Fail2Ban is excellent software, it is not as beneficial when your SSH server is configured to only public key authentication, as it is impossible to log in with a password.”
As I disabled password login, the tutorial would have been void for me. Would be cool, if you could specify that more clearly.
Hi Alex, thank you for this tutorial. I started looking at using fail2ban a few years ago and was horrified to find that hackers were using carfully crafted login attempts targetted at F2B. F2B was actually being exploited as a new attack vector. So then I came across “knocking” which I thought was a better solution but still not great. Basically, all accesses to sshd are initially blocked at the firewall. You knock on the server door (do a couple of invalid accesses in a row) the system recognises it and allows your IP address access through the firewall to sshd (you still need to authenticate). I’ve taken this one step further by adding a knock page to my website so there is nothing scouring error logs for special sequences of events. The knock page is also password protected and comes with a single use token (you can’t try brute forcing this puppy). To make things even more interesting, if the webserver detects an intrusion attempt then the IP address that it’s comming from gets firewalled.
Perhaps a tutorial about “knocking” would be useful to some of your followers.
Thanks for the great tutorial! Unfortunately after activating it, I can’t access the server from the DO Recovery Console.
I can see in the logs:
fail2ban.filter [774431]: INFO [sshd] Found 198.211.111.194
fail2ban.filter [774431]: INFO [sshd] Found 198.211.111.194
fail2ban.filter [774431]: INFO [sshd] Found 198.211.111.194
fail2ban.actions [774431]: NOTICE [sshd] Ban 198.211.111.194
Is it just a matter of adding DO IP range to the “allow list” or…?
Thanks!
A very useful addition to protect against self banning is to putting in a list of IP ranges that cover expected client & admin source addresses into the ignoreip parameter in the jail.conf file. While not foolproof in preventing self bans or cases where bad traffic is originating from the same range as a valid client it many cases this is not so & you avoid getting panicked “I can’t get in” messages outside of office hours.
On some installations, fail2ban don’t recognize systemd logs and it fails to start with error “have not found any log file for sshd jail”. In that way, it’s wise to force backend=systemd on [sshd] section.
So, I have a site hosted on one of the droplets. I have been getting a lot of failed login attempts to my droplet even though I have kept the IP hidden by using CloudFlare and using proxified connections only.
By following your tutorial step-by-step, I have managed to enable fail2ban service successfully but the problem is that I made changes in the .local file that we created in the post. The changes are not reflected somehow when I tried to login through an IP. Do we have to mention somehow how to use the updated configuration file for fail2ban?