Skip to content

SSH Hardening: 5-Layer Defense

SSH on port 22 is the most-scanned port on the internet. Your server sees an average of thousands of attempts per day, not hundreds of millions — the volume comes from broad background scanning, and Fail2ban's job is to make that background noise stop mattering. The five layers below compound: each one cuts the attack surface for the next.

The 5 layers: 1. Authentication — keys only, no password 2. Access control — no root, whitelist users 3. Exposure — non-standard port, host-based firewall 4. Logging — verbose logs, Fail2ban, alerting 5. Host — PAM, MFA, agent-forwarding, host key rotation

Do them in order. Each layer assumes the previous is done.

Safety Workflow: The 3 Steps That Save You

Every hardening failure I've seen traces back to skipping this. Memorize it:

Step 1 — keep a second session open. Never log in, harden, and close the only session you have. Two terminals, two SSH sessions. If the second fails, the first is still live.

Step 2 — add another machine's key first. Before changing any sshd_config, scp a key from a different machine to this host's ~/.ssh/authorized_keys for the same user. If you somehow get locked out, this key is your off-ramp.

Step 3 — test with ssh -v before closing. After changing config, open a third terminal and try to log in with the new settings. ssh -v shows the whole handshake — if it succeeds, you can safely close the old sessions.

The failure mode without this: you close the only session that works, then find out you've locked yourself out. Cloud console (VNC / serial) is the escape hatch, but not every server has one, and not every datacenter gives you one.

Layer 1: Authentication (Highest Priority)

Everything else is decoration without this. Two settings in sshd_config:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
UsePAM no           # if you don't need sudo/PAM; else: yes

Test before reload:

sshd -t                    # syntax check
sudo systemctl reload sshd  # hot-reload, don't restart

The three settings that look similar and mean different things:

  • PasswordAuthentication no disables password logins for SSH.
  • KbdInteractiveAuthentication no disables the interactive keyboard challenge (some distros use this for password auth via PAM).
  • ChallengeResponseAuthentication no is the older name for KbdInteractiveAuthentication — keep both set to no on older OpenSSH versions.

Leaving any of them at default (or missing them in older OpenSSH) means password auth is still on. This is the single most common misconfiguration.

Key Management Basics

# Generate an Ed25519 key (modern default, faster, shorter)
ssh-keygen -t ed25519 -C "user@machine" -f ~/.ssh/id_ed25519

# Deploy the public key to the server
ssh-copy-id user@server
# Or manually:
cat ~/.ssh/id_ed25519.pub | ssh user@server "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

~/.ssh permissions matter. If they're wrong, SSH refuses:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Ownership matters too — must be owned by the user, not root, not a group:

chown -R user:user ~/.ssh

Common trap: rsync from another machine without --preserve-perms, and ~/.ssh ends up 755. Login will silently fail with Permission denied — check sshd debug logs:

sudo journalctl -u ssh -n 50 --no-pager | grep -i 'auth\|refuse'

Layer 2: Access Control

Three settings that look similar and are all necessary.

Block direct root login:

PermitRootLogin no

Alternatives if you truly need root access: - PermitRootLogin prohibit-password — root login allowed but only via key - PermitRootLogin forced-commands-only — root only allowed with a specific command (e.g., a restricted admin SSH key)

Both are still weaker than no. Prefer no + sudo for regular users.

Deny common bad guys (defense in depth):

DenyUsers nobody daemon bin sync man news uucp ftp

Or the inverse — a denylist of known-bad usernames that scanners try:

DenyUsers root admin test guest ubuntu@10.0.0.*

AllowUsers whitelist (strongest form):

AllowUsers alice@10.0.0.0/24 bob@192.168.1.0/24

Note the user@source syntax — this restricts by both user and source IP range. A user from the wrong subnet can't log in even with a valid key. This is the piece people miss: AllowUsers alone without source restriction still allows anyone in the world to try.

Layer 3: Exposure (Port + Network)

Non-Standard Port

Change port 22 to something less common:

Port 2222

Update the firewall at the same time:

# ufw
sudo ufw allow 2222/tcp && sudo ufw deny 22/tcp

# firewalld
sudo firewall-cmd --permanent --add-port=2222/tcp
sudo firewall-cmd --permanent --remove-port=ssh  # or 22/tcp
sudo firewall-cmd --reload

# nftables / iptables
sudo iptables -A INPUT -p tcp --dport 2222 -m state --state NEW -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 22 -j DROP

Reality check on port change: it doesn't stop a determined attacker (nmap finds the port in seconds). What it does is eliminate ~95% of background noise. Measure with Fail2ban logs — after changing ports, you'll typically see a 95%+ drop in blocked IPs.

Cost: bastion hosts, VPNs, monitoring, and CI/CD pipelines all need updating. Do this last among the five layers — after key auth, root blocking, and Fail2ban are in place.

Firewall at the Network Layer

Before sshd sees the connection, the firewall should drop it. This is separate from port hardening — even a port-22 server behind a firewall is exposed to the SSH handshake.

Allow only known sources, if possible:

# ufw — restrict SSH to your office IP and bastion
sudo ufw allow from 203.0.113.0/24 to any port 2222 proto tcp
sudo ufw allow from 198.51.100.5 to any port 2222 proto tcp   # bastion
sudo ufw deny 2222/tcp  # deny others

If you can't restrict sources, restrict protocols and rates at the firewall:

# Limit new SSH connections per IP
sudo iptables -A INPUT -p tcp --dport 2222 -m state --state NEW -m recent --set
sudo iptables -A INPUT -p tcp --dport 2222 -m state --state NEW -m recent --update --seconds 60 --hitcount 5 -j DROP

This drops the 6th new connection from any IP within 60 seconds. Legitimate users with 2-3 connections per minute won't notice; scanners will get blocked before reaching sshd.

Layer 4: Logging + Fail2ban

Verbose Logging

LogLevel VERBOSE

The four LogLevel values in practice: - QUIET — useless for security - ERROR — misses pre-auth failures (bad) - INFO — catches most auth failures (default, okay) - VERBOSE — captures everything, including pre-auth probes (recommended)

VERBOSE is chatty — expect more disk. On a busy server, ship logs off-box:

# systemd-journal-remote, fluentd, rsyslog forward, etc.
sudo systemctl enable systemd-journal-remote

Fail2ban Setup

sudo apt install fail2ban            # Debian/Ubuntu
sudo dnf install fail2ban            # RHEL/Fedora

Configure the SSH jail — extend defaults:

# /etc/fail2ban/jail.local
[sshd]
enabled = true
port    = 2222
filter  = sshd
backend = systemd
mode    = aggressive          # bans on pre-auth activity too
bantime = 1h
findtime = 10m
maxretry = 3

The 5 Fail2ban jails worth enabling:

Jail Purpose When to enable
sshd Standard SSH brute force Always
sshd-ddos Slow scanners / DDoS probes Always on internet-facing
sshd-pam PAM-level auth failures If UsePAM yes
nginx-botsearch Nginx bot scanning If Nginx in front
recycler Cleanup of stale ban entries Always

Watch fail2ban's own log — a healthy jail bans IP ranges, not just IPs:

sudo journalctl -u fail2ban -n 50 --no-pager
sudo fail2ban-client status sshd

Anti-lockout guardrail: put your bastion's IP on the ignoreip list. If Fail2ban is misconfigured and starts banning you, this is your escape:

[DEFAULT]
ignoreip = 127.0.0.1/8 ::1 203.0.113.0/24 198.51.100.5

Layer 5: Host-Level Hardening

PAM

sshd can either use PAM (UsePAM yes) or not. Most distros default to yes because sudo integrates with PAM.

If you want pure key auth with no PAM involvement:

UsePAM no

This is the cleanest config. Downside: you lose sudo integration unless sudo itself talks directly to SSH agent. On most setups, this works because sudo queries the PAM modules directly, not through sshd.

If you must use PAM, configure /etc/pam.d/sshd carefully — disable modules that do password auth even when the user claims key auth:

auth       required     pam_nologin.so
# auth     optional     pam_permit.so   # DO NOT ENABLE
# auth     sufficient   pam_unix.so     # DO NOT ENABLE (allows password fallback)

Google Authenticator via PAM:

# /etc/pam.d/sshd
auth      required      pam_google_authenticator.so nullok

sshd_config:

ChallengeResponseAuthentication yes
KbdInteractiveAuthentication yes

This is the one case where you want KbdInteractiveAuthentication enabled — it's what serves the one-time code prompt. If you enable MFA, add Match Group restrictions so only MFA-registered users are prompted.

Agent Forwarding

If you enable AllowAgentForwarding yes, a user with key access on this host can use their SSH agent from their laptop. That's a lateral-movement risk. Disable unless needed:

AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no

Host Key Rotation

Every 90 days, rotate host keys. Attackers who phish a private key from the client's known_hosts cache (or compromise a workstation) can impersonate your server. Rotate and force re-fingerprint:

ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N "" -t ed25519
sudo systemctl restart sshd

Clients will need to re-accept the fingerprint. Announce the change via your change management system before executing.

5 Common Self-Lockout Traps

Trap 1: Changed sshd_config, closed the only session, can't get back in. - Fix: cloud console (VNC / serial). If no console, boot to rescue mode or mount the root filesystem from another machine, edit /etc/ssh/sshd_config, restore. - Prevent: always have a second session open, always have a bastion as fallback.

Trap 2: Removed authorized_keys, permission now wrong. - Fix: chmod 600 ~/.ssh/authorized_keys && chown user:user ~/.ssh/authorized_keys. - Prevent: never edit authorized_keys while logged in as that user. Do it from another machine.

Trap 3: Changed port to 2222, forgot to update firewall. - Fix: sudo iptables -A INPUT -p tcp --dport 2222 -j ACCEPT. - Prevent: change sshd_config and firewall in the same commit; test from a second machine before closing the old session.

Trap 4: AllowUsers typo blocks legit users. - Fix: sudo vim /etc/ssh/sshd_config, sudo systemctl reload sshd. - Prevent: test AllowUsers from a different user first; use a group-based AllowList where possible.

Trap 5: Disabled PAM but sudo still needs it. - Fix: flip UsePAM yes back on, keep PasswordAuthentication no. - Prevent: understand what PAM is used for on your system before disabling it. sudo -l still needs PAM.

Cheat Sheet

15 sshd_config Lines That Matter

Port 2222
AddressFamily inet
ListenAddress 0.0.0.0

Protocol 2
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
ChallengeResponseAuthentication no
PermitEmptyPasswords no

PermitRootLogin no
MaxAuthTries 3
LoginGraceTime 30

AllowUsers alice@10.0.0.0/24 bob@192.168.1.0/24

ClientAliveInterval 300
ClientAliveCountMax 2

AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no

LogLevel VERBOSE
Subsystem sftp internal-sftp

12 Diagnostic Commands

# Command What it tells you
1 sshd -t Config syntax check
2 sshd -T \| grep -Ei 'port\|auth\|allow' Effective config (all merged)
3 sudo systemctl status sshd Service state
4 sudo journalctl -u ssh -n 50 Recent auth attempts
5 sudo fail2ban-client status Jails and ban counts
6 sudo fail2ban-client get sshd banned Currently banned IPs
7 sudo fail2ban-client unban IP Manual unban
8 grep -i auth /var/log/auth.log \| tail -50 Raw auth log (Debian)
9 journalctl -u ssh -f Live auth log
10 nmap -sV -p 2222 target Verify exposed services
11 ssh -v user@host Verbose handshake (debug auth failures)
12 ss -tn state established '( sport = :2222 )' Active SSH sessions

3 Minimal Configs

Hardening base (5-layer minimum):

Port 2222
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
MaxAuthTries 3
LoginGraceTime 30
AllowUsers alice@10.0.0.0/24
LogLevel VERBOSE

Fail2ban aggressive:

[sshd]
enabled = true
port    = 2222
backend = systemd
mode    = aggressive
bantime = 1h
findtime = 10m
maxretry = 3

PAM with MFA:

auth       required     pam_google_authenticator.so nullok
account    required     pam_nologin.so

Where This Doesn't Help

SSH hardening is a partial defense, not a comprehensive one. If an attacker has your private key, none of the above matters. That's why:

  • Rotate keys on any compromise signal
  • Use short-lived credentials where possible (e.g., short-lived authorized_keys from a just-in-time system)
  • Monitor key fingerprints — a new key appearing is a red flag
  • Consider bastion hosts — the bastion becomes the only SSH-exposed endpoint, and your internal servers have sshd bound to 127.0.0.1 only

Authoritative References

Written for ops.jiunotes.top. Original work based on production hardening experience. Not copied from any third-party blog. CC BY-NC-SA 4.0.