Introduction

In a public cloud environment, every IP is under constant surveillance by automated botnets. Traditional security (like standard 404 errors) is often insufficient because it still consumes server resources to process malicious requests. This guide outlines a layered defense strategy that identifies malicious behavior, isolates it into dedicated logs, and bans the source IP at the firewall level using Fail2ban.

Core Concepts

1. The “Default Deny” Philosophy

Most bots scan IP ranges directly rather than specific domains. By configuring a Default Server in Nginx that catches all requests not matching your legitimate hostnames, you create a “sinkhole” for 90% of global background noise.

2. The Power of Nginx Status Code 444

Nginx has a non-standard status code: 444 (No Response). When Nginx returns 444, it immediately terminates the TCP connection without sending any headers or data back to the client. This:

• Saves bandwidth.
• Reduces CPU overhead.
• Confuses scanners, making your server appear as if it’s offline or protected by an advanced firewall.

3. Log Isolation (Noise vs. Signal)

Instead of searching for attacks in a massive access.log, we redirect confirmed malicious probes to a dedicated scanners.log. This makes our Fail2ban triggers high-fidelity—if an IP appears in this log, it is 100% a malicious actor.

Step-by-Step Implementation

Step 1: Create the Hardening Snippet

We define common attack patterns (probing for .env files, wp-admin, cgi-bin, etc.) in a reusable snippet.

File Location (on server): /etc/nginx/snippets/hardening.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

# Block .env / .env.* probes
location ~* \.env(\.|$) {
    access_log /var/log/nginx/scanners.log;
    log_not_found off;
    return 444;
}

# Block common php/wordpress/profiler probes
location ~* ^/(wp-admin|wp-login\.php|xmlrpc\.php|phpinfo(\.php)?|app_dev\.php|_profiler)(/|$) {
    access_log /var/log/nginx/scanners.log;
    log_not_found off;
    return 444;
}

# Block hidden files (keep .well-known for ACME, etc.)
location ~ /\.(?!well-known).* {
    access_log /var/log/nginx/scanners.log;
    log_not_found off;
    return 444;
}

# Block common VCS/config/secret file probes (only if requested paths are exact)
location = /.git {
    access_log /var/log/nginx/scanners.log;
    return 444;
}
location = /.svn {
    access_log /var/log/nginx/scanners.log;
    return 444;
}
location = /.hg {
    access_log /var/log/nginx/scanners.log;
    return 444;
}

# Block common backup / editor swap files
location ~* \.(bak|old|orig|save|swp|swo|tmp)$ {
    access_log /var/log/nginx/scanners.log;
    log_not_found off;
    return 444;
}

Step 2: Configure the Stealth Default Server

This handles all traffic directed at your IP address or non-existent subdomains.

File Location (on server): /etc/nginx/conf.d/00-default-deny.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

server {
    listen 80 default_server;
    server_name _;
    access_log /var/log/nginx/scanners.log;
    return 444;
}

server {
    listen 443 ssl default_server;
    server_name _;

    # Reuse an existing cert so TLS handshake can complete before dropping
    ssl_certificate /etc/nginx/ssl/your-domain/fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl/your-domain/key.pem;

    access_log /var/log/nginx/scanners.log;
    return 444;
}

Step 3: Apply Hardening to Production Vhosts

Include the snippet in all your legitimate domain configurations to protect against targeted path scans.

Example Site Config: /etc/nginx/conf.d/my-app.conf

1
2
3
4
5
6
7
8
9
10

server {
    listen 443 ssl;
    server_name my-app.com;

    include snippets/hardening.conf;

    location / {
        proxy_pass http://127.0.0.1:8080;
    }
}

Step 4: Configure Fail2ban Layer

With malicious traffic isolated in scanners.log, we can implement a “Zero Tolerance” policy.

A. Create a minimalist Filter

File Location (on server): /etc/fail2ban/filter.d/nginx-aggressive.conf

1
2
3
4

[Definition]
# If it's in the scanners log, it's a confirmed bot. Catch everything.
failregex = ^<HOST> -.*
ignoreregex =

B. Configure the Jail

Use a unique Jail name (e.g., nginx-scanner-trap) to avoid conflicts with system default naming conventions which may force-override paths.

File Location (on server): /etc/fail2ban/jail.d/nginx-scanners.conf

1
2
3
4
5
6
7
8
9

[nginx-scanner-trap]
enabled  = true
port     = http,https
filter   = nginx-aggressive
logpath  = /var/log/nginx/scanners.log
backend  = polling # Reliable file-based monitoring
findtime = 600      # 10 minute window
maxretry = 1        # One strike and you're out
bantime  = 604800   # Ban for 1 week (or -1 for permanent)

Verification & Monitoring

1. Test the Trap

Run a scan against your own IP from a secondary network (e.g., mobile hotspot):

1

curl -I http://YOUR_SERVER_IP/.env

The connection should be immediately reset (or return no data).

2. Check the “Harvest”

Verify that the IP was logged and subsequently banned:

1
2
3
4
5
6
7
8

# Verify the log entry has been generated
sudo cat /var/log/nginx/scanners.log

# Check Fail2ban jail status
sudo fail2ban-client status nginx-scanner-trap

# View real-time ban actions
sudo tail -f /var/log/fail2ban.log | grep "nginx-scanner-trap"

Phase 2: High-Performance Optimization with ipset

As your banned list grows (e.g., beyond 1,000+ IPs), standard iptables rules can introduce network latency due to linear chain searching (O(n)). By switching to ipset, we utilize hash tables (O(1)), ensuring near-zero performance impact regardless of the blacklist size.

1. Install Kernel Tools

1

sudo apt update && sudo apt install ipset -y

2. Update Fail2ban Global Configuration

Refactor jail.local to use the high-performance action variables.

File: /etc/fail2ban/jail.local

1
2
3
4

[DEFAULT]
# Global high-performance ban action using ipset
banaction = iptables-ipset
banaction_allports = iptables-ipset[type=allports]

3. Implement “Total Lockdown” (All-Ports Ban)

Apply the allports version to critical jails like SSH and your Nginx trap. This ensures that once a host is marked as malicious, it is blocked from every port on your server.

File: /etc/fail2ban/jail.d/sshd-permban.conf

1
2
3
4
5
6

[sshd]
enabled = true
# Use the all-ports ban action
banaction = %(banaction_allports)s
bantime = -1
...

4. Restart Fail2ban to Apply Changes

After modifying fail2ban jail conf, fully restart Fail2ban to ensure the jail is reloaded and the updated banaction takes effect.

1

sudo systemctl restart fail2ban

5. Verify Performance Gains

1
2
3
4
5

# Check the clean iptables ruleset (only one rule per jail)
sudo iptables -L -n

# Inspect the high-speed hash set
sudo ipset list

Conclusion

By shifting security from Response (sending 403 Forbidden) to Stealth (dropping connections) and Automated Retaliation (firewall banning), you significantly reduce the attack surface of your server. This setup allows your backend applications to focus their resources on legitimate users while the silent guard handles the noise.

Phase 2 takes the system from “works well” to “scales indefinitely”: when the banned list grows into the thousands, ipset prevents performance degradation by replacing linear iptables chain growth with O(1) hash-set lookups. Combined with an all-ports ban policy for high-risk offenders (e.g., persistent SSH brute-force), you get a defense that remains fast, predictable, and operationally simple even under constant internet-wide scanning.