Nginx as reverse proxy with Let's encrypt certificate for HTTPS using docker-compose

31 May 2021
#Nginx#Docker-Compose#Docker

I have been thinking for a long time that I should try Nginx as my reverse proxy. So finally it was time to get myself into learning about Nginx.

Short about Reverse Proxy

There is already loads of information about what a reverse proxy is, so I will make it short very short.

A reverse proxy is the server that will accept all the incomming requests from the web. The reverse proxy will look at each request and match against a set of rules and then forward the request to another server matching the rules. The other server can be both internal (no direct access from the web) or an external server.

The reverse proxy can handle the secure certificate stuff needed for HTTPS so the internal servers doesn’t have to have it’s own certificate.

NGINX

Let’s start with the proxy and then we add certificates later.

version: '3'
services:
  nginx:
    image: nginx:1.21.0-alpine
    container_name: nginx
    restart: unless-stopped
    ports:
      - "80:80"
    volumes:
      - /srv/nginx/nginx.conf:/etc/nginx/nginx.conf
      - /srv/nginx/conf.d:/etc/nginx/conf.d

Pretty simple docker-compose file, it will start Nginx and listen on port 80. Nginx will server it’s own default HTML because I have not defined any volume for HTML files. I don’t need it, I will only use it as a reverse proxy so I only need access to nginx.conf and /etc/nginx/conf.d folder.

Reverse proxy config

Well Nginx can’t read your mind (yet) so we have to write some config files so it can proxy the service. By default will Nginx read all .conf files in /srv/nginx/conf.d when Nginx reads the config file.

server {
  listen 80;
  listen [::]:80;
  server_name raddinox.com www.raddinox.com blog.raddinox.com;

  location / {
    proxy_pass 10.0.0.14:3001;
  }
}

So this is going to pass any request to raddinox.com, www.raddinox.com, blog.raddinox.com to my internal server with IP 10.0.0.14 and the service listening on port 3001.

I like to split my different subdomains into separate files. So for my Gitlab service I made another config file.

server {
   listen 80;
   listen [::]:80;
   server_name git.raddinox.com;

  location / {
    proxy_pass 10.0.0.15:4590;
  }
}

So whenever a request is made to git.raddinox.com, Nginx will forward that request to the server with IP 10.0.0.15 and the service listening on port 4590.

Since none of my services listen on the same port, I could actually run them on the same server (Docker FTW!) and keep them on separate ports. The proxy server will make the services appear on port 80 regardless of the internal port.

Now Let’s Encrypt this!

Old http (port 80) is ancient and insecure and everyone is moving to https (port 443). It is possible to just change listen 80; to listen 443; but every modern browser will complain about certificate

This is where Let’s encrypts is great. Free certificates for your web server to server pages over the more secure HTTPS.

Before we start hacking away in our docker comopse file, we need to create our certificates.

I made a script for this, certbot-first.sh

#!/bin/bash

docker run -it --rm --name certbot \
            -p 80:80 -p 443:443 \
            -v "/srv/nginx/cert:/etc/letsencrypt" \
            certbot/certbot certonly --standalone -d www.raddinox.com -d raddinox.com \
                 -d git.raddinox.com -d blog.raddinox.com \
                 --non-interactive --agree-tos \
                 --email <YOUR_EMAIL> --expand

Just remember to change <YOUR_EMAIL> to your actual email. Before you run this you have to stop Nginx, because certbot must be listening on port 80 (I’m not sure about 443).

When certbot is finished all the data and certificates will be in /srv/nginx/cert.

Secure Nginx

Now we finally have certificates, so let’s make Nginx use HTTPS.

First we need to update our docker-compose file so nginx service will use port 443. And we will add the volume where our certificates are

version: '3'
services:
  nginx:
    image: nginx:1.21.0-alpine
    container_name: nginx
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /srv/nginx/nginx.conf:/etc/nginx/nginx.conf
      - /srv/nginx/conf.d:/etc/nginx/conf.d
      - /srv/nginx/cert:/etc/letsencrypt:ro

and then our config files for our services will need to listen to port 443

server {
  listen 443 ssl http2;
  server_name raddinox.com www.raddinox.com blog.raddinox.com;
  ssl_certificate /etc/letsencrypt/live/www.raddinox.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/www.raddinox.com/privkey.pem;
  location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $http_host;
    proxy_pass 10.0.0.14:3001;
  }
}
server {
  listen 80;
  listen [::]:80;
  server_name raddinox.com www.raddinox.com blog.raddinox.com;

  location / {
    return 301 https://www.raddinox.com
  }
}

When we are using proxy_pass on https we need to set some headers so the internal service will know that a certificate has already been served. The above is what worked for my Ghost blog. But from what I have been reading this is different depending on the service. My Gitlab service seems to work fine without any of the headers set.

This is all that is needed to get HTTPS working.

But it will not work for long, because certificates must be renewed. You can manually renew your certificates using certbot.

Just run

docker run -it --rm --name certbot \
                -p 80:80 -p 443:443 \
                -v "/srv/nginx/cert:/etc/letsencrypt" \
                certbot/certbot renew

This will run certbot in renew mode, it will read the config data in /etc/letsencrypt and renew the certificates it already has created. Running certbot renew will ONLY renew if it must, so it is safe to run renew every day.

The problem is that certbot needs to listen to port 80 (and maybe 443) so whenever we want to run this we need to stop our Nginx service.

Certbot autorenew service

Because I most likely will forget to manuall stop Nginx and then manually run certbot renew and then manually start Nginx again (and I’m lazy). It has to run by itself. So lets put it in a docker container!

The existing certbot container will just run one command and then stop. Not useful, but we can make something useful from it. Let’s add crontab to the certbot image.

FROM certbot/certbot:v1.15.0
MAINTAINER PeterH (https://blog.raddinox.com)

VOLUME /etc/letsencrypt

RUN apk update && apk add openssl curl

# Add crontab
ADD crontab /etc/crontabs
RUN crontab /etc/crontabs/crontab

COPY *.sh /opt/
RUN chmod -R +x /opt/

ENTRYPOINT ["/opt/start_crond.sh"]

So this will build a Docker image with certbot and crontab. But before we build the image we need a crontab file and some scripts

# Run at 03:00 every day
0 3 * * * /opt/certbot_renew.sh

So this will run /opt/certbot_renew.sh script every day at 03:00.

#!/bin/sh

certbot --quiet renew

And finally we need the start_crond.sh entry point script

#!/bin/sh

# Stuff to do at container start before crond starts

# Lets start crond for perodic updates
exec crond -f

Well, we don’t actually need the start_crond.sh script. It’s possible to start crond -f directly as the ENTRYPOINT in the Dockerfile. I just made a script just in case I want to do some checks on container startup.

So now we have our certbot image we can build a container from to auto renew the certificates.

But we still have the problem of Nginx can’t be running at the same time as certbot.

Nginx is up for a challange

We can solve our last problem using a shared folder where certbot will place the certificate, acme-challage as it is called, response files. And then Nginx will serve those files.

First we force certbot to not start it’s own built-in webserver and write those files to disk.

#!/bin/sh

certbot --quiet renew --webroot -w /var/www/certbot

so now certbot will write the acme-challange files in /var/www/certbot.

Let’s add this path to Nginx config, so the files will be accessable whenever certbot tries to renew the certificates.

In my /srv/nginx/conf.d/default.conf I have added a few lines to serve the acme-challenge. It will be empty whenever it’s not needed.

server {
  listen 80;
  listen [::]:80;
  server_name raddinox.com www.raddinox.com;

  location /.well-known/acme-challenge/ {
    allow all;
    root /var/www/certbot;
  }

  location / {
    return 301 https://www.raddinox.com;
  }
}

Now whenever someone, like Let’s encrypts servers, accessing www.raddinox.com/.well-known/acme-challange/ Nginx will serve whatever files matching that path that exists in /var/www/certbot.

This is great, because thats exactly where certbot will put the acme-challange response files!.. but in another container..

Finalize docker compose

So now it’s time to finalize the docker compose file, to build our own certbot image and add /var/www/certbot to both services to make sure the read and write to the same folder, this way Nginx can serve the acme-challange response files when certbot creates them.

version: '3'
services:
  certbot:
    build: certbot-img/.  # Build custom certbot image, with crond auto-renew feature
    container_name: certbot
    restart: unless-stopped
    volumes:
      - /srv/nginx/cert:/etc/letsencrypt          # Certificates
      - /srv/nginx/certbot-www:/var/www/certbot   # ACME challange
  nginx:
    image: nginx:1.21.0-alpine
    container_name: nginx
    restart: unless-stopped
    depends_on:
      - certbot
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /srv/nginx/nginx.conf:/etc/nginx/nginx.conf
      - /srv/nginx/conf.d:/etc/nginx/conf.d
      - /srv/nginx/cert:/etc/letsencrypt:ro        # Certificates from certbot
      - /srv/nginx/certbot-www:/var/www/certbot:ro  # ACME challange files when certbot renews

All files is available here https://github.com/Raddinox/nginx-certbot-autorenew

Raddinox © 2024 - Powered by