Building this Website on Git Push

Preface

In the spirit of my recent fascination with self-sovereignity and decentralization I will replace the fancy friendly mature reliable GitHub Pages with a hack.

As a side note, I like GitHub Pages, the service democratized static website hosting and made it easily approachable to many developers. GitHub Actions allow flexible builds outside of the default Jekyll system. For example, I’m using a Makefile with SphinxDocs, and quiet happy with it. And all of it is completely free. But this article is not about GitHub Pages, this is about an alternative.

The hack is to run a Debian VM on my home server with a git repo and post-receive hook that builds a static website. The built artifact is then committed to another repo, which is pushed to a Cloud VPS. I could’ve simplified the setup by building the website directly on the VPS, the problem is that it’s so tiny, I doubt it can handle the build process.

The website build process got pretty involved over the years since I’m hoarding all my petty experiments for no good reason. The builder needs Python, graphviz, NodeJS, and customary imperial tonne of npm packages.

I’ll keep the VM running at all times and preserve the build files so it can run incrementally. I’ll also ship the build artifact through git, which should be much faster then a complete artifact every time.

Virtual Machine

My favorite way of running virtual machines is with Vagrant and KVM.

Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
    config.vm.box = "debian/trixie64"
    config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
    config.vm.provider :libvirt do |libvirt|
      libvirt.driver = "kvm"
      libvirt.uri = 'qemu:///system'
      libvirt.cpus = 2
      libvirt.memory = "2048"
      libvirt.memorybacking :access, :mode => "shared"
    end
    config.vm.provision "shell", path: "provision_builder.sh"
end

Upon creation (vagrant up) it immediately runs the provisioning script that sets everything up. The script is reentrant, because I had to iterate a bit to polish all the kinks.

provision_builder.sh:

#!/bin/bash
set -euo pipefail

apt-get update
apt-get install -y      \
    curl                \
    build-essential     \
    git                 \
    python3-venv        \
    python-is-python3   \
    graphviz            \
    nodejs              \
    npm                 \
    rsync

tailscale status || ( \
    mkdir -p /etc/apt/keyrings \
    && curl -fsSL https://pkgs.tailscale.com/stable/debian/trixie.noarmor.gpg > /usr/share/keyrings/tailscale-archive-keyring.gpg \
    && curl -fsSL https://pkgs.tailscale.com/stable/debian/trixie.tailscale-keyring.list > /etc/apt/sources.list.d/tailscale.list \
    && apt-get update \
    && apt-get install -y tailscale \
    && tailscale login \
    && tailscale up \
)

id builder || useradd -rms /usr/bin/git-shell builder
install -o builder -g builder -m 0700 -d ~builder/.ssh
install -o builder -g builder -m 0700 -d /var/www/site
install -o builder -g builder -m 0600 /dev/stdin ~builder/.ssh/authorized_keys <<'EOF'
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYxOnUHnt2KZ8kdjYjO/xWflaFKxXXJLv6V8/TiXgow8L+QdFmcEJ/NRdR6/LVLEwiJ5h9l26mY8XxlpVAIY43NqbhPUdBp6SoeX2tpHFQa4R1i7coO3bO1sjAVqeTmTby4iROtWZ89OEsqYnWyYco4py+sn6X+h8TDRIbrl2zYQI9IwK8O2UJTV9qT2Vy4s4fitLTeO6AI7935OsrLzXV+iaGGmhoUfpZcHZ5I9puaaTOyxuJ3q4nA0PNiZ9Lw7+TYOo73eXPA+qRrsvEy6b6x3+izyj4WX31YSklksw5CX+jjc23d7muV8cHFaoO1GkueVYyve8ncqy0dGn9CiDQudVqUyhqkF49MvWO1Hjg9SeidaKGqalh0Pv8RJquTJ8aUXcVS9GwCmYu+/JfBVcCGYKEpcwrLOt/iYa9iHCsImb/wlO08n3R+HBIF4At0Jxgd4wWM8ZhSXoA2UjCBojZwcWLPuS+S/zplFgi3stv+mkfEf9WDQo1g5bueFJ+gK8= peterdemin@MBA
EOF
test -d ~builder/repo.git || sudo -u builder -s /bin/bash -c "git init --bare ~builder/repo.git"
test -d ~builder/pages.git || sudo -u builder -s /bin/bash -c "git init --bare ~builder/pages.git && git -C ~builder/pages.git remote add origin pages@demin-dev.tail13c89.ts.net:repo.git"
test -d ~builder/venv || sudo -u builder -s /bin/bash -c "python3 -m venv ~builder/venv"
test -f ~builder/.ssh/id_ed25519 || sudo -u builder -s /bin/bash -c 'ssh-keygen -t ed25519 -f ~builder/.ssh/id_ed25519 -N ""'
echo "Copy public key to serving host:"
sudo -u builder -s /bin/bash -c 'ssh-keygen -yf ~builder/.ssh/id_ed25519'
echo

sudo -u builder /bin/bash -c 'git config --global user.email "builder@demin.dev"'
sudo -u builder /bin/bash -c 'git config --global user.name "Builder Bot"'
sudo -u builder /bin/bash -c 'git config --global init.defaultBranch master'
sudo -u builder /bin/bash -c 'ssh-keyscan -t ed25519 demin-dev.tail13c89.ts.net > ~/.ssh/known_hosts'

install -o builder -g builder -m 0700 -d ~builder/worktree
install -m 0755 /dev/stdin ~builder/repo.git/hooks/post-receive <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

HOME=/home/builder
WORK_TREE="$HOME/worktree"
LIVE_DIR="/var/www/site"

BRANCH="master"
BRANCH_REF="refs/heads/$BRANCH"

read oldrev newrev REFNAME
if [[ "$REFNAME" != "$BRANCH_REF" ]]; then
  echo "Ignoring push to $REFNAME (only deploys $BRANCH_REF)"
  exit 0
fi

git --git-dir="$HOME/repo.git" --work-tree="$WORK_TREE" checkout -f $BRANCH

cd "$WORK_TREE"
. $HOME/venv/bin/activate
make install lightweight compress pages
EOF

And that’s how I cut my website publish time from 5 minutes down to 10 seconds.

Setting up the serving host is similar, except that instead of building it just needs to check out.

provision_pages.sh:

#!/bin/bash
set -euo pipefail

apt-get update
apt-get install -y git

id pages || useradd -rms /usr/bin/git-shell pages
install -o pages -g pages -m 0700 -d ~pages/.ssh
install -o pages -g www-data -m 0750 -d /var/www/pages
install -o pages -g pages -m 0600 /dev/stdin ~pages/.ssh/authorized_keys <<'EOF'
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILG64GMcBIxl4rGuRum2n07Kf7dE9CUlzLl84e/TWvTM builder@trixie
EOF
test -d ~pages/repo.git || sudo -u pages /bin/bash -c "git init --bare ~pages/repo.git"
install -m 0755 /dev/stdin ~pages/repo.git/hooks/post-receive <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

git --git-dir="/home/pages/repo.git" --work-tree="/var/www/pages" checkout -f master
EOF

# 8< - - - - - Abort if nginx config already exist - - - - -
test -f /etc/nginx/sites-available/pages && exit 0

cat > /etc/nginx/sites-available/pages <<'EOF'
server {
    server_name pages.demin.dev;
    root /var/www/pages;
    index index.html index.htm;
    location / {
		gzip_static on;
        try_files $uri $uri/ =404;
    }
}
EOF
ln -fs /etc/nginx/sites-available/pages /etc/nginx/sites-enabled/pages
certbot --agree-tos --nginx -m peter@demin.dev --non-interactive -d pages.demin.dev
systemctl restart nginx.service

I added a flag to nginx to serve static precompressed gzip files to save CPU on the serving side.