StarlingTek

How it works

Simple parts, put together carefully

StarlingTek runs on one DigitalOcean droplet. Caddy takes every request and handles HTTPS. Each site is a Docker container that only listens on the server's loopback address. A control plane StarlingTek wrote deploys, watches and recovers them.

The architecture

StarlingTek architecture Visitors reach Caddy on ports 80 and 443 of one DigitalOcean droplet. Caddy routes each domain to its site's container on loopback. A control plane on the same droplet manages the containers, takes deploys from GitHub, sends alerts to ntfy and Discord, and backs databases on a managed Postgres cluster up to DigitalOcean Spaces. One DigitalOcean droplet Visitors HTTPS Caddy :80 / :443 · automatic HTTPS · routes by domain Site A container · loopback Site B container · loopback Site C container · loopback Control plane deploys · health checks · rollback · backups · stk and the admin panel GitHub webhook · poll Alerts ntfy · Discord Managed Postgres off the droplet DigitalOcean Spaces database backups
Only ports 22, 80 and 443 are open. Sites, the control plane's API and the admin panel's backend all listen on loopback.

The parts

Caddy
Terminates TLS, gets and renews Let's Encrypt certificates, and routes each request to the right site by its hostname.
One container per site
Each site is a Docker container, replaced whole on every deploy. Containers run as non-root, with dropped capabilities, and publish only to 127.0.0.1.
The control plane
A Node service under systemd that deploys, runs the 30-second health loop, rolls back, runs backups and sends alerts. Its API listens on loopback behind a bearer token.
stk and the admin panel
stk is a thin command-line client for that API. The admin panel forwards an allowlist of the same operations, behind its own sign-in.
Managed Postgres
Databases live off the server on a managed cluster, with their own backups, so rolling back code or rebuilding the server leaves the data where it is.
Spaces
Scheduled database backups are copied to DigitalOcean Spaces, so a copy exists away from the server.

Deploy

A deploy, step by step

Each site has two ports on loopback: one for the live container, one for the candidate.

  1. Resolve

    Fetch from GitHub and pin the branch tip to an exact commit.

  2. Build

    Archive that commit into a clean directory and build an image. A Dockerfile is generated when the repository doesn't have one.

  3. Start

    Run the image as a new container on the site's free port, next to the live one.

  4. Probe

    Call the health route on loopback until it answers. If it never does, the candidate is removed and the live site never noticed.

  5. Cut over

    Point Caddy at the new container, reload, let the old one drain for a few seconds, then remove it.

  6. Keep

    The image stays on the server, so it can be rolled back to later without a rebuild.

Monitor

Two views of every site

A loopback probe checks the container itself. A public probe goes through DNS, TLS and Caddy like a visitor. When the first passes and the second fails, the problem is in front of the app, not the app, and the admin panel says so.

Recover

When a release goes bad

The supervisor restarts the container first. If it is still unhealthy, it rolls back to the last known-good release, with a cooldown so it can't loop. Either way an alert goes out, once.

What a site needs to run here

Static sites need nothing. A Node server needs three things:

  • Listen on process.env.PORT.
  • Answer a cheap health route, /healthz by default.
  • Commit a lockfile, so builds are repeatable.

Handling SIGTERM is a nice extra: it lets requests in flight finish during the cutover drain. This website does all four.