Accepted16 Nov 2026
0080. Host a public server from a Compose kit, behind Caddy or a Cloudflare Tunnel
Context and Problem Statement#
ArchSim is public, and its owner wants a live demo at https://app.archsim.io on one small Linux
server, as small as 2 cores and 4 GB, with the domain's DNS on Cloudflare. The pieces exist: release
images for amd64 and arm64
(ADR-0077), sign-in, workspace
limits and the outbound block (SECURITY.md), and a checklist in
deployment.md. But docker-compose.yml is a local
stack: no sign-in, ports on 127.0.0.1, the password archsim, no TLS and no backups, and the guide
said a hosted stack was still to come. What should a server run, how do visitors reach it, and how
does it go from an empty machine to a site, and from one release to the next?
Decision Drivers#
- Going live is: install Docker, clone the kit, fill in one file of settings, start it.
- Nothing but Docker and official images on the server; any provider, either processor.
- SECURITY.md's trust boundaries hold by construction, not by care: browser, the way in, web server, API, database.
- HTTPS without anyone renewing a certificate.
- Rate limits count each visitor's address, whichever proxy stands in front.
- The server's secrets stay in one file on the server, outside git.
- The release check that keeps
docker-compose.ymlat the release's version keeps the kit there too. - Upgrades and backups a person can run in one command, and a deploy from CI that cannot do more.
Considered Options#
- (a) A Compose kit in
deploy/: the release's images with sign-in, behind one way in that a Compose profile chooses, Caddy or a Cloudflare Tunnel, with a backup service, a script that moves the server to a release, and an optional SSH deploy workflow - (b) A hosted profile or override file of
docker-compose.yml - (c) Kubernetes manifests or a Helm chart
- (d) A platform's own configuration (Fly.io, Render and the like)
- For the way in: Caddy, nginx with certbot, Traefik, or a Cloudflare Tunnel
Decision Outcome#
Chosen option: (a), with Caddy and a Cloudflare Tunnel as its two ways in, because one file of settings and one command bring up the whole site on any server with Docker, the networks enforce the trust boundaries, and neither way in leaves a certificate to renew.
deploy/compose.yaml(projectarchsim-hosted, apart from the local stack'sarchsimand itsarchsim_pgdata) runs PostgreSQL 16 with a password from the settings and no published port, the migration once, the API withARCHSIM_AUTH_MODE=oauth, auto-migration and outbound calls off and the web server as its only trusted proxy, a worker, the web app withARCHSIM_TRUSTED_HOPS=1, the way in and a backup service. Its images are the release's, pinned to the version, as indocker-compose.yml. Runs are capped for a public demo at 1,000,000 expected requests and 120 s (against 2,000,000 and 300 locally), more than any template needs; the settings can change both. Every service restarts unless stopped, rotates its logs (five files of 10 MB) and has a memory limit.- Sized for 2 cores and 4 GB. The limits add up to 3.3 GB: the worker simulates two runs at once in a 1 GB heap, the API has 512 MB, and a JVM's limit is its heap and 384 MB. That is less a run than archsim-app's rule (512 MB a run and 256 MB besides), which is for runs of up to 2,000,000 requests; a run's heap shrinks by about 128 MB a million. On 4 cores and 8 GB, the settings give the worker three runs and the rule's 2 GB, and the API 1 GB; the guide has the values.
- Two ways in, one profile each.
COMPOSE_PROFILESin the settings starts one:caddy: Caddy (the official image, pinned) publishes 80, 443 and 443/udp, the only service that publishes a port.deploy/Caddyfileserveshttps://<domain>with a certificate from Let's Encrypt, the redirect from HTTP, HTTP/3 and compression, and proxies everything to the web server. It adds HSTS,nosniff, framing and referrer headers only where an answer lacks them, so the app's own headers, its Content-Security-Policy among them, stand.tunnel:cloudflared(the official image, pinned) runstunnel runwith the tunnel's token from the settings and connects out to Cloudflare, which serves HTTPS and sends the tunnel's public hostname tohttp://web:3000. The server publishes no port at all.
- The visitor's address. Either way the web server reads the last entry of
X-Forwarded-For(ARCHSIM_TRUSTED_HOPS=1) and the API believes only the web server (ARCHSIM_TRUSTED_PROXIES=web). Caddy replaces the header with the address it saw. Through the tunnel, Cloudflare appends the visitor's address to whatever the visitor sent, andcloudflaredpasses the header on as it is, as cloudflared#1426 shows; Cloudflare sends the same address alone asCF-Connecting-IP(Cloudflare's headers). A forged entry stays on the left and is ignored. - Three networks follow the trust boundaries:
edgeholds the way in and the web server,appthe web server and the API, anddata,internal(no way out), the database with the migration, the API, the workers and the backup. The API and the database are unreachable from the internet and from the way in.edgehas IPv6 on, so that Docker forwards IPv6 visitors to Caddy with their own address instead of its proxy's. deploy/hosted.env, fromhosted.env.example, holds the profile, the tunnel's token, the domain, the database password (openssl rand -hex 32, since it goes into a URL), the secret key, the OAuth app, the admin emails and the sizes. Git ignores it. Compose reads it with--env-file hosted.env,COMPOSE_PROFILESincluded, and gives each container only the variables it needs. It is not named.env, which Compose would read by itself: the repository keeps.env*names for a developer's local secrets, which its tooling and assistants never touch.deploy/backup.shruns in the official PostgreSQL image: apg_dumpat 03:00 UTC every night, written under a temporary name and renamed when complete, the newest seven kept in a volume.archsim-backup nowmakes one on demand, andpg_restorefrom the same container restores one.deploy/update.shmoves a server's clone to a release's tag (ormain), pulls its images and runsup -d, which migrates and restarts what changed, and recreates Caddy or the backup service when the release changed the file it mounts. When the images cannot be pulled, it goes back to the commit it was on..github/workflows/deploy-demo.ymlruns it for the project's demo, on a published release that is not a pre-release or by hand, with plainsshand the secrets of ademoenvironment. On the server, the key's forced command isupdate.sh, which accepts only a tag ormain, so the key can deploy and nothing else. The workflow has no token and is skipped unless the repository variableDEMO_ENABLEDistrue, as the CLA workflow is.- archsim-tools'
CheckReleasechecksdeploy/compose.yamlas it checksdocker-compose.yml: a release whose kit runs another version, or has no kit, does not publish.
Consequences#
- Good, because a server goes live in a few commands, which deployment.md lists from the firewall to a restore, on any provider.
- Good, because a mistake in the settings cannot open the API or the database: only Caddy has a port, and the database's network has no route out. Through the tunnel, the server has no web port at all, and its address can stay unknown.
- Good, because certificates are Caddy's or Cloudflare's job, renewed in the background.
- Bad, because it is one server: no failover, and a restart or an upgrade is a short outage. The backups live on its disk too, until someone copies them off.
- Bad, because every Compose command for the kit needs
--env-file hosted.env, and changing the way in means removing the old one's container by hand. - Bad, because through the tunnel the visitor's address is Cloudflare's word: anyone who could send requests to the web server other than through the tunnel could forge it. Nothing else can reach the web server, and turning on Cloudflare's "Remove visitor IP headers" would make every visitor count as one; the guide says to leave it off.
- Bad, because the workers receive the sign-in settings, OAuth secrets included: with sign-in on, the app checks them in every role, the worker's too. The workers sit on the network with no way out.
- Bad, because the images of Caddy and
cloudflaredare pinned by hand: Dependabot does not watchdeploy/. - Bad, because the IPv6 network needs Docker Engine 27 or later, and a Cloudflare proxy in front of Caddy would make the rate limits count Cloudflare's addresses (the guide says to use the tunnel instead).
- Bad, because the deploy user is in the
dockergroup, which is as good as root. The forced command narrows what the workflow's key can do, not what that user can.
Confirmation#
- archsim-app's
ComposeFileTests:deploy/compose.yamlruns both release images atArchsimVersion.PRODUCTand builds nothing; onlycaddypublishes ports;caddyandcloudflaredeach start with a profile of their own;edgeholds only them and the web server; PostgreSQL is only ondata, which is internal; the API and the workers run with sign-in; and the API believes only the web server about visitors' addresses. - archsim-tools'
CheckReleaseTests: a kit at another version, or none, stops a release;ReleaseWorkflowTestsruns the check on the checkout as it is. CI'sRulesrun the Java job whendeploy/compose.yamlchanges, and no job for the rest ofdeploy/. - Brought up locally on 2026-11-16 with
ARCHSIM_DOMAIN=localhost(Caddy's own certificate authority) and a dummy OAuth app, profilecaddy: the web app and/api/v1/healthanswered through Caddy over HTTP/2, HTTP redirected to HTTPS, the sign-in redirect named the public callback,GET /metareported the kit's limits, the API and PostgreSQL published no port and were unreachable from Caddy (and the database from the web server), the workers had no route out while the API reached GitHub, the API counted the address Caddy saw while spoofedX-Forwarded-ForandX-ArchSim-Client-IPheaders were ignored, and a dump was made, pruned to the newest and restored. With the default sizes, the worker simulated two runs of 950,000 requests at once (queueing-laboverloaded andsocial-feed-hot-key, scaled up) in 88 s; its memory peaked at 1.21 GB of its 1.375 GB limit, the heap grown to its 1 GB. Profiletunnelstartedcloudflaredonedgewith the token from the settings and no port on the host; a real tunnel needs a Cloudflare account, so the path from Cloudflare to the web server is first checked on the demo server (the guide's check of the rate-limit keys).
Pros and Cons of the Options#
(b) A hosted profile or override of docker-compose.yml#
- Good, because one file describes both stacks.
- Bad, because the local stack's choices (ports on
127.0.0.1, thearchsimpassword, no sign-in, the sharedarchsim_pgdata) would be overridden rather than absent, and Compose merges lists such asportsinstead of replacing them: a forgotten override would publish the API.
(c) Kubernetes or Helm#
- Good, because it scales past one machine and is what larger installs use.
- Bad, because one small server would first need a cluster (k3s or the like), an ingress, a certificate manager and a chart to maintain: far more than the demo needs.
(d) A platform's configuration#
- Good, because the platform handles TLS, restarts and sometimes the database.
- Bad, because it ties the kit to one provider and its prices, and most free tiers cannot run a JVM worker with a 1 GB heap beside the API's.
The ways in#
- Caddy gets and renews its certificates itself, with HTTP/2, HTTP/3 and the redirect in a few lines.
- nginx is familiar, but certbot is a second process with its own renewal timer and volume, and the configuration for TLS, HTTP/2, HTTP/3 and the redirect is several times longer.
- Traefik gets certificates itself too, but is configured through labels on each service or a file of its own, and reading labels needs the Docker socket, a privilege Caddy does not need.
- A Cloudflare Tunnel needs the domain on Cloudflare and makes Cloudflare part of the path, but the server opens no port, Cloudflare's edge serves TLS and absorbs floods, and no certificate lives on the server. Kept as the second way in rather than the only one, so that the kit runs without Cloudflare.
More Information#
- ADR-0014: one image, started as
migrate,apiorworker. - ADR-0053: the workers and their heap.
- ADR-0076 and ADR-0077: the version and the images the kit runs.
- docs/deployment.md, docs/ci.md and SECURITY.md.