What CI costs on Hetzner versus GitHub Actions

Ryle · Aug 24, 2026

I run CI for a large Rust monorepo I work on, on my own Hetzner machines under Kubernetes, and per runner-hour it comes out several times cheaper than GitHub's hosted runners at list price. For a compile-heavy workload the gap gets wider as the runners get bigger. What took me longer to figure out is that most of the money and wall-clock went to things hosted runners would have charged for too: builds that failed, builds nobody would read, cold caches, and capacity sitting idle because my autoscaler wouldn't let go of it. Self-hosting made those visible and fixable, and also made them my problem.

The workload

The repository has a little under a thousand workspace crates. 784 of them depend on at least one generated schema crate, so a change near the bottom of the graph fans out across most of the tree. Over one week I measured, the CI lane used 1,409 runner-hours. cargo build inside those jobs accounted for 433 hours over 2,553 invocations, an average of 611 seconds each. Daily runner time swung between 120 and 320 hours. A lot of the pull request traffic comes from coding agents, so the load is bursty around the clock.

Two things about it drive everything else. The work is compile-bound, so CPU count and cache hit rate decide both speed and cost. It's also very cache-sensitive: in one benchmark a warm build of a small crate restored from cache took 14 seconds, against 140 seconds cold.

Memory mattered less than I'd assumed. Across 989,169 rustc invocations that week, median peak RSS was 118 MB, p99 was 1.35 GB and the largest compile hit 3.06 GB, so I'd been over-cautious running the heaviest jobs one compile at a time.

The architecture

The setup went through two versions. The first was pretty standard: Forgejo Actions runners as Kubernetes StatefulSets, in several size classes, on a mix of Hetzner dedicated servers and Hetzner Cloud nodes. The cloud nodes were CPX62s (16 shared vCPU, 32 GB) that a small autoscaler I wrote bought and released through the Hetzner Cloud API. It scaled the StatefulSets on queued jobs and added or removed nodes when pods couldn't be scheduled or nodes sat empty. Each runner node mounted a persistent target directory and a node-local compile cache.

The second version replaced the Actions lane for pull requests, and it separates where the job runs from where cargo runs. A build machine is a small HTTP server: it takes a branch, a commit and a cargo command, checks the commit out into a slot, runs cargo, and streams the output back with the exit code on the last line. Each slot owns a persistent target directory on local NVMe, so successive builds of the same area stay incremental. The builders run on dedicated servers, one per machine, and share a node-wide make jobserver so several slots can't oversubscribe the CPU. A small CLI that dispatches builds asks each builder for its queue depth and sends the request to the least busy one. Uncommitted files get sent along as an overlay.

When the dedicated builders are saturated, burst builders come up on autoscaled cloud nodes. I made the burst policy lazy on purpose because a cold builder is expensive: the first heavy build on a fresh node takes around 20 minutes even with cache hits. One burst builder starts when a request has waited 30 seconds, all of them start when a request has waited five minutes, and they're released after the dedicated fleet has gone 20 minutes without being saturated.

Pull request checks run once per pushed commit on long-lived workspace pods that keep warm target directories, and a planner picks which crates a change can affect, so most check runs only build a fraction of the tree. (That planner gets its own post.)

Caching

I started with sccache and switched to kache, a content-addressed rustc wrapper that restores finished artifacts by hardlink and also caches proc-macros and dylibs, which sccache treats as uncacheable. On the same wiped-target protocol:

The gap is smaller on the heavy binary because neither tool caches final linking. kache's store is about 4.4 times larger (5.7 GB against 1.3 GB for that binary) and it disables incremental compilation, which is fine for CI.

Sharing a cache across nodes took some work. A per-compile S3 lookup against Hetzner's object storage started returning 503s under CI load, so I moved to a batch approach: a push job uploads the store to a bucket with a 14-day expiry, and a seeder DaemonSet pulls the whole thing when an autoscaled node joins and every six hours after that. A fresh node is cold until that pull finishes. I caught one pair of new nodes 45 minutes after boot still partway through downloading 31,731 cache objects, and any job placed there in the meantime paid cold-build prices.

Hit rates depend a lot on how warm the node is. A 20-crate soak replaying recent PR units against a seeded builder hit 92.9% (92.3% weighted by compile cost) and avoided about 2.9 hours of compile work against 14 minutes of misses. The busy PR lane's baseline was 54.3% (42.7% weighted).

The cost comparison

My cost collector stamps two numbers on every job: a hosted equivalent (duration times a GitHub per-minute rate) and a self-hosted estimate (a fixed per-runner pod footprint of 2 cores and 4 GB, priced from my cluster's per-core-hour and per-GB-hour rate table). For that week it reported $140.5 self-hosted against $675 hosted. Both numbers need caveats. The self-hosted figure is an estimate built from the runner footprint and was never checked against a Hetzner invoice. The hosted figure used $0.008 per minute, the old 2-core Linux price; GitHub has since cut hosted runner prices, and the 2-core rate is now $0.006. It also sums raw job seconds, while GitHub rounds every job up to the next whole minute, which matters a lot for a workflow that fans out dozens of jobs that skip in two seconds.

Here's the same 1,409 hours repriced at current GitHub list prices (my arithmetic):

My heavy runners requested about 4 CPUs and 8.5 GB, so the 4-core row is the closest match, and it puts the hosted cost at roughly seven times the modeled self-hosted cost. Looking per machine gives the same answer. A CPX62 at Hetzner's current new-order price is $0.2452 an hour, about $0.0041 a minute for 16 shared vCPU, against $0.042 a minute for a 16-core GitHub runner. That's roughly a tenth of the price, before accounting for shared versus dedicated cores. A dedicated AX102 at the current $302.10 a month costs the same as about 61 hours of a 32-core GitHub runner (similar in thread count), so it pays for itself if it does around two hours of real work a day. On a busy day my three dedicated builders went over ten hours without a single minute where they were all empty.

Those ratios assume the capacity is busy, though, and mine often wasn't.

The hidden costs

Idle capacity. For most of one day the autoscaled pool sat at five CPX62s around the clock, roughly $12.63 a day or $380 a month at the price table I was using, while the whole runner fleet did about 15 core-hours of real work, under 1% of the 1,920 core-hours available. The autoscaler had computed the right target and wasn't acting on it. Three bugs combined here. The idle clock reset on any job, including jobs a runner claimed and skipped in two seconds, so it never expired. Scale-down removed one replica per pod termination, and an idle runner took about ten minutes to exit. And scale-up sized on the momentary burst of skip-jobs from each push, so a single push pinned the heaviest class at its maximum. I fixed it by gating only scale-to-zero on idleness, shrinking straight to the target, and sizing growth on the minimum demand over a 90-second window. Hosted runners don't bill you for any of that idle time.

Overbuying. A Hetzner Cloud node takes three to five minutes to become schedulable, and my scale-up cooldown was 90 seconds, so one pending pod could buy a node every cooldown window. Even after I started counting joining nodes, a node reads Ready about 45 seconds before the scheduler will place a pod on it, and that gap bought one more extra node. Restarting the autoscaler also reset its in-memory idle clock, and once that gave a node that had already been idle for about 40 minutes a fresh 30-minute window; I moved the clock onto the node as an annotation. The burst builders had their own version of this. They were only released after 20 minutes in which every builder was completely empty, which never happened on a busy day, so burst nodes stayed up for hours with nothing queued.

Work nobody reads. 1,050 of the 1,409 runner-hours, 75%, went to runs that ended in failure. On the newer lane I found four check runs going for one pull request, one per push, when only the newest result mattered, and another one still compiling for a PR that had already merged. Killing that stale work took one machine's load average from 29 to 9. At GitHub's 2-core rate the failed hours alone come to about $380 a week. Superseding stale runs and failing fast saved more than any pricing decision did.

Prices change. Hetzner raised prices for new orders and rescales, and the CPX62 went from $59.49 to $152.99 a month. An autoscaler creates new servers all the time, so this hits autoscaled pools directly, and my internal price table still had the old rate. If those autoscaled nodes were billed at the new rate, which I expect they were, the dashboards understated their cost by more than half. The comparison still favors Hetzner by a wide margin, just a smaller one than before the increase.

Observability. For a while the main compile cache had no hit-rate telemetry at all. Also, inside a container nproc and load average report the host: one pod saw 32 cores and a load of 24 to 31, while its cgroup quota was 16 CPUs. One check run drew about 4.3 CPUs, so the lane was already full.

Reliability and ops time. When a template changed, my pod controller drained all of the CI pods at once and killed in-flight runs, until I capped it to one at a time. Hosted runners spare you this kind of thing. I didn't track the hours I spent on it, so I can't honestly put a price on it, but it was a lot.

Distance. My dev machine is on the US west coast and the cluster is in Europe, about 150 ms away. The remote builders sit next to the registry and object storage, so release binaries and image layers never cross the Atlantic. What does cross is mostly a source overlay and compiler output.

Would I do it again

Yes, for this workload. Compile-heavy CI on large machines is where per-minute pricing hurts most, and a warm builder with a persistent target directory and a local cache is faster than any cold runner. But I only saw the savings once I had telemetry for the cache and for real CPU use, cancellation of stale work, an autoscaler that gives capacity back as readily as it buys it, and a price table I actually keep up to date. If I set this up again I'd size dedicated servers for the steady load first, add cloud burst capacity after that, and expect to spend real time on the scaler.

Sources

• GitHub Docs: Actions runner pricing (current per-minute rates and per-job rounding)
• GitHub Changelog: Update to GitHub Actions pricing (January 2026 price reduction)
• Hetzner Docs: Price adjustment, 15 June 2026 (CPX62 and AX102-1 prices)