ci: stop rebuilding the whole workspace on every run
Nightly Build / build (push) Canceled after 6m16s
Nightly Build / build (push) Canceled after 6m16s
Cargo decides freshness by mtime, and the Gitea runner deletes the job workspace after each run. So `actions/checkout` stamped every source file with "now" and all 20 workspace crates recompiled regardless of what the commit touched: measured on a JS-only commit, 20 of 722 rlibs rebuilt — the ~700 third-party deps stayed cached, our own code never did. That, not the size of skald-core, was the 4 minutes per architecture. Restore mtimes from git history after checkout (needs the full history, hence fetch-depth: 0 — cheap here, ~170 commits against a Gitea instance on the same machine). Also: - nightly: CARGO_INCREMENTAL=1. Release builds have incremental off by default, the worst case for a 51k-line crate. The nightly trades a marginally less optimised binary for the rebuild time; release does not. - nightly: concurrency with cancel-in-progress. The runner has capacity 1 and the nightly publishes to a fixed filename, so a queued build was 8 minutes spent on a tarball the next one overwrites. - release: its own CARGO_TARGET_DIR. CARGO_INCREMENTAL is part of cargo's profile fingerprint, so one shared cache between a workflow that sets it and one that does not would have each invalidate the other's workspace crates — reintroducing the very rebuild this removes. - packaging steps derive --target-dir from $CARGO_TARGET_DIR instead of repeating the path, so the two cannot drift.
This commit is contained in:
@@ -5,15 +5,50 @@ on:
|
||||
branches:
|
||||
- main
|
||||
|
||||
# A push that lands while a nightly is still building makes that build obsolete:
|
||||
# the nightly publishes to a fixed filename, so only the last one survives
|
||||
# anyway. The runner has capacity 1, so without this a second push waits out a
|
||||
# full 8-minute build whose tarball is overwritten minutes later. Cancelling
|
||||
# keeps the queue one deep and the published nightly always the newest commit.
|
||||
concurrency:
|
||||
group: nightly
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: linux-amd64
|
||||
|
||||
env:
|
||||
CARGO_TARGET_DIR: /home/dguiducci/.cache/skald-ci/target
|
||||
# Release builds have incremental compilation OFF by default, which is the
|
||||
# worst case for this tree: skald-core is 51k lines in one crate, so a
|
||||
# one-line change recodegens all of it. The nightly trades a marginally
|
||||
# less optimised binary for the rebuild time. The release workflow
|
||||
# deliberately does NOT set this — there the binary quality wins.
|
||||
CARGO_INCREMENTAL: 1
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# `git restore-mtime` dates each file from the last commit that
|
||||
# touched it, so it needs the full history. A shallow clone would date
|
||||
# every file at HEAD and defeat the whole point. Cheap here: ~170
|
||||
# commits, and the Gitea instance runs on this same machine.
|
||||
fetch-depth: 0
|
||||
|
||||
# Cargo decides what to recompile by mtime, and the runner deletes the job
|
||||
# workspace after every run — so a fresh clone stamps every source file
|
||||
# with "now" and all 20 workspace crates rebuild even on a commit that
|
||||
# only touched web/*.js. Measured: 20 of 722 rlibs rebuilt on a JS-only
|
||||
# commit, i.e. the ~700 third-party deps stay cached and our own code
|
||||
# never does. Restoring mtimes from git history is what makes
|
||||
# CARGO_TARGET_DIR actually cache the workspace and not just the deps.
|
||||
#
|
||||
# Consequence to respect: do NOT reuse this target dir to rebuild an older
|
||||
# commit. Mtimes would go backwards and cargo could treat artifacts built
|
||||
# from newer code as fresh. Use a separate CARGO_TARGET_DIR for that.
|
||||
- name: Restore source mtimes from git history
|
||||
run: git restore-mtime
|
||||
|
||||
- name: Build native (linux/amd64)
|
||||
run: |
|
||||
@@ -36,7 +71,7 @@ jobs:
|
||||
--version nightly \
|
||||
--os linux \
|
||||
--arch amd64 \
|
||||
--target-dir /home/dguiducci/.cache/skald-ci/target/release \
|
||||
--target-dir "$CARGO_TARGET_DIR/release" \
|
||||
--output dist/
|
||||
|
||||
- name: Package arm64
|
||||
@@ -46,7 +81,7 @@ jobs:
|
||||
--version nightly \
|
||||
--os linux \
|
||||
--arch arm64 \
|
||||
--target-dir /home/dguiducci/.cache/skald-ci/target/aarch64-unknown-linux-gnu/release \
|
||||
--target-dir "$CARGO_TARGET_DIR/aarch64-unknown-linux-gnu/release" \
|
||||
--output dist/
|
||||
|
||||
- name: Deploy to builds.skaldagent.net
|
||||
|
||||
@@ -29,10 +29,28 @@ jobs:
|
||||
version: ${{ steps.extract-version.outputs.version }}
|
||||
|
||||
env:
|
||||
CARGO_TARGET_DIR: /home/dguiducci/.cache/skald-ci/target
|
||||
# Deliberately NOT the nightly's target dir. No CARGO_INCREMENTAL here —
|
||||
# a release binary is the one people install, so it gets the fully
|
||||
# optimised non-incremental build — and that flag is part of cargo's
|
||||
# profile fingerprint. Sharing one cache between a workflow that sets it
|
||||
# and one that doesn't would make each run invalidate the other's
|
||||
# workspace crates, which is exactly the cost this whole change removes.
|
||||
CARGO_TARGET_DIR: /home/dguiducci/.cache/skald-ci/target-release
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
# Full history: `git restore-mtime` below dates each file from the
|
||||
# last commit that touched it.
|
||||
fetch-depth: 0
|
||||
|
||||
# See the long note in nightly.yml: the runner wipes the job workspace
|
||||
# after every run, so without this every source file is stamped "now" on
|
||||
# checkout and all 20 workspace crates rebuild from scratch regardless of
|
||||
# what the commit changed. This is what lets CARGO_TARGET_DIR cache our
|
||||
# own crates and not just the ~700 third-party deps.
|
||||
- name: Restore source mtimes from git history
|
||||
run: git restore-mtime
|
||||
|
||||
- name: Extract version from Cargo.toml
|
||||
id: extract-version
|
||||
@@ -66,7 +84,7 @@ jobs:
|
||||
--version "${{ steps.extract-version.outputs.version }}" \
|
||||
--os linux \
|
||||
--arch amd64 \
|
||||
--target-dir /home/dguiducci/.cache/skald-ci/target/release \
|
||||
--target-dir "$CARGO_TARGET_DIR/release" \
|
||||
--output dist/
|
||||
|
||||
- name: Package arm64
|
||||
@@ -76,7 +94,7 @@ jobs:
|
||||
--version "${{ steps.extract-version.outputs.version }}" \
|
||||
--os linux \
|
||||
--arch arm64 \
|
||||
--target-dir /home/dguiducci/.cache/skald-ci/target/aarch64-unknown-linux-gnu/release \
|
||||
--target-dir "$CARGO_TARGET_DIR/aarch64-unknown-linux-gnu/release" \
|
||||
--output dist/
|
||||
|
||||
- name: Deploy to builds.skaldagent.net
|
||||
|
||||
Reference in New Issue
Block a user