Daniele
905fc54775
Nightly Build / build (push) Successful in 9m28s
The previous commit tried to fix the mtime invalidation with `git restore-mtime`. It does not work on this box, in the worst way: the packaged version (2022.12) drives `git whatchanged`, which git 2.53 refuses to run without --i-still-use-this — and the tool swallows that failure and exits 0 having updated nothing. Verified on the runner: "0 commits evaluated, 675 files missing, 0 files updated", while the job happily went on to rebuild everything. Fix the cause instead of the symptom. Both building workflows now sync a tree that survives between runs and build there. `git checkout` only rewrites files whose content actually changed, so mtimes are correct as a consequence rather than as a reconstruction — and no external tool is involved. Gitea serves this repo from the same machine the runner runs on, so the sync reads the bare repo directly: no network, no token. Two properties this buys that restore-mtime did not: - The absolute source path is pinned. The runner derives its workspace path from the job definition, so editing a workflow moved it and invalidated every workspace crate by itself — the previous commit paid that cost without knowing it. - It cannot fail towards staleness. Checking out an older commit stamps those files newer, which costs an extra rebuild; restore-mtime moved mtimes backwards, which could have let cargo reuse artifacts built from newer code. Each workflow gets its own tree, for the same reason they already have their own CARGO_TARGET_DIR: they track different branches, and one shared tree would rewrite half the files on every switch.
145 lines
6.5 KiB
YAML
145 lines
6.5 KiB
YAML
name: Nightly Build
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- main
|
|
|
|
# A push that lands while a nightly is still building makes that build obsolete:
|
|
# the nightly publishes to a fixed filename, so only the last one survives
|
|
# anyway. The runner has capacity 1, so without this a second push waits out a
|
|
# full 8-minute build whose tarball is overwritten minutes later. Cancelling
|
|
# keeps the queue one deep and the published nightly always the newest commit.
|
|
concurrency:
|
|
group: nightly
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
build:
|
|
runs-on: linux-amd64
|
|
|
|
env:
|
|
CARGO_TARGET_DIR: /home/dguiducci/.cache/skald-ci/target
|
|
# The persistent build tree — see the sync step. Kept separate from the
|
|
# release workflow's: the two track different branches, and one shared
|
|
# tree would rewrite half the files on every switch, which is exactly the
|
|
# mtime churn this whole arrangement removes.
|
|
SRC: /home/dguiducci/.cache/skald-ci/src-nightly
|
|
# Release builds have incremental compilation OFF by default, which is the
|
|
# worst case for this tree: skald-core is 51k lines in one crate, so a
|
|
# one-line change recodegens all of it. The nightly trades a marginally
|
|
# less optimised binary for the rebuild time. The release workflow
|
|
# deliberately does NOT set this — there the binary quality wins.
|
|
CARGO_INCREMENTAL: 1
|
|
|
|
steps:
|
|
# Deliberately not actions/checkout. Cargo decides what to recompile by
|
|
# mtime, and the runner deletes its own workspace after every job — so a
|
|
# fresh clone stamps every source file with "now" and all 20 workspace
|
|
# crates rebuilt on every run whatever the commit touched. Measured on a
|
|
# commit that only changed web/*.js: 20 of 722 rlibs rebuilt, i.e. the
|
|
# ~700 third-party deps stayed cached (their sources live in
|
|
# ~/.cargo/registry, with stable mtimes) and our own code never did.
|
|
#
|
|
# A tree that survives between runs fixes it at the source: `git checkout`
|
|
# only rewrites files whose content actually changed, so everything else
|
|
# keeps its mtime and cargo skips it. No external tool is involved — note
|
|
# that the obvious alternative, `git restore-mtime`, is a trap here: the
|
|
# packaged version drives the deprecated `git whatchanged`, which git 2.53
|
|
# refuses to run, and it reports that failure by exiting 0 having updated
|
|
# nothing.
|
|
#
|
|
# This also pins the absolute source path, which the runner's workspace
|
|
# does not: that path is derived from the job definition, so every edit to
|
|
# this file moved it and invalidated every workspace crate on its own.
|
|
#
|
|
# Note which way this fails: checking out an older commit stamps those
|
|
# files *newer*, which can only cost an extra rebuild — it can never let
|
|
# cargo reuse an artifact built from newer code.
|
|
- name: Sync the persistent build tree
|
|
run: |
|
|
set -eu
|
|
# Gitea serves this repo from the same machine the runner runs on, so
|
|
# the tree syncs straight off the bare repo: no network, no token.
|
|
ORIGIN=/home/dguiducci/skald/gitea/data/git/repositories/dguiducci/skald-circle.git
|
|
if [ ! -d "$SRC/.git" ]; then
|
|
mkdir -p "$(dirname "$SRC")"
|
|
git clone --no-checkout "$ORIGIN" "$SRC"
|
|
fi
|
|
cd "$SRC"
|
|
git remote set-url origin "$ORIGIN"
|
|
git fetch --prune --force origin
|
|
git checkout -f --detach "$GITHUB_SHA"
|
|
# Clear leftovers from the previous run (dist/ above all) so nothing
|
|
# stale can be packaged or deployed. Tracked files are untouched, and
|
|
# CARGO_TARGET_DIR lives outside this tree.
|
|
git clean -ffdxq
|
|
echo "[sync] $(git log --oneline -1)"
|
|
|
|
- name: Build native (linux/amd64)
|
|
run: |
|
|
cd "$SRC"
|
|
RUSTFLAGS="-A warnings" cargo build --release --no-default-features
|
|
RUSTFLAGS="-A warnings" cargo build --release --no-default-features -p skald-setup
|
|
|
|
- name: Cross-compile (linux/arm64)
|
|
env:
|
|
CC_aarch64_unknown_linux_gnu: aarch64-linux-gnu-gcc
|
|
AR_aarch64_unknown_linux_gnu: aarch64-linux-gnu-ar
|
|
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER: aarch64-linux-gnu-gcc
|
|
run: |
|
|
cd "$SRC"
|
|
RUSTFLAGS="-A warnings" cargo build --release --no-default-features --target aarch64-unknown-linux-gnu
|
|
RUSTFLAGS="-A warnings" cargo build --release --no-default-features -p skald-setup --target aarch64-unknown-linux-gnu
|
|
|
|
- name: Package amd64
|
|
run: |
|
|
cd "$SRC"
|
|
./ci/package.sh \
|
|
--version nightly \
|
|
--os linux \
|
|
--arch amd64 \
|
|
--target-dir "$CARGO_TARGET_DIR/release" \
|
|
--output dist/
|
|
|
|
- name: Package arm64
|
|
run: |
|
|
cd "$SRC"
|
|
./ci/package.sh \
|
|
--version nightly \
|
|
--os linux \
|
|
--arch arm64 \
|
|
--target-dir "$CARGO_TARGET_DIR/aarch64-unknown-linux-gnu/release" \
|
|
--output dist/
|
|
|
|
- name: Deploy to builds.skaldagent.net
|
|
run: |
|
|
cd "$SRC"
|
|
DEST=/var/www/builds.skaldagent.net/nightly
|
|
mkdir -p "$DEST"
|
|
# Nightly reuses a fixed filename, so publish atomically: copy to a
|
|
# temp name on the same filesystem, then rename over the target. A
|
|
# concurrent download never sees a half-written tarball.
|
|
for f in dist/*.tar.gz; do
|
|
name="$(basename "$f")"
|
|
cp "$f" "$DEST/.$name.tmp"
|
|
mv -f "$DEST/.$name.tmp" "$DEST/$name"
|
|
done
|
|
echo "[nightly] Deployed:"
|
|
ls -lh "$DEST/"
|
|
|
|
- name: Publish the nightly installer
|
|
run: |
|
|
cd "$SRC"
|
|
# install-nightly.sh is served straight from the web root
|
|
# (curl -fsSL https://builds.skaldagent.net/install-nightly.sh | bash),
|
|
# so without this it stays whatever was copied there by hand and drifts
|
|
# from the repo — a fix to the installer would reach every existing box
|
|
# through update.sh but never a new one. Same atomic publish as the
|
|
# tarballs: a client mid-download never sees a half-written script.
|
|
ROOT=/var/www/builds.skaldagent.net
|
|
cp install-nightly.sh "$ROOT/.install-nightly.sh.tmp"
|
|
chmod 644 "$ROOT/.install-nightly.sh.tmp"
|
|
mv -f "$ROOT/.install-nightly.sh.tmp" "$ROOT/install-nightly.sh"
|
|
echo "[nightly] Published install-nightly.sh"
|