From b67cc1290e7c00ee4f6a4678d564cf9dc92a007a Mon Sep 17 00:00:00 2001 From: TheRON Date: Wed, 19 Aug 2026 01:10:08 -0500 Subject: [PATCH] verify.sh: reach the restore on the diff branch set -euo pipefail aborted the script on the diff pipeline one line before the cp that restores the pre-run oracle, so --full left a regenerated fixture file in the working tree while printing that nothing had been overwritten. Appended || true. Recorded as F-033. The fix is not yet exercised: the restore branch runs only under --full and has not been entered since the change. --- docs/FAILURES.md | 44 +++++++++++++++++++++++++++++ tools/reference-toolchain/verify.sh | 2 +- 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/docs/FAILURES.md b/docs/FAILURES.md index b01bdeb..c175770 100644 --- a/docs/FAILURES.md +++ b/docs/FAILURES.md @@ -728,6 +728,49 @@ specifies both; if CT 102 runs no containers, `keyctl` can be dropped. --- +### F-033 — a restore the tool announced and never performed +CT 100. Repository tooling, Shapely port gate. + +**Observed:** `bash tools/reference-toolchain/verify.sh --full` regenerated all 123 +cases identically and diffed in exactly two adjacent lines, `fixtures_sha256` and +`frozen` — the expected result. The script printed the `DIFFERS` banner and the +diff body, then exited 1. Its closing line, `The committed oracle has been restored. +Nothing was overwritten.`, **did not appear in the journal**. + +Afterwards the working tree carried a modified oracle: + +``` + M fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json +``` + +and `make_fixtures.py --verify` reported `recorded` and `computed` both +`577d6575...` — the regenerated hash — and declared `OK - oracle is intact`. +The committed value is `ddd0f154...`. + +**Cause:** **Proven.** The script runs under `set -euo pipefail`. On the diff branch +it executes `diff ... | head -40 >&2`. `diff` exits 1 when files differ, `pipefail` +propagates that past `head`, and `set -e` aborts the script there. The `cp` that +restores the pre-run copy is the next statement and never runs. The reassurance is +printed after the `cp`, so it is unreachable on the only branch that prints it. + +**Correction:** `|| true` appended to that pipeline. The working tree was restored +with `git checkout --` on the fixture file. Nothing was lost: the committed bytes +were in Gitea throughout. + +**Consequence:** Two things. + +**A tool's claim that it did something is not evidence that it did.** Verify the +effect, not the announcement. This is the F-027 class again — the failure mode was +indistinguishable from success, and it printed the success message. + +**`make_fixtures.py --verify` cannot detect substitution of the whole file.** It +recomputes the hash from the document it reads and compares against the value stored +inside that same document, so a wholly regenerated oracle is self-consistent and +passes. It proves internal integrity, never identity with the committed oracle. Pair +it with `git status` whenever the question is *which* oracle is present. + +--- + ## Open, not closed | # | Status | @@ -747,5 +790,6 @@ specifies both; if CT 102 runs no containers, `keyctl` can be dropped. | F-030 | **Corrected** 2026-08-18. | | F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. | | F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. | +| F-033 | **Corrected** 2026-08-19. Restore path unreachable under `set -e`. | Everything else is closed with a proven cause and a proven correction. diff --git a/tools/reference-toolchain/verify.sh b/tools/reference-toolchain/verify.sh index 2e6fdb9..cd8be29 100644 --- a/tools/reference-toolchain/verify.sh +++ b/tools/reference-toolchain/verify.sh @@ -48,7 +48,7 @@ if [[ "${1:-}" == "--full" ]]; then echo "diff there is expected. Any other difference means the toolchain" >&2 echo "has drifted; investigate before proceeding." >&2 diff "${tmp}/before.json" \ - "${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" | head -40 >&2 + "${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" | head -40 >&2 || true cp "${tmp}/before.json" \ "${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" echo >&2