verify.sh: reach the restore on the diff branch

set -euo pipefail aborted the script on the diff pipeline one line
before the cp that restores the pre-run oracle, so --full left a
regenerated fixture file in the working tree while printing that
nothing had been overwritten. Appended || true.

Recorded as F-033. The fix is not yet exercised: the restore branch
runs only under --full and has not been entered since the change.
This commit is contained in:
2026-08-19 01:10:08 -05:00
parent 1d3eed07cd
commit b67cc1290e
2 changed files with 45 additions and 1 deletions
+44
View File
@@ -728,6 +728,49 @@ specifies both; if CT 102 runs no containers, `keyctl` can be dropped.
--- ---
### F-033 — a restore the tool announced and never performed
CT 100. Repository tooling, Shapely port gate.
**Observed:** `bash tools/reference-toolchain/verify.sh --full` regenerated all 123
cases identically and diffed in exactly two adjacent lines, `fixtures_sha256` and
`frozen` — the expected result. The script printed the `DIFFERS` banner and the
diff body, then exited 1. Its closing line, `The committed oracle has been restored.
Nothing was overwritten.`, **did not appear in the journal**.
Afterwards the working tree carried a modified oracle:
```
M fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json
```
and `make_fixtures.py --verify` reported `recorded` and `computed` both
`577d6575...` — the regenerated hash — and declared `OK - oracle is intact`.
The committed value is `ddd0f154...`.
**Cause:** **Proven.** The script runs under `set -euo pipefail`. On the diff branch
it executes `diff ... | head -40 >&2`. `diff` exits 1 when files differ, `pipefail`
propagates that past `head`, and `set -e` aborts the script there. The `cp` that
restores the pre-run copy is the next statement and never runs. The reassurance is
printed after the `cp`, so it is unreachable on the only branch that prints it.
**Correction:** `|| true` appended to that pipeline. The working tree was restored
with `git checkout --` on the fixture file. Nothing was lost: the committed bytes
were in Gitea throughout.
**Consequence:** Two things.
**A tool's claim that it did something is not evidence that it did.** Verify the
effect, not the announcement. This is the F-027 class again — the failure mode was
indistinguishable from success, and it printed the success message.
**`make_fixtures.py --verify` cannot detect substitution of the whole file.** It
recomputes the hash from the document it reads and compares against the value stored
inside that same document, so a wholly regenerated oracle is self-consistent and
passes. It proves internal integrity, never identity with the committed oracle. Pair
it with `git status` whenever the question is *which* oracle is present.
---
## Open, not closed ## Open, not closed
| # | Status | | # | Status |
@@ -747,5 +790,6 @@ specifies both; if CT 102 runs no containers, `keyctl` can be dropped.
| F-030 | **Corrected** 2026-08-18. | | F-030 | **Corrected** 2026-08-18. |
| F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. | | F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. |
| F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. | | F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. |
| F-033 | **Corrected** 2026-08-19. Restore path unreachable under `set -e`. |
Everything else is closed with a proven cause and a proven correction. Everything else is closed with a proven cause and a proven correction.
+1 -1
View File
@@ -48,7 +48,7 @@ if [[ "${1:-}" == "--full" ]]; then
echo "diff there is expected. Any other difference means the toolchain" >&2 echo "diff there is expected. Any other difference means the toolchain" >&2
echo "has drifted; investigate before proceeding." >&2 echo "has drifted; investigate before proceeding." >&2
diff "${tmp}/before.json" \ diff "${tmp}/before.json" \
"${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" | head -40 >&2 "${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" | head -40 >&2 || true
cp "${tmp}/before.json" \ cp "${tmp}/before.json" \
"${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" "${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json"
echo >&2 echo >&2