verify.sh: reach the restore on the diff branch
set -euo pipefail aborted the script on the diff pipeline one line before the cp that restores the pre-run oracle, so --full left a regenerated fixture file in the working tree while printing that nothing had been overwritten. Appended || true. Recorded as F-033. The fix is not yet exercised: the restore branch runs only under --full and has not been entered since the change.
This commit is contained in:
@@ -728,6 +728,49 @@ specifies both; if CT 102 runs no containers, `keyctl` can be dropped.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### F-033 — a restore the tool announced and never performed
|
||||||
|
CT 100. Repository tooling, Shapely port gate.
|
||||||
|
|
||||||
|
**Observed:** `bash tools/reference-toolchain/verify.sh --full` regenerated all 123
|
||||||
|
cases identically and diffed in exactly two adjacent lines, `fixtures_sha256` and
|
||||||
|
`frozen` — the expected result. The script printed the `DIFFERS` banner and the
|
||||||
|
diff body, then exited 1. Its closing line, `The committed oracle has been restored.
|
||||||
|
Nothing was overwritten.`, **did not appear in the journal**.
|
||||||
|
|
||||||
|
Afterwards the working tree carried a modified oracle:
|
||||||
|
|
||||||
|
```
|
||||||
|
M fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json
|
||||||
|
```
|
||||||
|
|
||||||
|
and `make_fixtures.py --verify` reported `recorded` and `computed` both
|
||||||
|
`577d6575...` — the regenerated hash — and declared `OK - oracle is intact`.
|
||||||
|
The committed value is `ddd0f154...`.
|
||||||
|
|
||||||
|
**Cause:** **Proven.** The script runs under `set -euo pipefail`. On the diff branch
|
||||||
|
it executes `diff ... | head -40 >&2`. `diff` exits 1 when files differ, `pipefail`
|
||||||
|
propagates that past `head`, and `set -e` aborts the script there. The `cp` that
|
||||||
|
restores the pre-run copy is the next statement and never runs. The reassurance is
|
||||||
|
printed after the `cp`, so it is unreachable on the only branch that prints it.
|
||||||
|
|
||||||
|
**Correction:** `|| true` appended to that pipeline. The working tree was restored
|
||||||
|
with `git checkout --` on the fixture file. Nothing was lost: the committed bytes
|
||||||
|
were in Gitea throughout.
|
||||||
|
|
||||||
|
**Consequence:** Two things.
|
||||||
|
|
||||||
|
**A tool's claim that it did something is not evidence that it did.** Verify the
|
||||||
|
effect, not the announcement. This is the F-027 class again — the failure mode was
|
||||||
|
indistinguishable from success, and it printed the success message.
|
||||||
|
|
||||||
|
**`make_fixtures.py --verify` cannot detect substitution of the whole file.** It
|
||||||
|
recomputes the hash from the document it reads and compares against the value stored
|
||||||
|
inside that same document, so a wholly regenerated oracle is self-consistent and
|
||||||
|
passes. It proves internal integrity, never identity with the committed oracle. Pair
|
||||||
|
it with `git status` whenever the question is *which* oracle is present.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Open, not closed
|
## Open, not closed
|
||||||
|
|
||||||
| # | Status |
|
| # | Status |
|
||||||
@@ -747,5 +790,6 @@ specifies both; if CT 102 runs no containers, `keyctl` can be dropped.
|
|||||||
| F-030 | **Corrected** 2026-08-18. |
|
| F-030 | **Corrected** 2026-08-18. |
|
||||||
| F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. |
|
| F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. |
|
||||||
| F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. |
|
| F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. |
|
||||||
|
| F-033 | **Corrected** 2026-08-19. Restore path unreachable under `set -e`. |
|
||||||
|
|
||||||
Everything else is closed with a proven cause and a proven correction.
|
Everything else is closed with a proven cause and a proven correction.
|
||||||
|
|||||||
@@ -48,7 +48,7 @@ if [[ "${1:-}" == "--full" ]]; then
|
|||||||
echo "diff there is expected. Any other difference means the toolchain" >&2
|
echo "diff there is expected. Any other difference means the toolchain" >&2
|
||||||
echo "has drifted; investigate before proceeding." >&2
|
echo "has drifted; investigate before proceeding." >&2
|
||||||
diff "${tmp}/before.json" \
|
diff "${tmp}/before.json" \
|
||||||
"${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" | head -40 >&2
|
"${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json" | head -40 >&2 || true
|
||||||
cp "${tmp}/before.json" \
|
cp "${tmp}/before.json" \
|
||||||
"${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json"
|
"${REPO}/fixtures/strap-beam-8.0.0/strap-beam-fixtures-8.0.0.json"
|
||||||
echo >&2
|
echo >&2
|
||||||
|
|||||||
Reference in New Issue
Block a user