diff --git a/docs/ACCEPTANCE.md b/docs/ACCEPTANCE.md new file mode 100644 index 0000000..85f24c8 --- /dev/null +++ b/docs/ACCEPTANCE.md @@ -0,0 +1,158 @@ +# ACCEPTANCE.md + +**What the acceptance suite asserts, and to what tolerance.** + +This document exists because until now the answer lived in two places, neither of +them readable: a `tolerance` block inside a hashed JSON fixture, and one line of +`test_oracle.py`. A person asking "how close does the port have to be?" had to +read code to find out, and the reason for the numbers was nowhere. + +Last updated 2026-08-22, closing F-034. + +--- + +## 1. What is being compared + +The port is measured against a frozen 123-case oracle generated by the OpenSCAD +reference at revision 8.0.0, under OpenSCAD 2021.01 with BOSL2 `92d697c2`. 113 +cases the reference accepted, 10 it rejected. + +The oracle does **not** record full-precision geometry. It records what +OpenSCAD's `echo` printed, which is C's `%g` at default precision — **six +significant figures**. That fact is load-bearing and most of this document +follows from it. + +## 2. Two tolerance regimes + +**Regime A — declared absolute.** The oracle's own `tolerance` block: +`lengths_mm = 1e-4`, `areas_mm2 = 1e-3`. Applies to every compared float except +the three keys named below. + +**Regime B — six-significant-figure last place.** Applies to +`SECTION_AREA_MM2`, `VOLUME_MM3` and `MASS_G` only. + +``` +ulp(v) = 10 ** (floor(log10(|v|)) - 5) the value of the last digit + the reference actually wrote down +bound = 8 * ulp(expected) +``` + +Concretely, across the oracle's observed magnitudes: + +| key | magnitude | last place | worst observed | bound | +|---|---|---|---|---| +| `SECTION_AREA_MM2` | 117 – 474 mm² | 0.001 | 0.003 (3 ULP) | 0.008 | +| `VOLUME_MM3` | 11,742 – 47,364 mm³ | 0.1 | 0.3 (3 ULP) | 0.8 | +| `MASS_G` | 14.6 – 58.7 g | 0.0001 | 0.0005 (5 ULP) | 0.0008 | + +The bound scales with magnitude by construction, so it needs no revision if a +future case is an order of magnitude larger or smaller. + +## 3. Expectation statements + +These are what a passing suite means. Each is asserted by a test. + +**E-1 — Every accepted case builds.** `build(family, profile, params)` returns a +`Result` for all 113. A raise is a failure, not a rejection. + +**E-2 — Every rejected case is rejected.** All 10 raise `ProfileRejected`, and +the message is non-empty and names the parameter and the limit. Reproducing the +geometry while accepting a case the reference refused is not a passing port. + +**E-3 — Counts are exact.** `SECTION_PARTS`, `STRAP_CHANNELS` and +`BUNDLE_COUNT` match with zero tolerance. + +**E-4 — Everything that positions material is exact to regime A.** +`ENVELOPE_X_MM`, `ENVELOPE_Y_MM`, `MIN_WALL_ACTUAL_MM` and every profile extra +(`AF_*`, `FIN_*`, `SPOKE_*`, `RING_*`, `T_*`) agree within 1e-4 mm. + +Measured: **exact in all 123 cases, no exceptions.** This is the strongest claim +the project makes and regime B must never be extended to cover any of it. If a +future change makes one of these keys need a looser bound, the change is wrong. + +**E-5 — The three discretisation-limited keys agree to regime B.** +`SECTION_AREA_MM2`, `VOLUME_MM3` and `MASS_G` agree within 8 ULP of the oracle's +six-significant-figure record. + +These three are one underlying quantity reported three times: volume is area × +100 mm, mass is volume × 1.24 g/cm³ ÷ 1000. Verified across all 113 accepted +cases — `VOLUME/AREA` is exactly 100.0 with a maximum deviation of 2.8e-14, and +`MASS/VOLUME` is uniform to 1e-8. One bound therefore governs all three +honestly, rather than three unrelated policies. + +**E-6 — The tolerance policy is not vacuous.** For every compared value, the +bound in force is at most 1e-4 relative. This is asserted directly, so a future +edit that loosens the comparison until it stops catching regressions fails a +test rather than passing silently. + +**E-7 — The oracle is unmodified.** Its recorded hash covers the whole document +except the hash field. Editing the fixture to make a test pass breaks this by +design. + +**E-8 — The 2D path imports no CAD kernel.** Building a cross-section must not +pull in `cadquery`, `OCP` or `build123d`. + +**E-9 — The suite is green.** There are no expected failures. A red `make test` +means something is wrong. + +## 4. Why the accuracy criterion does not source these numbers + +The project's stated accuracy criterion is 0.01 mm over the entire set +(`PRECISION.md`). The obvious move — bound the area error by +`perimeter × 0.01 mm` — was measured on 20260822 and rejected. + +Re-expressing every measured area discrepancy as the uniform boundary +displacement that would produce it gives `|dA| / P`, which is directly +comparable to the criterion: + +``` +median 0.000e+00 mm +p95 9.075e-06 mm +max 1.434e-05 mm one seven-hundredth of the criterion +``` + +A `perimeter × 0.01` bound would be 1.86 mm² at the smallest section and +4.54 mm² at the largest — between 1,800 and 4,500 times the worst real +discrepancy. It would catch nothing. + +Perimeter is also the wrong normaliser. It anti-correlates with the error: the +largest discrepancy, 0.003 mm², occurs at P = 209.26 mm, and the smallest, +0.001 mm², at P = 454.41 mm. The error is driven by how many corners tip from 12 +arc segments to 13 (F-034), which is a property of the arrangement, not of +boundary length. Dividing by perimeter widens the spread from a factor of 3 to a +factor of 6.5. + +The real quantiser is the reference's own six-significant-figure `echo`. Every +discrepancy in the set is 0.001, 0.002 or 0.003 mm² — one, two or three units in +the last digit the reference ever recorded. Regime B is a bound on the mechanism +that actually produces the disagreement. + +Physically, both boundaries sit within 0.0027 mm (4x) and 0.0043 mm (3x) of a +true arc and within ~0.4 µm of each other. Two orders inside the criterion. The +geometry was never in question; only the comparison was. + +## 5. What would invalidate this + +- **Any case exceeding 3 ULP.** The mechanism would have changed and the bound + would no longer describe it. Investigate before widening. +- **A case where `VOLUME/AREA` is not the model length**, or where + `MASS/VOLUME` is not the density ÷ 1000. E-5's single-bound justification + rests on that propagation. +- **Regime B extended to any key in E-4.** See E-4. +- **A surviving mutation.** The bound must be demonstrated to catch a + deliberately introduced regression. A comparison that no longer notices is + worse than the 30 known failures were, because it is silent. + +## 6. What this document does not say + +It says nothing about whether a member is structurally adequate, and nothing +about manufacturing tolerance. Passing acceptance means *this implementation +reproduces the reference*, not *this part is fit for use*. `PRECISION.md` §7 +holds the list of what the compiler does not do, and it governs. + +## 7. Correction to HANDOFF.md §7 + +The per-profile breakdown recorded there — Three-Fin 9, Y 7, A Frame 5, +Rectangle 5, T 1, Four-Fin 1 — sums to 28 against a stated total of 30. +Measured, the distribution is **Three-Fin 10, Y 7, A Frame 6, Rectangle 5, T 1, +Four-Fin 1 = 30**. Three-Fin and A Frame were each undercounted by one. diff --git a/docs/FAILURES.md b/docs/FAILURES.md index bae57d5..17a44eb 100644 --- a/docs/FAILURES.md +++ b/docs/FAILURES.md @@ -6,7 +6,7 @@ Compiler environment. | | | |---|---| | Scope | All instances. Staging entries are marked `srv-b`. | -| Updated | 2026-08-18, after repository seeding and container standardisation | +| Updated | 2026-08-22, closing F-034 | | Method | `PROCESS.md` section 7 | | Rule | Append only. Never edit an entry except to add a `Resolution` line. | | Numbering | Sequential, never reused. See §0 on the renumbering. | @@ -904,6 +904,44 @@ Whichever is chosen, the checks that catch structural error — `SECTION_PARTS`, genuine geometry fault runs 1e-2 relative or worse and would still be caught with three orders of margin. +**Resolution (2026-08-22): closed.** Option 1's scope was kept and its +derivation was rejected on measurement. + +A read-only pass over all 113 accepted cases re-expressed every area +discrepancy as `|dA| / P` — the uniform boundary displacement that would +produce it, directly comparable to the 0.01 mm criterion. Median 0, p95 +9.075e-06 mm, **max 1.434e-05 mm**. A `perimeter x 0.01` bound would have run +1.86 mm^2 at the smallest section and 4.54 mm^2 at the largest — 1,800 to +4,500 times the worst real discrepancy — and would have caught nothing. +Perimeter is also the wrong normaliser: it anti-correlates with the error, +widening the spread from a factor of 3 to a factor of 6.5, because the error +is driven by how many corners tip from 12 segments to 13 rather than by +boundary length. + +The real quantiser is the reference's own six-significant-figure `echo`. +**Every discrepancy in the set is 0.001, 0.002 or 0.003 mm^2** — one, two or +three units in the last digit the reference ever recorded. The bound adopted +is `8 * ulp(expected)` where `ulp(v) = 10 ** (floor(log10 |v|) - 5)`, applied +to `SECTION_AREA_MM2`, `VOLUME_MM3` and `MASS_G` only. + +Propagation verified rather than assumed: `VOLUME/AREA` is exactly 100.0 in +all 113 cases with maximum deviation 2.8e-14, and `MASS/VOLUME` is uniform to +1e-8, so one bound governs all three keys honestly. + +**Correction to the per-profile line above.** It records Three-Fin 9, Y 7, A +Frame 5, Rectangle 5, T 1, Four-Fin 1 — which sums to 28 against the stated +total of 30. Measured: **Three-Fin 10, Y 7, A Frame 6, Rectangle 5, T 1, +Four-Fin 1 = 30.** The entry above is left as written, per the append-only +rule; this is the correct distribution. + +Result: **468 passed, 0 failed.** The 30 expected failures are resolved, not +suppressed. Mutation-tested before landing — worst case consumes 37.5% of its +bound, offsets of 12 or more last-place units are caught in all 113 cases on +all three keys, and a 1e-4 relative scaling is caught everywhere, while 1e-6 +and 1e-5 correctly are not. `test_tolerance_policy_is_not_vacuous` asserts the +derived bound never exceeds 1e-4 relative, so a future widening fails a test +instead of passing quietly. Specification in `docs/ACCEPTANCE.md`. + --- ### F-035 — `su` cannot run as a `nologin` service user @@ -953,6 +991,37 @@ running every repository operation as `mechcomp`. All are now stated as facts in --- +### F-036 — a work order used the system interpreter instead of the virtualenv +CT 100. F-034 measurement pass. + +**Observed:** a measurement script delivered by the architect was invoked as +`runuser -u mechcomp -- python3 /tmp/f034/f034-measure.py` and died at +`import pytest` with `ModuleNotFoundError`, three frames into loading +`tests/conftest.py`. + +**Cause:** **Proven.** `Makefile` line 3 sets `PY ?= venv/bin/python`, and +`make deps` runs `pip install -r requirements-base.txt -r requirements-cad.txt` +followed by `pip install -e .` into that virtualenv. pytest, Shapely, numpy and +`mechcomp` itself all live in `/var/www/mechcomp/venv`. System `python3` has +none of them. Had the script got past the pytest import it would have failed +again on Shapely. + +**Correction:** invoke `/var/www/mechcomp/venv/bin/python`. The script itself +needed no change; it ran first time on re-invocation. + +**Consequence:** **This is F-035 exactly, one layer up, and it is the reason +F-035's own consequence was written.** `HANDOFF.md` §1 stated `runuser` as a +fact but left the interpreter to be inferred from the Makefile — learnable by +pattern-matching an existing command, not by reading. That fails precisely when +an assistant composes a command from scratch, which is what happened both +times. Now stated as a fact in §1. + +The general form is worth keeping: **an operational fact that appears only +inside example commands has not been documented.** Grep the handoff for facts +that exist only as examples; each is a future F-035. + +--- + ## Open, not closed | # | Status | @@ -973,7 +1042,8 @@ running every repository operation as `mechcomp`. All are now stated as facts in | F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. | | F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. | | F-033 | **Corrected** 2026-08-19. Restore path unreachable under `set -e`. | -| F-034 | **Open** 2026-08-20. Measured: the port is exact, the reference is noisy. 30 of 113 accepted cases, worst relative error 2.24e-05, confined to `SECTION_AREA_MM2` and its two derivatives. Nothing correctable in the port; awaiting a CIVICVS decision on the tolerance model. | +| F-034 | **Closed** 2026-08-22. Measured: the port is exact, the reference is noisy. 30 of 113 accepted cases, worst relative error 2.24e-05, confined to `SECTION_AREA_MM2` and its two derivatives. Nothing correctable in the port; resolved in `test_oracle.py` by bounding at eight units in the last place of the oracle's six-significant-figure record. Suite green at 468 passed. Specification in `docs/ACCEPTANCE.md`. | | F-035 | **Corrected** 2026-08-19. Use `runuser`, never `su`; `mechcomp` is `nologin`. | +| F-036 | **Corrected** 2026-08-22. The interpreter is `venv/bin/python`, never system `python3`. Same class as F-035. | Everything else is closed with a proven cause and a proven correction. diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index 59c9b59..c7d75e8 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -6,7 +6,12 @@ It is the only handoff you need to read. Dated handoffs in `docs/archive/` are historical and are not required reading — do not diff them against this to work out what is true. If something here is wrong, correct it here. -Last updated 2026-08-20 at commit `b255ebb`. +Last updated 2026-08-22, closing F-034. + +This line used to carry the commit hash of its own rewrite. It cannot: the +hash is not known until the commit is made, so the value was always the +*previous* commit and section 3 drifted two commits behind without anyone +noticing. Date only, from here. --- @@ -20,6 +25,11 @@ which executes directly. **`su - mechcomp` cannot work** and fails with `This account is currently not available` — a message that looks like a broken account and is not. See F-035. +**The interpreter is `/var/www/mechcomp/venv/bin/python`, not `python3`.** +`Makefile` line 3 sets `PY ?= venv/bin/python`, and `make deps` installs +pytest, Shapely, numpy and `mechcomp` itself into that virtualenv. System +`python3` has none of them and fails at the first import (F-036). + **All repository operations run as `mechcomp`, never root.** The clone is at `/var/www/mechcomp` in CT 100. Never add a git `safe.directory` exception to work around an ownership complaint; fix the ownership (F-008). @@ -130,10 +140,11 @@ terminate rather than recur. ### The port — COMPLETE -Gitea `main` at `114189c`. CT 100 clean and matching. +Gitea `main` at `52d00b5` before this commit. CT 100 clean and matching. -**Suite: 436 passed, 30 failed. The 30 failures are expected.** Do not treat a -red `make test` as a broken port — read §7 before doing anything about it. +**Suite: 468 passed, 0 failed.** The 30 expected failures are gone, resolved +rather than suppressed, by the tolerance model in `docs/ACCEPTANCE.md`. A red +`make test` now means something is actually wrong. | Module | Ported from | Contents | |---|---|---| @@ -155,28 +166,41 @@ red `make test` as a broken port — read §7 before doing anything about it. ## 4. What to do next -**Nothing is blocked. §7 is decided and awaiting implementation; -everything else is new capability.** +**Nothing is blocked. F-034 is closed and the suite is green; everything +below is new capability.** -**Immediate: the F-034 tolerance change — decided, not yet -implemented.** CIVICVS approved **option 1** (§7) on 20 AUG: -scale-aware bounds in `test_oracle.py` for `SECTION_AREA_MM2`, -`VOLUME_MM3` and `MASS_G`, derived from the 0.01 mm criterion and the -section perimeter. It was deferred deliberately — implementation, -mutation testing and a green suite are one piece of work, not a partial -landing. **Do not touch the oracle JSON** (§7). +**CIVICVS stated the goal on 22 AUG: he will not print anything that is not +fully configurable.** The compiler is a half-visual composer — the T encloses +poly pallet straps in a T, the Y takes a metal electrical conduit core at its +centre. That reorders what follows, because a fixed-profile STL is not a +deliverable he wants. Two consequences: -`FAILURES.md` still records F-034 as Open, which is correct until the -change lands. Its measurement rewrite is already done (commit -`b255ebb`) — do not redo it. +- **The bore is a declared interface**, with a diameter and a fit class, + specified by what goes through it. It is not a byproduct of the inside + walls. Conduit for the Y, not only rebar. +- **The catalogue front end is the product, not a convenience** that comes + after export. `ROADMAP.md` §4 orders it third; that ordering predates this + and should be read against it. -After that, the roadmap items — read `ROADMAP.md` before starting any: +Roadmap items — read `ROADMAP.md` before starting any: -- STL and STEP export. No CAD kernel is installed in CT 100 (§5). - Dihedral parameterisation. If two panels meet at 137°, none of the eleven - profiles gives you a member for it. This is the highest-value gap. + profiles gives you a member for it. This is the highest-value gap and it + blocks a catalogue that could otherwise not serve the reference structure. +- Rebar- and conduit-core bore as a declared interface. See above. +- The catalogue front end. Nothing of it exists: no SVG emitter, and + `src/mechcomp/web/__init__.py` and `worker/__init__.py` are 33- and 36-byte + stubs. `STAGING-STATE.md` records deployment as blocked on application + code, which is still true. +- STL and STEP export. No CAD kernel is installed in CT 100 (§5). - Nodes (non-prismatic) and panels (sheet). No representation exists for either. +Note for anyone tempted to shortcut a test print: the OpenSCAD reference in +`legacy/` does export printable STL today — `sb_extrude_section` does a +`linear_sweep` and `legacy/openscad/README.md` documents the invocation. That +is not what he asked for, and offering it again would be repeating a mistake +already made once. + --- ## 5. Facts established by porting @@ -214,9 +238,16 @@ precision — not full-precision geometry. `echo_num()` does this; `echo_vec()` handles vectors, which reach the oracle as strings like `'[20.5209, 20.5209, 20.5209]'`. -Load-bearing, not cosmetic. `VOLUME_MM3` ends in `_MM3`, so `test_oracle.py` -compares it at the **lengths** tolerance of 1e-4, not the areas tolerance of -1e-3. +Load-bearing, not cosmetic — and it is now the basis of the tolerance model. +Because the oracle holds six significant figures, the last recorded digit of +an area near 200 mm² is worth 0.001 mm², and the whole measured disagreement +between port and reference is one, two or three units in that place. See +`docs/ACCEPTANCE.md`. + +Historical note, since it was the visible symptom for two sessions: +`VOLUME_MM3` ends in `_MM3`, so the old comparison judged it at the lengths +tolerance of 1e-4 against a magnitude near 20,000 — 5e-9 relative, demanded +of discretised geometry, by accident of key naming. ### Angles are degrees; the arbitrary constants are contract @@ -289,7 +320,7 @@ module calls it through the module global, so patching the attribute works. --- -## 7. F-034 — measured, and the decision is ready +## 7. F-034 — measured, decided, closed **The port is exact. The reference is noisy.** That is the opposite of what the entry previously implied, and it changes what can be done about it. @@ -347,9 +378,25 @@ The change belongs in `test_oracle.py`, which is not hashed: 3. **Accept 30 known failures.** Honest but `make test` is never green and a real regression hides among them. -**Decided 20 AUG: option 1**, scale-aware bounds for the three -discretisation-limited keys. The change goes in `test_oracle.py`, never -the oracle JSON. Not yet implemented — see §4. +**Closed 22 AUG.** Option 1's *scope* was kept — the three keys, the change in +`test_oracle.py`, the oracle JSON untouched — but its *derivation* was +rejected on measurement and replaced. + +Measuring `|dA| / P`, the boundary displacement that would produce each area +discrepancy, gives a maximum of 1.434e-05 mm: one seven-hundredth of the +0.01 mm criterion. A `perimeter × 0.01` bound would have been 1.86 mm² at the +smallest section and 4.54 mm² at the largest, between 1,800 and 4,500 times +the worst real discrepancy, and would have caught nothing. Perimeter also +anti-correlates with the error — the largest discrepancy is at P = 209 mm and +the smallest at P = 454 mm — because the error is driven by how many corners +tip from 12 segments to 13, not by boundary length. + +What replaced it: **eight units in the last place of the oracle's +six-significant-figure record**, for those three keys only. Worst case uses +37.5% of its bound, mutation-tested before landing. **`docs/ACCEPTANCE.md` is +the specification** — read it before touching any tolerance, and note E-4: +everything that positions material stays exact at 1e-4 mm and must never be +moved into that regime. --- @@ -440,7 +487,8 @@ held to it. When in doubt, narrow. | 2 | Backup strategy — USB, IPFS, optical? | CIVICVS | | 3 | Where is the 3+ TB USB disk attached? | CIVICVS | | 4 | Kane Fabric participant mail, send and receive | Cross-project | -| 5 | Tolerance model for the three discretisation-limited keys — see §7 | **Decided 20 AUG: option 1.** Implementation pending | +| 5 | ~~Tolerance model for the three discretisation-limited keys~~ | **Closed 22 AUG.** See §7 and `docs/ACCEPTANCE.md` | +| 6 | Is the bore's fit class per-material, or one clearance for all inserts? | CIVICVS — raised by the conduit-core requirement, §4 | Question 4 is real and unaddressed. Containers do not send mail by standard and nothing can reach `vmbr1` from outside, so receiving has no path at all. It needs @@ -451,6 +499,8 @@ a design conversation, not a configuration change. ## 12. Commit log ``` +52d00b5 docs: HANDOFF header commit line moved to b255ebb +b255ebb docs: F-034 rewritten from measurement; handoff updated 114189c docs: PRECISION.md -- scope, guarantees and limits, for lay readers 7b3182c port: the 3x and 4x profile catalogues; build() now exists af196a2 docs: one canonical handoff, rewritten in place; F-035 diff --git a/tests/test_oracle.py b/tests/test_oracle.py index 220a0ae..5f8c67c 100644 --- a/tests/test_oracle.py +++ b/tests/test_oracle.py @@ -10,6 +10,21 @@ skips with a clear reason. Those integrity checks always run -- an oracle that has been edited is worse than no oracle, and that should fail loudly on any machine, at any time, with no dependencies. +TOLERANCE + Two regimes, and `docs/ACCEPTANCE.md` holds the reasoning. + + A declared absolute -- the oracle's own tolerance block. Everything that + positions material. Exact in all 123 cases; never loosen it. + + B six-significant-figure last place -- SECTION_AREA_MM2, VOLUME_MM3 and + MASS_G only. The oracle records what OpenSCAD's echo printed, which is + %g at six significant figures, so the last recorded digit of an area + near 200 mm2 is worth 0.001 mm2. Measured disagreement across the whole + set is one, two or three units in that place, never more. + + The change belongs here and never in the oracle JSON: the tolerance block is + inside the hashed document and `test_integrity_hash` covers it. + pytest -n auto # all of it pytest -m oracle # acceptance only pytest -k integrity # oracle checks alone, always runnable @@ -17,11 +32,61 @@ machine, at any time, with no dependencies. from __future__ import annotations +import math + import pytest pytestmark = pytest.mark.oracle +# --------------------------------------------------------------------------- +# Tolerance policy +# --------------------------------------------------------------------------- + +# The three keys limited by arc-segment discretisation rather than by the port. +# One underlying quantity reported three times: volume is area times the model +# length, mass is volume times density over 1000. See F-034. +SIXFIG_KEYS = ("SECTION_AREA_MM2", "VOLUME_MM3", "MASS_G") + +# How many units in that last place are allowed. Worst measured is 3 for area +# and volume, 5 for mass; 8 leaves margin without reaching 1e-4 relative, which +# `test_tolerance_policy_is_not_vacuous` enforces. +SIXFIG_ULPS = 8 + +# Counts, compared exactly. +EXACT_KEYS = ("SECTION_PARTS", "STRAP_CHANNELS", "BUNDLE_COUNT") + +# The ceiling every derived bound must stay under, as a fraction of the value. +RELATIVE_CEILING = 1e-4 + + +def last_place(value: float) -> float: + """ + The value of the last digit the reference actually recorded. + + The oracle holds `echo` output, six significant figures. For 213.950 that + is 0.001; for 21395.0 it is 0.1. Returns 0.0 for a zero value, which has no + meaningful last place -- callers fall back to the declared tolerance. + """ + if value == 0.0 or not math.isfinite(value): + return 0.0 + return 10.0 ** (math.floor(math.log10(abs(value))) - 5) + + +def bound_for(key: str, want: float, tolerance: dict) -> float: + """The tolerance in force for one key at one magnitude.""" + if key in SIXFIG_KEYS: + ulp = last_place(want) + if ulp: + return SIXFIG_ULPS * ulp + return tolerance["areas_mm2"] if key.endswith("_MM2") else tolerance["lengths_mm"] + + +def comparable(key: str, want: object) -> bool: + """Float-valued report keys that are not counts.""" + return isinstance(want, float) and key not in EXACT_KEYS + + # --------------------------------------------------------------------------- # Integrity - no dependency on the port # --------------------------------------------------------------------------- @@ -74,6 +139,75 @@ def test_integrity_invariants_hold_in_the_oracle(accepted_cases): f"{where}: wall below its declared minimum" +# --------------------------------------------------------------------------- +# The tolerance policy itself - no dependency on the port +# --------------------------------------------------------------------------- + +def test_tolerance_policy_is_not_vacuous(accepted_cases, tolerance): + """ + No derived bound ever exceeds 1e-4 relative. + + This is the guard against the failure mode that matters here. A comparison + quietly loosened until it no longer catches a real regression passes every + other test in this file while proving nothing, and it does so silently -- + which is worse than the 30 known failures this policy replaced, because + those were visible. ACCEPTANCE.md E-6. + + Scoped to regime B, which is the part of the policy that lives in this + unhashed file and could therefore be widened by an edit. Regime A is the + oracle's own declared tolerance and is protected by `test_integrity_hash`; + it is also legitimately absolute rather than relative, since it covers + quantities well below 1 mm such as CLEARANCE_MM and STRAP_THICK_MM. + """ + for case in accepted_cases: + where = f"{case['profile']}/{case['label']}" + for key in SIXFIG_KEYS: + want = case["report"][key] + if want == 0.0: + continue + bound = bound_for(key, want, tolerance) + assert bound <= abs(want) * RELATIVE_CEILING, ( + f"{where}: the bound for {key} is {bound} against a value of " + f"{want}, which is looser than {RELATIVE_CEILING} relative. " + f"The comparison has stopped being able to catch a regression." + ) + + +def test_sixfig_keys_are_the_derived_trio(accepted_cases): + """ + Volume is area times the model length; mass is volume times density. + + One bound governs all three keys only because they are one quantity + reported three times. If that stops being true, E-5's justification is + gone and the three need separate treatment. ACCEPTANCE.md E-5. + """ + ratios = set() + + for case in accepted_cases: + r = case["report"] + where = f"{case['profile']}/{case['label']}" + + area = r["SECTION_AREA_MM2"] + volume = r["VOLUME_MM3"] + ratio = volume / area + ratios.add(round(ratio, 6)) + + if "LENGTH_MM" in r: + assert ratio == pytest.approx(r["LENGTH_MM"], rel=1e-6), ( + f"{where}: VOLUME_MM3 / SECTION_AREA_MM2 is {ratio}, but the " + f"report says LENGTH_MM is {r['LENGTH_MM']}" + ) + + assert r["MASS_G"] == pytest.approx(volume * 1.24 / 1000, rel=5e-5), ( + f"{where}: MASS_G is not VOLUME_MM3 times density over 1000" + ) + + assert len(ratios) == 1, ( + f"the extrusion length is not constant across the oracle: {sorted(ratios)}. " + f"One bound can no longer serve all three keys." + ) + + # --------------------------------------------------------------------------- # Acceptance - requires the port # --------------------------------------------------------------------------- @@ -96,17 +230,18 @@ def test_accepted_case_matches_oracle(port, accepted_case, tolerance): got = result.report # Counts are exact. - for key in ("SECTION_PARTS", "STRAP_CHANNELS", "BUNDLE_COUNT"): + for key in EXACT_KEYS: assert got[key] == expected[key], f"{where}: {key}" - # Lengths and areas carry the tolerance the oracle declares. + # Everything else carries the bound its regime declares. for key, want in expected.items(): - if not isinstance(want, float) or key in ("SECTION_PARTS", "STRAP_CHANNELS"): + if not comparable(key, want): continue - tol = tolerance["areas_mm2"] if key.endswith("_MM2") else tolerance["lengths_mm"] + tol = bound_for(key, want, tolerance) assert key in got, f"{where}: port did not report {key}" assert abs(got[key] - want) <= tol, ( - f"{where}: {key} is {got[key]}, oracle says {want}" + f"{where}: {key} is {got[key]}, oracle says {want} " + f"(difference {got[key] - want:.6g}, bound {tol:.6g})" )