TheRON/mechanical-compiler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tests: close F-034; bound the derived trio at 8 ULP of the oracle record

Browse Source 
The oracle records six significant figures, so the last digit of an area near 200 mm2 is worth 0.001 mm2. Every measured disagreement between port and reference is one, two or three units in that place. SECTION_AREA_MM2, VOLUME_MM3 and MASS_G are now bounded at 8 ULP of the expected value. Everything that positions material keeps the declared 1e-4 mm and remains exact in all 123 cases.

Option 1 scope kept, derivation rejected on measurement. Max |dA|/P over the accepted set is 1.434e-05 mm, one seven-hundredth of the 0.01 mm criterion, so a perimeter x 0.01 bound would have run 1800 to 4500 times the worst real discrepancy and caught nothing. Perimeter also anti-correlates with the error.

Suite 468 passed, 0 failed. The 30 expected failures are resolved, not suppressed. Mutation tested before landing: worst case uses 37.5 percent of its bound, 12 ULP offsets and 1e-4 relative scalings are caught in all 113 cases, 1e-6 and 1e-5 correctly are not.

Adds docs/ACCEPTANCE.md as the specification. Adds F-036, the venv interpreter error, same class as F-035. Corrects the F-034 per-profile distribution to Three-Fin 10, Y 7, A Frame 6, Rectangle 5, T 1, Four-Fin 1, which sums to the stated 30.
This commit is contained in:
TheRON
2026-08-23 08:19:18 -05:00
parent 52d00b588b
commit 6836ece4ff
4 changed files with 447 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ACCEPTANCE.md
+158
View File
@@ -0,0 +1,158 @@
# ACCEPTANCE.md
**What the acceptance suite asserts, and to what tolerance.**
This document exists because until now the answer lived in two places, neither of
them readable: a `tolerance` block inside a hashed JSON fixture, and one line of
`test_oracle.py`. A person asking "how close does the port have to be?" had to
read code to find out, and the reason for the numbers was nowhere.
Last updated 2026-08-22, closing F-034.
---
## 1. What is being compared
The port is measured against a frozen 123-case oracle generated by the OpenSCAD
reference at revision 8.0.0, under OpenSCAD 2021.01 with BOSL2 `92d697c2`. 113
cases the reference accepted, 10 it rejected.
The oracle does **not** record full-precision geometry. It records what
OpenSCAD's `echo` printed, which is C's `%g` at default precision — **six
significant figures**. That fact is load-bearing and most of this document
follows from it.
## 2. Two tolerance regimes
**Regime A — declared absolute.** The oracle's own `tolerance` block:
`lengths_mm = 1e-4`, `areas_mm2 = 1e-3`. Applies to every compared float except
the three keys named below.
**Regime B — six-significant-figure last place.** Applies to
`SECTION_AREA_MM2`, `VOLUME_MM3` and `MASS_G` only.
```
ulp(v) = 10 ** (floor(log10(|v|)) - 5) the value of the last digit
the reference actually wrote down
bound = 8 * ulp(expected)
```
Concretely, across the oracle's observed magnitudes:
| key | magnitude | last place | worst observed | bound |
|---|---|---|---|---|
| `SECTION_AREA_MM2` | 117 – 474 mm² | 0.001 | 0.003 (3 ULP) | 0.008 |
| `VOLUME_MM3` | 11,742 – 47,364 mm³ | 0.1 | 0.3 (3 ULP) | 0.8 |
| `MASS_G` | 14.6 – 58.7 g | 0.0001 | 0.0005 (5 ULP) | 0.0008 |
The bound scales with magnitude by construction, so it needs no revision if a
future case is an order of magnitude larger or smaller.
## 3. Expectation statements
These are what a passing suite means. Each is asserted by a test.
**E-1 — Every accepted case builds.** `build(family, profile, params)` returns a
`Result` for all 113. A raise is a failure, not a rejection.
**E-2 — Every rejected case is rejected.** All 10 raise `ProfileRejected`, and
the message is non-empty and names the parameter and the limit. Reproducing the
geometry while accepting a case the reference refused is not a passing port.
**E-3 — Counts are exact.** `SECTION_PARTS`, `STRAP_CHANNELS` and
`BUNDLE_COUNT` match with zero tolerance.
**E-4 — Everything that positions material is exact to regime A.**
`ENVELOPE_X_MM`, `ENVELOPE_Y_MM`, `MIN_WALL_ACTUAL_MM` and every profile extra
(`AF_*`, `FIN_*`, `SPOKE_*`, `RING_*`, `T_*`) agree within 1e-4 mm.
Measured: **exact in all 123 cases, no exceptions.** This is the strongest claim
the project makes and regime B must never be extended to cover any of it. If a
future change makes one of these keys need a looser bound, the change is wrong.
**E-5 — The three discretisation-limited keys agree to regime B.**
`SECTION_AREA_MM2`, `VOLUME_MM3` and `MASS_G` agree within 8 ULP of the oracle's
six-significant-figure record.
These three are one underlying quantity reported three times: volume is area ×
100 mm, mass is volume × 1.24 g/cm³ ÷ 1000. Verified across all 113 accepted
cases — `VOLUME/AREA` is exactly 100.0 with a maximum deviation of 2.8e-14, and
`MASS/VOLUME` is uniform to 1e-8. One bound therefore governs all three
honestly, rather than three unrelated policies.
**E-6 — The tolerance policy is not vacuous.** For every compared value, the
bound in force is at most 1e-4 relative. This is asserted directly, so a future
edit that loosens the comparison until it stops catching regressions fails a
test rather than passing silently.
**E-7 — The oracle is unmodified.** Its recorded hash covers the whole document
except the hash field. Editing the fixture to make a test pass breaks this by
design.
**E-8 — The 2D path imports no CAD kernel.** Building a cross-section must not
pull in `cadquery`, `OCP` or `build123d`.
**E-9 — The suite is green.** There are no expected failures. A red `make test`
means something is wrong.
## 4. Why the accuracy criterion does not source these numbers
The project's stated accuracy criterion is 0.01 mm over the entire set
(`PRECISION.md`). The obvious move — bound the area error by
`perimeter × 0.01 mm` — was measured on 20260822 and rejected.
Re-expressing every measured area discrepancy as the uniform boundary
displacement that would produce it gives `|dA| / P`, which is directly
comparable to the criterion:
```
median 0.000e+00 mm
p95 9.075e-06 mm
max 1.434e-05 mm one seven-hundredth of the criterion
```
A `perimeter × 0.01` bound would be 1.86 mm² at the smallest section and
4.54 mm² at the largest — between 1,800 and 4,500 times the worst real
discrepancy. It would catch nothing.
Perimeter is also the wrong normaliser. It anti-correlates with the error: the
largest discrepancy, 0.003 mm², occurs at P = 209.26 mm, and the smallest,
0.001 mm², at P = 454.41 mm. The error is driven by how many corners tip from 12
arc segments to 13 (F-034), which is a property of the arrangement, not of
boundary length. Dividing by perimeter widens the spread from a factor of 3 to a
factor of 6.5.
The real quantiser is the reference's own six-significant-figure `echo`. Every
discrepancy in the set is 0.001, 0.002 or 0.003 mm² — one, two or three units in
the last digit the reference ever recorded. Regime B is a bound on the mechanism
that actually produces the disagreement.
Physically, both boundaries sit within 0.0027 mm (4x) and 0.0043 mm (3x) of a
true arc and within ~0.4 µm of each other. Two orders inside the criterion. The
geometry was never in question; only the comparison was.
## 5. What would invalidate this
- **Any case exceeding 3 ULP.** The mechanism would have changed and the bound
would no longer describe it. Investigate before widening.
- **A case where `VOLUME/AREA` is not the model length**, or where
`MASS/VOLUME` is not the density ÷ 1000. E-5's single-bound justification
rests on that propagation.
- **Regime B extended to any key in E-4.** See E-4.
- **A surviving mutation.** The bound must be demonstrated to catch a
deliberately introduced regression. A comparison that no longer notices is
worse than the 30 known failures were, because it is silent.
## 6. What this document does not say
It says nothing about whether a member is structurally adequate, and nothing
about manufacturing tolerance. Passing acceptance means *this implementation
reproduces the reference*, not *this part is fit for use*. `PRECISION.md` §7
holds the list of what the compiler does not do, and it governs.
## 7. Correction to HANDOFF.md §7
The per-profile breakdown recorded there — Three-Fin 9, Y 7, A Frame 5,
Rectangle 5, T 1, Four-Fin 1 — sums to 28 against a stated total of 30.
Measured, the distribution is **Three-Fin 10, Y 7, A Frame 6, Rectangle 5, T 1,
Four-Fin 1 = 30**. Three-Fin and A Frame were each undercounted by one.
docs/FAILURES.md
+72 -2
View File
@@ -6,7 +6,7 @@ Compiler environment.
| | |
|---|---|
| Scope | All instances. Staging entries are marked `srv-b`. |
| Updated | 2026-08-18, after repository seeding and container standardisation |
| Updated | 2026-08-22, closing F-034 |
| Method | `PROCESS.md` section 7 |
| Rule | Append only. Never edit an entry except to add a `Resolution` line. |
| Numbering | Sequential, never reused. See §0 on the renumbering. |
@@ -904,6 +904,44 @@ Whichever is chosen, the checks that catch structural error — `SECTION_PARTS`,
genuine geometry fault runs 1e-2 relative or worse and would still be caught with
three orders of margin.
**Resolution (2026-08-22): closed.** Option 1's scope was kept and its
derivation was rejected on measurement.
A read-only pass over all 113 accepted cases re-expressed every area
discrepancy as `|dA| / P` — the uniform boundary displacement that would
produce it, directly comparable to the 0.01 mm criterion. Median 0, p95
9.075e-06 mm, **max 1.434e-05 mm**. A `perimeter x 0.01` bound would have run
1.86 mm^2 at the smallest section and 4.54 mm^2 at the largest — 1,800 to
4,500 times the worst real discrepancy — and would have caught nothing.
Perimeter is also the wrong normaliser: it anti-correlates with the error,
widening the spread from a factor of 3 to a factor of 6.5, because the error
is driven by how many corners tip from 12 segments to 13 rather than by
boundary length.
The real quantiser is the reference's own six-significant-figure `echo`.
**Every discrepancy in the set is 0.001, 0.002 or 0.003 mm^2** — one, two or
three units in the last digit the reference ever recorded. The bound adopted
is `8 * ulp(expected)` where `ulp(v) = 10 ** (floor(log10 |v|) - 5)`, applied
to `SECTION_AREA_MM2`, `VOLUME_MM3` and `MASS_G` only.
Propagation verified rather than assumed: `VOLUME/AREA` is exactly 100.0 in
all 113 cases with maximum deviation 2.8e-14, and `MASS/VOLUME` is uniform to
1e-8, so one bound governs all three keys honestly.
**Correction to the per-profile line above.** It records Three-Fin 9, Y 7, A
Frame 5, Rectangle 5, T 1, Four-Fin 1 — which sums to 28 against the stated
total of 30. Measured: **Three-Fin 10, Y 7, A Frame 6, Rectangle 5, T 1,
Four-Fin 1 = 30.** The entry above is left as written, per the append-only
rule; this is the correct distribution.
Result: **468 passed, 0 failed.** The 30 expected failures are resolved, not
suppressed. Mutation-tested before landing — worst case consumes 37.5% of its
bound, offsets of 12 or more last-place units are caught in all 113 cases on
all three keys, and a 1e-4 relative scaling is caught everywhere, while 1e-6
and 1e-5 correctly are not. `test_tolerance_policy_is_not_vacuous` asserts the
derived bound never exceeds 1e-4 relative, so a future widening fails a test
instead of passing quietly. Specification in `docs/ACCEPTANCE.md`.
---
### F-035 — `su` cannot run as a `nologin` service user
@@ -953,6 +991,37 @@ running every repository operation as `mechcomp`. All are now stated as facts in
---
### F-036 — a work order used the system interpreter instead of the virtualenv
CT 100. F-034 measurement pass.
**Observed:** a measurement script delivered by the architect was invoked as
`runuser -u mechcomp -- python3 /tmp/f034/f034-measure.py` and died at
`import pytest` with `ModuleNotFoundError`, three frames into loading
`tests/conftest.py`.
**Cause:** **Proven.** `Makefile` line 3 sets `PY ?= venv/bin/python`, and
`make deps` runs `pip install -r requirements-base.txt -r requirements-cad.txt`
followed by `pip install -e .` into that virtualenv. pytest, Shapely, numpy and
`mechcomp` itself all live in `/var/www/mechcomp/venv`. System `python3` has
none of them. Had the script got past the pytest import it would have failed
again on Shapely.
**Correction:** invoke `/var/www/mechcomp/venv/bin/python`. The script itself
needed no change; it ran first time on re-invocation.
**Consequence:** **This is F-035 exactly, one layer up, and it is the reason
F-035's own consequence was written.** `HANDOFF.md` §1 stated `runuser` as a
fact but left the interpreter to be inferred from the Makefile — learnable by
pattern-matching an existing command, not by reading. That fails precisely when
an assistant composes a command from scratch, which is what happened both
times. Now stated as a fact in §1.
The general form is worth keeping: **an operational fact that appears only
inside example commands has not been documented.** Grep the handoff for facts
that exist only as examples; each is a future F-035.
---
## Open, not closed
| # | Status |
@@ -973,7 +1042,8 @@ running every repository operation as `mechcomp`. All are now stated as facts in
| F-031 | **Corrected** 2026-08-18. Root cause of `ct-baseline.sh`. |
| F-032 | **Closed** 2026-08-18. No correction required; encoded in `ct-baseline.sh`. |
| F-033 | **Corrected** 2026-08-19. Restore path unreachable under `set -e`. |
| F-034 | **Open** 2026-08-20. Measured: the port is exact, the reference is noisy. 30 of 113 accepted cases, worst relative error 2.24e-05, confined to `SECTION_AREA_MM2` and its two derivatives. Nothing correctable in the port; awaiting a CIVICVS decision on the tolerance model. |
| F-034 | **Closed** 2026-08-22. Measured: the port is exact, the reference is noisy. 30 of 113 accepted cases, worst relative error 2.24e-05, confined to `SECTION_AREA_MM2` and its two derivatives. Nothing correctable in the port; resolved in `test_oracle.py` by bounding at eight units in the last place of the oracle's six-significant-figure record. Suite green at 468 passed. Specification in `docs/ACCEPTANCE.md`. |
| F-035 | **Corrected** 2026-08-19. Use `runuser`, never `su`; `mechcomp` is `nologin`. |
| F-036 | **Corrected** 2026-08-22. The interpreter is `venv/bin/python`, never system `python3`. Same class as F-035. |
Everything else is closed with a proven cause and a proven correction.
docs/HANDOFF.md
+77 -27
View File
@@ -6,7 +6,12 @@ It is the only handoff you need to read. Dated handoffs in `docs/archive/` are
historical and are not required reading — do not diff them against this to work
out what is true. If something here is wrong, correct it here.
Last updated 2026-08-20 at commit `b255ebb`.
Last updated 2026-08-22, closing F-034.
This line used to carry the commit hash of its own rewrite. It cannot: the
hash is not known until the commit is made, so the value was always the
*previous* commit and section 3 drifted two commits behind without anyone
noticing. Date only, from here.
---
@@ -20,6 +25,11 @@ which executes directly. **`su - mechcomp` cannot work** and fails with
`This account is currently not available` — a message that looks like a broken
account and is not. See F-035.
**The interpreter is `/var/www/mechcomp/venv/bin/python`, not `python3`.**
`Makefile` line 3 sets `PY ?= venv/bin/python`, and `make deps` installs
pytest, Shapely, numpy and `mechcomp` itself into that virtualenv. System
`python3` has none of them and fails at the first import (F-036).
**All repository operations run as `mechcomp`, never root.** The clone is at
`/var/www/mechcomp` in CT 100. Never add a git `safe.directory` exception to work
around an ownership complaint; fix the ownership (F-008).
@@ -130,10 +140,11 @@ terminate rather than recur.
### The port — COMPLETE
Gitea `main` at `114189c`. CT 100 clean and matching.
Gitea `main` at `52d00b5` before this commit. CT 100 clean and matching.
**Suite: 436 passed, 30 failed. The 30 failures are expected.** Do not treat a
red `make test` as a broken port — read §7 before doing anything about it.
**Suite: 468 passed, 0 failed.** The 30 expected failures are gone, resolved
rather than suppressed, by the tolerance model in `docs/ACCEPTANCE.md`. A red
`make test` now means something is actually wrong.
| Module | Ported from | Contents |
|---|---|---|
@@ -155,28 +166,41 @@ red `make test` as a broken port — read §7 before doing anything about it.
## 4. What to do next
**Nothing is blocked. §7 is decided and awaiting implementation;
everything else is new capability.**
**Nothing is blocked. F-034 is closed and the suite is green; everything
below is new capability.**
**Immediate: the F-034 tolerance change — decided, not yet
implemented.** CIVICVS approved **option 1** (§7) on 20 AUG:
scale-aware bounds in `test_oracle.py` for `SECTION_AREA_MM2`,
`VOLUME_MM3` and `MASS_G`, derived from the 0.01 mm criterion and the
section perimeter. It was deferred deliberately — implementation,
mutation testing and a green suite are one piece of work, not a partial
landing. **Do not touch the oracle JSON** (§7).
**CIVICVS stated the goal on 22 AUG: he will not print anything that is not
fully configurable.** The compiler is a half-visual composer — the T encloses
poly pallet straps in a T, the Y takes a metal electrical conduit core at its
centre. That reorders what follows, because a fixed-profile STL is not a
deliverable he wants. Two consequences:
`FAILURES.md` still records F-034 as Open, which is correct until the
change lands. Its measurement rewrite is already done (commit
`b255ebb`) — do not redo it.
- **The bore is a declared interface**, with a diameter and a fit class,
specified by what goes through it. It is not a byproduct of the inside
walls. Conduit for the Y, not only rebar.
- **The catalogue front end is the product, not a convenience** that comes
after export. `ROADMAP.md` §4 orders it third; that ordering predates this
and should be read against it.
After that, the roadmap items — read `ROADMAP.md` before starting any:
Roadmap items — read `ROADMAP.md` before starting any:
- STL and STEP export. No CAD kernel is installed in CT 100 (§5).
- Dihedral parameterisation. If two panels meet at 137°, none of the eleven
profiles gives you a member for it. This is the highest-value gap.
profiles gives you a member for it. This is the highest-value gap and it
blocks a catalogue that could otherwise not serve the reference structure.
- Rebar- and conduit-core bore as a declared interface. See above.
- The catalogue front end. Nothing of it exists: no SVG emitter, and
`src/mechcomp/web/__init__.py` and `worker/__init__.py` are 33- and 36-byte
stubs. `STAGING-STATE.md` records deployment as blocked on application
code, which is still true.
- STL and STEP export. No CAD kernel is installed in CT 100 (§5).
- Nodes (non-prismatic) and panels (sheet). No representation exists for either.
Note for anyone tempted to shortcut a test print: the OpenSCAD reference in
`legacy/` does export printable STL today — `sb_extrude_section` does a
`linear_sweep` and `legacy/openscad/README.md` documents the invocation. That
is not what he asked for, and offering it again would be repeating a mistake
already made once.
---
## 5. Facts established by porting
@@ -214,9 +238,16 @@ precision — not full-precision geometry. `echo_num()` does this; `echo_vec()`
handles vectors, which reach the oracle as strings like
`'[20.5209, 20.5209, 20.5209]'`.
Load-bearing, not cosmetic. `VOLUME_MM3` ends in `_MM3`, so `test_oracle.py`
compares it at the **lengths** tolerance of 1e-4, not the areas tolerance of
1e-3.
Load-bearing, not cosmetic — and it is now the basis of the tolerance model.
Because the oracle holds six significant figures, the last recorded digit of
an area near 200 mm² is worth 0.001 mm², and the whole measured disagreement
between port and reference is one, two or three units in that place. See
`docs/ACCEPTANCE.md`.
Historical note, since it was the visible symptom for two sessions:
`VOLUME_MM3` ends in `_MM3`, so the old comparison judged it at the lengths
tolerance of 1e-4 against a magnitude near 20,000 — 5e-9 relative, demanded
of discretised geometry, by accident of key naming.
### Angles are degrees; the arbitrary constants are contract
@@ -289,7 +320,7 @@ module calls it through the module global, so patching the attribute works.
---
## 7. F-034 — measured, and the decision is ready
## 7. F-034 — measured, decided, closed
**The port is exact. The reference is noisy.** That is the opposite of what the
entry previously implied, and it changes what can be done about it.
@@ -347,9 +378,25 @@ The change belongs in `test_oracle.py`, which is not hashed:
3. **Accept 30 known failures.** Honest but `make test` is never green and a real
regression hides among them.
**Decided 20 AUG: option 1**, scale-aware bounds for the three
discretisation-limited keys. The change goes in `test_oracle.py`, never
the oracle JSON. Not yet implemented — see §4.
**Closed 22 AUG.** Option 1's *scope* was kept — the three keys, the change in
`test_oracle.py`, the oracle JSON untouched — but its *derivation* was
rejected on measurement and replaced.
Measuring `|dA| / P`, the boundary displacement that would produce each area
discrepancy, gives a maximum of 1.434e-05 mm: one seven-hundredth of the
0.01 mm criterion. A `perimeter × 0.01` bound would have been 1.86 mm² at the
smallest section and 4.54 mm² at the largest, between 1,800 and 4,500 times
the worst real discrepancy, and would have caught nothing. Perimeter also
anti-correlates with the error — the largest discrepancy is at P = 209 mm and
the smallest at P = 454 mm — because the error is driven by how many corners
tip from 12 segments to 13, not by boundary length.
What replaced it: **eight units in the last place of the oracle's
six-significant-figure record**, for those three keys only. Worst case uses
37.5% of its bound, mutation-tested before landing. **`docs/ACCEPTANCE.md` is
the specification** — read it before touching any tolerance, and note E-4:
everything that positions material stays exact at 1e-4 mm and must never be
moved into that regime.
---
@@ -440,7 +487,8 @@ held to it. When in doubt, narrow.
| 2 | Backup strategy — USB, IPFS, optical? | CIVICVS |
| 3 | Where is the 3+ TB USB disk attached? | CIVICVS |
| 4 | Kane Fabric participant mail, send and receive | Cross-project |
| 5 | Tolerance model for the three discretisation-limited keys — see §7 | **Decided 20 AUG: option 1.** Implementation pending |
| 5 | ~~Tolerance model for the three discretisation-limited keys~~ | **Closed 22 AUG.** See §7 and `docs/ACCEPTANCE.md` |
| 6 | Is the bore's fit class per-material, or one clearance for all inserts? | CIVICVS — raised by the conduit-core requirement, §4 |
Question 4 is real and unaddressed. Containers do not send mail by standard and
nothing can reach `vmbr1` from outside, so receiving has no path at all. It needs
@@ -451,6 +499,8 @@ a design conversation, not a configuration change.
## 12. Commit log
```
52d00b5 docs: HANDOFF header commit line moved to b255ebb
b255ebb docs: F-034 rewritten from measurement; handoff updated
114189c docs: PRECISION.md -- scope, guarantees and limits, for lay readers
7b3182c port: the 3x and 4x profile catalogues; build() now exists
af196a2 docs: one canonical handoff, rewritten in place; F-035