TheRON/mechanical-compiler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
eac601ed99d678e7db3403abed7e9822e94b8d91
mechanical-compiler/docs/STAGING-STATE.md
T
TheRON eac601ed99 Current state 
Live state of the Mechanical Compiler staging instance on `srv-b`.
2026-08-16 12:01:20 -04:00

11 KiB
Raw Blame History

STAGING-STATE.md

Live state of the Mechanical Compiler staging instance on srv-b.
Updated 2026-08-15, after network isolation
Instance Staging / development
Specification ENVIRONMENT.md revision 5
Failure log FAILURES.md
Method Manual, one command group at a time

0. How to use this file

This is the authoritative record of what is true on srv-b. Where it and ENVIRONMENT.md disagree, this file wins for facts and the specification is defective and must be corrected.

Completed and remaining work are in the same document deliberately. They are two halves of one boundary; separating them guarantees they drift.

Before any command: read this file, confirm the immediately relevant live state with a read-only command, then issue one command group. If it fails, record it in FAILURES.md before changing anything else.

Production will get its own PRODUCTION-STATE.md. The specification is shared; the state is not.

1. Instance values

These are the srv-b bindings for the parameters in ENVIRONMENT.md.

Host

hostname          srv-b  /  srv-b.dev.infra
platform          Proxmox VE 8.4.0, Debian 12, kernel 6.8.12-9-pve
hardware          HP ProLiant DL360 G7, 2 x Xeon X5650, 24 threads, 31 GiB
storage           P410i, 4 x EG0146FAWHU, RAID 1+0, all SMART OK
local             directory /var/lib/vz, ~70 GiB free   iso,vztmpl,backup
local-lvm         LVM-thin pve/data, 166.9 GiB          rootdir,images
vmbr0             10.0.0.12/24 on enp3s0f0, gw 10.0.0.1   management, LAN
vmbr1             10.20.0.1/24, bridge-ports none         service, portless
wg0               10.110.0.12/32, peer wg-pk.civicus.us:51820, allowed 10.110.0.0/22
resolver          75.75.75.75, search dev.infra
spare NICs        enp3s0f1, enp4s0f0, enp4s0f1 — unconfigured, deliberately
template          local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst

Containers

CT 100 CT 101
hostname mechcomp mcproxy
role application, worker reverse proxy, TLS
features nesting=1,keyctl=1 nesting=1
cores / RAM / swap 16 / 16384 / 4096 2 / 1024 / 512
rootfs 40 GiB local-lvm 8 GiB local-lvm
mp0 60 GiB → /var/lib/mechcomp, backup=0 —
net0 eth0 on vmbr1, 10.20.0.10/24, gw 10.20.0.1 eth0 on vmbr1, 10.20.0.11/24, gw 10.20.0.1
Debian 12.15 12.15

Neither container has a LAN interface. See F-017.

Names and identity

mechanical-compiler.dev.infra   ->  10.20.0.11    (CT 101, the proxy)
mechcomp.dev.infra              ->  10.20.0.10    PVE-generated
mcproxy.dev.infra               ->  10.20.0.11    PVE-generated

ADMIN_USER                      sandor
                                uid 1000, groups sandor + mechcomp(996)
                                shell /bin/bash, both containers
authorized key                  SHA256:2pNffCscUUW5Wbs9uMepLEvMLKPqUV/7Lpk5tw7iWSY
                                root@srv-b — bastion pattern, see §4
service user                    mechcomp, uid 999, gid 996
                                home /var/www/mechcomp, shell /usr/sbin/nologin

Host NAT, as persisted in /etc/iptables/rules.v4

nat POSTROUTING
  -s 10.20.0.0/24 -d 10.0.0.0/24            -j RETURN       # LAN not translated
  -s 10.0.0.0/24                  -o wg0    -j MASQUERADE   # pre-existing
  -s 10.20.0.0/24                 -o wg0    -j MASQUERADE   # containers -> WireGuard
  -s 10.20.0.0/24                 -o vmbr0  -j MASQUERADE   # containers -> internet

filter FORWARD
  -s 10.20.0.0/24 -d 10.20.0.0/24           -j ACCEPT       # container <-> container
  -s 10.20.0.0/24 -d 10.0.0.0/24            -j DROP         # LAN blocked

Rule order is load-bearing. RETURN must precede both masquerades.

TLS

CA        CN = Mechanical Compiler Staging CA   (locally generated)
leaf      CN = mechanical-compiler.dev.infra
          SAN = DNS:mechanical-compiler.dev.infra
validity  2026-08-16 -> 2028-11-18            <-- expires, nothing renews it
trusted   srv-b, CT 100, CT 101
key mode  0600 on CT 101

No Kane County Civic Infrastructure CA issuance path exists on srv-b: no trust anchor, no step, no cfssl, no EasyRSA. Proxmox's own CA was deliberately not reused. Replacing this leaf later is two file copies and a reload.

Application environment

/etc/mechcomp/mechcomp.env, root:mechcomp, 0640:

MECHCOMP_ENV=staging
MECHCOMP_BIND=10.20.0.10
MECHCOMP_PORT=8770
MECHCOMP_BASE_URL=https://mechanical-compiler.dev.infra
MECHCOMP_DATA_DIR=/var/lib/mechcomp
MECHCOMP_LOG_LEVEL=info
MECHCOMP_DB_URL=sqlite:////var/lib/mechcomp/db/mechcomp.sqlite3
MECHCOMP_WORKER_CONCURRENCY=4
MECHCOMP_CAD_BACKEND=none
MECHCOMP_ARTIFACT_RETENTION_DAYS=30
MECHCOMP_SECRET_KEY=<generated, not recorded>

Rollback copies on disk

srv-b     /etc/network/interfaces.before-mechcomp
          /etc/hosts.before-mechcomp
          /etc/hosts.before-svcfqdn
          /etc/iptables/rules.v4.before-svcnat
          /etc/iptables/rules.v4.before-lanblock
          /root/pct-100.before-svcnet
          /root/pct-101.before-svcnet
CT 100    /etc/hosts.before-svcfqdn
CT 101    /etc/hosts.before-svcfqdn

2. Verified

Each line was demonstrated by command output, not inferred.

Host and network

  • vmbr1 created, active, 10.20.0.1/24, portless
  • Duplicate address detection run before each container creation
  • Container → internet reachable (deb.debian.org, gitea.barternetwork.us, both 200)
  • Container → WireGuard network reachable (10.110.0.12)
  • Container → LAN unreachable — the isolation requirement
  • Container → srv-b reachable at 10.0.0.12 (INPUT path, required)
  • Container ↔ container reachable over vmbr1
  • LAN → Proxmox console unaffected (10.0.0.12:8006 → 200)
  • Network configuration persisted, survives reboot

CT 100

  • Debian 12.15, systemd running, zero failed units, no pending upgrades
  • en_US.UTF-8 generated and active
  • /var/lib/mechcomp is a real separate ext4 filesystem
  • Service user mechcomp, home /var/www/mechcomp, writable
  • Application directory layout created with correct ownership
  • Base packages installed; OpenSCAD / Qt / X11 absent — verified clean
  • Docker working, overlay2 / systemd, no fuse-overlayfs needed
  • Repository cloned at e85c4f4e, verified as the owning user
  • Python 3.11.2 venv created, owned by mechcomp
  • mechcomp.env written with generated secret
  • openssh-server installed, enabled, active
  • Application listener bound to service network only — LAN-side bind refused

CT 101

  • Debian 12.15, systemd running, zero failed units after nesting=1
  • en_US.UTF-8 generated and active
  • nginx 1.22.1 installed, nginx -t passes
  • Debian default site removed; only the project vhost is enabled
  • Local CA created, leaf issued, trusted on all three hosts
  • HTTP → HTTPS redirect, TLS termination, proxy to 10.20.0.10:8770
  • openssh-server installed, enabled, active
  • X-Forwarded-Proto: https observed at the backend — needs re-proof, see §3

3. Remaining — infrastructure

In dependency order. Application-dependent work is in §5.

Immediate

  • mechcomp-placeholder.service — recreate the backend as a supervised unit. It died on the F-017 reboot and nothing currently listens on 8770. Everything below that touches the proxy depends on this. See F-019.
  • Re-prove X-Forwarded-Proto end to end. The earlier proof was taken when clients were on 10.0.0.x; header values will now read 10.20.0.x. Re-establish rather than assume it survived the topology change.
  • default_server on the CT 101 vhost, both listen 443 lines. Two words. Prevents catch-all behaviour depending on file ordering once a second server block exists. See F-012.

Mail — blocks alerting

  • Postfix relay. Currently inet_interfaces = loopback-only, relayhost empty, no root: alias. Alerts go to a mailbox nobody opens.
  • root: alias to a real destination.
  • Needs: address of the wg-pk relay, and whether it accepts unauthenticated from 10.110.0.12. Open question for CIVICVS.

Monitoring and backup

  • smartd on /dev/sda -d cciss,0 through cciss,3. Depends on mail.
  • vzdump job — both containers, local, snapshot, zstd, 02:30. mp0 already carries backup=0.
  • First vzdump run — report the archive size. Retention is a placeholder keep-last=3 until this number exists.
  • Free-space guard — refuse and alert below 20 GiB on /var/lib/vz.
  • /var/lib/vz/mechcomp-app/ and the host-side pull script.
  • mechcomp-backup --stdout in CT 100. Structure can be built and tested against the current data directories without application output.

Gold media

  • 32 GB USB stick: LUKS2, ext4, label MC-GOLD-01, /mnt/gold noauto.
  • gold-archive.sh with SHA256SUMS verification before unmount.
  • Open: where the 3+ TB USB disk is attached now that annales is out of scope. Until known, stick-to-disk copying is a manual step.

4. Access model

Confirmed by CIVICVS: no workstation access from the home LAN is required.

internet -> WireGuard -> srv-b -> containers

srv-b is the bastion. The authorized key is root@srv-b, which is consistent: root on the host can pct enter regardless, so SSH to the containers adds no privilege. It does mean every path to a container runs through srv-b — deliberate, not a limitation.

If direct WireGuard-side access to the catalogue is wanted later, that is a route addition for 10.20.0.0/24 on the hub. Not now.

No DHCP anywhere. Two containers with fixed addresses on a portless bridge is the entire address space; a DHCP server would add a daemon, a lease database and a failure mode in exchange for nothing.

5. Blocked on application code

None of this can be honestly completed while the repository is LICENSE and README.md. It is not provisioning work and should not be attempted as such.

  • requirements-base.txt / requirements-cad.txt and dependency install
  • mechcomp.service and mechcomp-worker.service
  • Reference toolchain image mechcomp/reference-toolchain:8.0.0
  • Fixture reproduction against ddd0f154…
  • Application-runtime acceptance
  • Replacing the placeholder with the real service

The Shapely port is the architect's work item and gates all of the above.

6. Open questions

# Question Blocks
1 wg-pk relay address; unauthenticated from 10.110.0.12? mail, smartd, backup alerts
2 Where is the 3+ TB USB disk attached? gold redundancy step
3 First vzdump archive size final retention value

Question 3 is answered by doing. Questions 1 and 2 need CIVICVS.

7. Closed questions

Question Answer
Kane County CA issuance path None on srv-b. Local staging CA generated instead.
openssh-server present Yes, both containers.
ADMIN_USER / key sandor; root@srv-b key, bastion pattern.
DHCP pool on 10.0.0.0/24 Not applicable. Containers are no longer on that network.
Docker storage driver overlay2 / systemd. No fallback needed.
LAN workstation access Not required. WireGuard through srv-b.
Reference in New Issue View Git Blame Copy Permalink