Add infrastructure documentation and project setup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
otivm
285
docs/provisioning.md
Normal file
285
docs/provisioning.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Container Provisioning — srv-a Ground Rules
|
||||
|
||||
*Established: April 2026*
|
||||
|
||||
This document records the correct procedure for provisioning new Debian 12 LXC
|
||||
containers on srv-a (dev Proxmox, 10.0.0.11). Follow this exactly.
|
||||
Every step was learned from direct experience — do not skip any of them.
|
||||
|
||||
---
|
||||
|
||||
## 1. Create the Container
|
||||
|
||||
```bash
|
||||
pct create <VMID> local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst \
|
||||
--hostname <name> \
|
||||
--memory 512 \
|
||||
--cores 2 \
|
||||
--rootfs local-lvm:8 \
|
||||
--net0 name=eth0,bridge=vmbr0,gw=10.0.0.1,ip=10.0.0.<X>/24,type=veth \
|
||||
--nameserver 10.0.0.1 \
|
||||
--unprivileged 1 \
|
||||
--start 1
|
||||
```
|
||||
|
||||
Check available template name first:
|
||||
```bash
|
||||
pveam list local
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. IP Address Allocation
|
||||
|
||||
| IP | Container |
|
||||
|----|-----------|
|
||||
| 10.0.0.10 | apt-cache (CT 1104) |
|
||||
| 10.0.0.20 | tessera-pipeline (CT 1101) |
|
||||
| 10.0.0.21 | tessera-store (CT 1102) |
|
||||
| 10.0.0.22 | tessera-dev (CT 1103) |
|
||||
| 10.0.0.23 | otivm-dev (CT 1105) |
|
||||
|
||||
Next available: 10.0.0.24
|
||||
|
||||
---
|
||||
|
||||
## 3. WireGuard IP Allocation
|
||||
|
||||
| IP | Node |
|
||||
|----|------|
|
||||
| 10.110.0.1 | wg-pk (Linode hub) |
|
||||
| 10.110.0.2 | shell.infra.civicus.us |
|
||||
| 10.110.0.4 | fw (CVSTOS push node) |
|
||||
| 10.110.0.5 | OVH node |
|
||||
| 10.110.0.6 | corpusdb.infra.civicus.us |
|
||||
| 10.110.0.7–0.14 | Home lab containers |
|
||||
| 10.110.0.15 | OVH node |
|
||||
| 10.110.0.16 | OVH node |
|
||||
| 10.110.0.17 | mcp.civicus.us |
|
||||
| 10.110.0.18 | otivm-dev |
|
||||
|
||||
Next available: 10.110.0.19
|
||||
|
||||
---
|
||||
|
||||
## 4. Set Root Password
|
||||
|
||||
From srv-a (not from inside the container):
|
||||
```bash
|
||||
pct exec <VMID> -- passwd root
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Fix Locale
|
||||
|
||||
⚠ Do NOT use `update-locale` — it fails in this environment.
|
||||
|
||||
```bash
|
||||
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
|
||||
locale-gen
|
||||
echo "LANG=en_US.UTF-8" > /etc/default/locale
|
||||
export LANG=en_US.UTF-8
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Configure apt Proxy
|
||||
|
||||
The apt-cache container (10.0.0.10) runs apt-cacher-ng on port 3142.
|
||||
All new containers must be configured to use it.
|
||||
|
||||
⚠ Two lines required — HTTP through cache, HTTPS direct:
|
||||
|
||||
```bash
|
||||
echo 'Acquire::http::Proxy "http://10.0.0.10:3142";' > /etc/apt/apt.conf.d/01proxy
|
||||
echo 'Acquire::https::Proxy "DIRECT";' >> /etc/apt/apt.conf.d/01proxy
|
||||
```
|
||||
|
||||
The HTTPS direct line is required because apt-cacher-ng cannot proxy
|
||||
HTTPS tunnels. Without it, any HTTPS apt source (e.g. nodesource) will
|
||||
fail with "403 CONNECT denied".
|
||||
|
||||
---
|
||||
|
||||
## 7. Install Base Packages
|
||||
|
||||
```bash
|
||||
apt update && apt upgrade -y
|
||||
apt install -y git python3 python3-venv python3-pip curl wget wireguard
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Install Node.js v22
|
||||
|
||||
⚠ Do NOT use the nodesource setup script (`curl ... | bash`) — it
|
||||
creates conflicting source files. Use the manual method:
|
||||
|
||||
```bash
|
||||
mkdir -p /etc/apt/keyrings
|
||||
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \
|
||||
gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
|
||||
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] \
|
||||
https://deb.nodesource.com/node_22.x nodistro main" \
|
||||
> /etc/apt/sources.list.d/nodesource.list
|
||||
apt update && apt install -y nodejs
|
||||
node --version # should be v22.x.x
|
||||
npm --version
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Install WireGuard
|
||||
|
||||
```bash
|
||||
apt install -y wireguard
|
||||
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
|
||||
chmod 600 /etc/wireguard/private.key
|
||||
cat /etc/wireguard/public.key # give this to wg-pk
|
||||
```
|
||||
|
||||
Create /etc/wireguard/wg0.conf:
|
||||
```ini
|
||||
[Interface]
|
||||
Address = 10.110.0.<X>/32
|
||||
ListenPort = 51820
|
||||
PrivateKey = <contents of /etc/wireguard/private.key>
|
||||
|
||||
[Peer]
|
||||
PublicKey = 1+Wb++fjXNbY0joOvj4AZvJgF6b125YOPSFsmNqVo3I=
|
||||
AllowedIPs = 10.110.0.0/22
|
||||
Endpoint = 198.58.111.109:51820
|
||||
PersistentKeepalive = 25
|
||||
```
|
||||
|
||||
```bash
|
||||
systemctl enable wg-quick@wg0
|
||||
systemctl start wg-quick@wg0
|
||||
ping -c 3 10.110.0.1 # confirm mesh
|
||||
```
|
||||
|
||||
On wg-pk — add peer (live, no restart needed):
|
||||
```bash
|
||||
wg set wg0 peer <pubkey> allowed-ips 10.110.0.<X>/32
|
||||
# append to /etc/wireguard/wg0.conf as well
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Create App User
|
||||
|
||||
```bash
|
||||
useradd -m -s /bin/bash <appuser>
|
||||
passwd <appuser>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. Python Venv
|
||||
|
||||
Always as the app user, never as root:
|
||||
```bash
|
||||
su - <appuser>
|
||||
python3 -m venv /home/<appuser>/venv
|
||||
echo 'source /home/<appuser>/venv/bin/activate' >> /home/<appuser>/.bashrc
|
||||
```
|
||||
|
||||
⚠ Never run Python outside the venv. This is an absolute rule.
|
||||
|
||||
---
|
||||
|
||||
## 12. Install PM2
|
||||
|
||||
As root:
|
||||
```bash
|
||||
npm install -g pm2
|
||||
```
|
||||
|
||||
Set up systemd service under app user:
|
||||
```bash
|
||||
env PATH=$PATH:/usr/bin pm2 startup systemd -u <appuser> --hp /home/<appuser>
|
||||
```
|
||||
|
||||
PM2 must always run as the app user. Never as root.
|
||||
|
||||
---
|
||||
|
||||
## 13. Install Claude Code
|
||||
|
||||
As the app user:
|
||||
```bash
|
||||
su - <appuser>
|
||||
mkdir -p ~/.npm-global
|
||||
npm config set prefix '~/.npm-global'
|
||||
echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.bashrc
|
||||
source ~/.bashrc
|
||||
npm install -g @anthropic-ai/claude-code
|
||||
claude --version
|
||||
```
|
||||
|
||||
Authenticate on first run:
|
||||
```bash
|
||||
cd ~/REPONAME
|
||||
claude
|
||||
# Select: Claude account with subscription
|
||||
# Follow browser-based login
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 14. Install Webmin
|
||||
|
||||
As root:
|
||||
```bash
|
||||
curl -o webmin-setup-repo.sh https://raw.githubusercontent.com/webmin/webmin/master/webmin-setup-repo.sh
|
||||
sh webmin-setup-repo.sh
|
||||
apt install -y webmin
|
||||
systemctl disable webmin # do not autostart
|
||||
```
|
||||
|
||||
Add aliases to /root/.bashrc:
|
||||
```bash
|
||||
echo "alias webmin-on='systemctl start webmin && echo Webmin started'" >> /root/.bashrc
|
||||
echo "alias webmin-off='systemctl stop webmin && echo Webmin stopped'" >> /root/.bashrc
|
||||
source /root/.bashrc
|
||||
```
|
||||
|
||||
Webmin listens on port 10000. Start it only when needed.
|
||||
|
||||
---
|
||||
|
||||
## 15. SSL Certificate Rule
|
||||
|
||||
⚠ ABSOLUTE RULE for all nodes with Nginx:
|
||||
|
||||
Issue the SSL certificate FIRST with --nginx authenticator.
|
||||
Add any basic auth AFTER the certificate is issued.
|
||||
Never reverse this order — basic auth blocks the ACME challenge.
|
||||
|
||||
```bash
|
||||
# 1. Create vhost with no auth
|
||||
# 2. Dry run
|
||||
certbot certonly --nginx -d <domain> --dry-run
|
||||
# 3. Issue real certificate
|
||||
certbot certonly --nginx -d <domain>
|
||||
# 4. Add SSL stanzas to vhost
|
||||
# 5. Add basic auth if needed
|
||||
# 6. Reload nginx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 16. Take Baseline Backup
|
||||
|
||||
Before deploying any code, take a backup:
|
||||
```bash
|
||||
vzdump <VMID> --storage local --compress zstd --mode stop
|
||||
```
|
||||
|
||||
Download via Webmin → File Manager → /var/lib/vz/dump/
|
||||
Store offline. This is the recovery point.
|
||||
|
||||
---
|
||||
|
||||
*docs/provisioning.md — TheRON srv-a — April 2026*
|
||||
Reference in New Issue
Block a user