286 lines
6.1 KiB
Markdown
286 lines
6.1 KiB
Markdown
# Container Provisioning — srv-a Ground Rules
|
||
|
||
*Established: April 2026*
|
||
|
||
This document records the correct procedure for provisioning new Debian 12 LXC
|
||
containers on srv-a (dev Proxmox, 10.0.0.11). Follow this exactly.
|
||
Every step was learned from direct experience — do not skip any of them.
|
||
|
||
---
|
||
|
||
## 1. Create the Container
|
||
|
||
```bash
|
||
pct create <VMID> local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst \
|
||
--hostname <name> \
|
||
--memory 512 \
|
||
--cores 2 \
|
||
--rootfs local-lvm:8 \
|
||
--net0 name=eth0,bridge=vmbr0,gw=10.0.0.1,ip=10.0.0.<X>/24,type=veth \
|
||
--nameserver 10.0.0.1 \
|
||
--unprivileged 1 \
|
||
--start 1
|
||
```
|
||
|
||
Check available template name first:
|
||
```bash
|
||
pveam list local
|
||
```
|
||
|
||
---
|
||
|
||
## 2. IP Address Allocation
|
||
|
||
| IP | Container |
|
||
|----|-----------|
|
||
| 10.0.0.10 | apt-cache (CT 1104) |
|
||
| 10.0.0.20 | tessera-pipeline (CT 1101) |
|
||
| 10.0.0.21 | tessera-store (CT 1102) |
|
||
| 10.0.0.22 | tessera-dev (CT 1103) |
|
||
| 10.0.0.23 | otivm-dev (CT 1105) |
|
||
|
||
Next available: 10.0.0.24
|
||
|
||
---
|
||
|
||
## 3. WireGuard IP Allocation
|
||
|
||
| IP | Node |
|
||
|----|------|
|
||
| 10.110.0.1 | wg-pk (Linode hub) |
|
||
| 10.110.0.2 | shell.infra.civicus.us |
|
||
| 10.110.0.4 | fw (CVSTOS push node) |
|
||
| 10.110.0.5 | OVH node |
|
||
| 10.110.0.6 | corpusdb.infra.civicus.us |
|
||
| 10.110.0.7–0.14 | Home lab containers |
|
||
| 10.110.0.15 | OVH node |
|
||
| 10.110.0.16 | OVH node |
|
||
| 10.110.0.17 | mcp.civicus.us |
|
||
| 10.110.0.18 | otivm-dev |
|
||
|
||
Next available: 10.110.0.19
|
||
|
||
---
|
||
|
||
## 4. Set Root Password
|
||
|
||
From srv-a (not from inside the container):
|
||
```bash
|
||
pct exec <VMID> -- passwd root
|
||
```
|
||
|
||
---
|
||
|
||
## 5. Fix Locale
|
||
|
||
⚠ Do NOT use `update-locale` — it fails in this environment.
|
||
|
||
```bash
|
||
echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
|
||
locale-gen
|
||
echo "LANG=en_US.UTF-8" > /etc/default/locale
|
||
export LANG=en_US.UTF-8
|
||
```
|
||
|
||
---
|
||
|
||
## 6. Configure apt Proxy
|
||
|
||
The apt-cache container (10.0.0.10) runs apt-cacher-ng on port 3142.
|
||
All new containers must be configured to use it.
|
||
|
||
⚠ Two lines required — HTTP through cache, HTTPS direct:
|
||
|
||
```bash
|
||
echo 'Acquire::http::Proxy "http://10.0.0.10:3142";' > /etc/apt/apt.conf.d/01proxy
|
||
echo 'Acquire::https::Proxy "DIRECT";' >> /etc/apt/apt.conf.d/01proxy
|
||
```
|
||
|
||
The HTTPS direct line is required because apt-cacher-ng cannot proxy
|
||
HTTPS tunnels. Without it, any HTTPS apt source (e.g. nodesource) will
|
||
fail with "403 CONNECT denied".
|
||
|
||
---
|
||
|
||
## 7. Install Base Packages
|
||
|
||
```bash
|
||
apt update && apt upgrade -y
|
||
apt install -y git python3 python3-venv python3-pip curl wget wireguard
|
||
```
|
||
|
||
---
|
||
|
||
## 8. Install Node.js v22
|
||
|
||
⚠ Do NOT use the nodesource setup script (`curl ... | bash`) — it
|
||
creates conflicting source files. Use the manual method:
|
||
|
||
```bash
|
||
mkdir -p /etc/apt/keyrings
|
||
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | \
|
||
gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
|
||
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] \
|
||
https://deb.nodesource.com/node_22.x nodistro main" \
|
||
> /etc/apt/sources.list.d/nodesource.list
|
||
apt update && apt install -y nodejs
|
||
node --version # should be v22.x.x
|
||
npm --version
|
||
```
|
||
|
||
---
|
||
|
||
## 9. Install WireGuard
|
||
|
||
```bash
|
||
apt install -y wireguard
|
||
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
|
||
chmod 600 /etc/wireguard/private.key
|
||
cat /etc/wireguard/public.key # give this to wg-pk
|
||
```
|
||
|
||
Create /etc/wireguard/wg0.conf:
|
||
```ini
|
||
[Interface]
|
||
Address = 10.110.0.<X>/32
|
||
ListenPort = 51820
|
||
PrivateKey = <contents of /etc/wireguard/private.key>
|
||
|
||
[Peer]
|
||
PublicKey = 1+Wb++fjXNbY0joOvj4AZvJgF6b125YOPSFsmNqVo3I=
|
||
AllowedIPs = 10.110.0.0/22
|
||
Endpoint = 198.58.111.109:51820
|
||
PersistentKeepalive = 25
|
||
```
|
||
|
||
```bash
|
||
systemctl enable wg-quick@wg0
|
||
systemctl start wg-quick@wg0
|
||
ping -c 3 10.110.0.1 # confirm mesh
|
||
```
|
||
|
||
On wg-pk — add peer (live, no restart needed):
|
||
```bash
|
||
wg set wg0 peer <pubkey> allowed-ips 10.110.0.<X>/32
|
||
# append to /etc/wireguard/wg0.conf as well
|
||
```
|
||
|
||
---
|
||
|
||
## 10. Create App User
|
||
|
||
```bash
|
||
useradd -m -s /bin/bash <appuser>
|
||
passwd <appuser>
|
||
```
|
||
|
||
---
|
||
|
||
## 11. Python Venv
|
||
|
||
Always as the app user, never as root:
|
||
```bash
|
||
su - <appuser>
|
||
python3 -m venv /home/<appuser>/venv
|
||
echo 'source /home/<appuser>/venv/bin/activate' >> /home/<appuser>/.bashrc
|
||
```
|
||
|
||
⚠ Never run Python outside the venv. This is an absolute rule.
|
||
|
||
---
|
||
|
||
## 12. Install PM2
|
||
|
||
As root:
|
||
```bash
|
||
npm install -g pm2
|
||
```
|
||
|
||
Set up systemd service under app user:
|
||
```bash
|
||
env PATH=$PATH:/usr/bin pm2 startup systemd -u <appuser> --hp /home/<appuser>
|
||
```
|
||
|
||
PM2 must always run as the app user. Never as root.
|
||
|
||
---
|
||
|
||
## 13. Install Claude Code
|
||
|
||
As the app user:
|
||
```bash
|
||
su - <appuser>
|
||
mkdir -p ~/.npm-global
|
||
npm config set prefix '~/.npm-global'
|
||
echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.bashrc
|
||
source ~/.bashrc
|
||
npm install -g @anthropic-ai/claude-code
|
||
claude --version
|
||
```
|
||
|
||
Authenticate on first run:
|
||
```bash
|
||
cd ~/REPONAME
|
||
claude
|
||
# Select: Claude account with subscription
|
||
# Follow browser-based login
|
||
```
|
||
|
||
---
|
||
|
||
## 14. Install Webmin
|
||
|
||
As root:
|
||
```bash
|
||
curl -o webmin-setup-repo.sh https://raw.githubusercontent.com/webmin/webmin/master/webmin-setup-repo.sh
|
||
sh webmin-setup-repo.sh
|
||
apt install -y webmin
|
||
systemctl disable webmin # do not autostart
|
||
```
|
||
|
||
Add aliases to /root/.bashrc:
|
||
```bash
|
||
echo "alias webmin-on='systemctl start webmin && echo Webmin started'" >> /root/.bashrc
|
||
echo "alias webmin-off='systemctl stop webmin && echo Webmin stopped'" >> /root/.bashrc
|
||
source /root/.bashrc
|
||
```
|
||
|
||
Webmin listens on port 10000. Start it only when needed.
|
||
|
||
---
|
||
|
||
## 15. SSL Certificate Rule
|
||
|
||
⚠ ABSOLUTE RULE for all nodes with Nginx:
|
||
|
||
Issue the SSL certificate FIRST with --nginx authenticator.
|
||
Add any basic auth AFTER the certificate is issued.
|
||
Never reverse this order — basic auth blocks the ACME challenge.
|
||
|
||
```bash
|
||
# 1. Create vhost with no auth
|
||
# 2. Dry run
|
||
certbot certonly --nginx -d <domain> --dry-run
|
||
# 3. Issue real certificate
|
||
certbot certonly --nginx -d <domain>
|
||
# 4. Add SSL stanzas to vhost
|
||
# 5. Add basic auth if needed
|
||
# 6. Reload nginx
|
||
```
|
||
|
||
---
|
||
|
||
## 16. Take Baseline Backup
|
||
|
||
Before deploying any code, take a backup:
|
||
```bash
|
||
vzdump <VMID> --storage local --compress zstd --mode stop
|
||
```
|
||
|
||
Download via Webmin → File Manager → /var/lib/vz/dump/
|
||
Store offline. This is the recovery point.
|
||
|
||
---
|
||
|
||
*docs/provisioning.md — TheRON srv-a — April 2026*
|