Update CLAUDE.md for SQLite architecture

Reflects the JSON → SQLite migration: architecture diagram now shows K8s pod + SSH/SCP remote sync, data flow documentation, updated file descriptions, startup sequence, and write flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:03:50 +09:00
parent 12fd3b5e8f
commit 1be615be99
1 changed files with 78 additions and 46 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -11,20 +11,24 @@ HAProxy Dynamic Load Balancer Management Suite - a system for runtime management
 ## Architecture

 ```
-┌──────────────────────────────────────────────────┐
-│ HAProxy (host network)                           │
-│ TCP 80/443 (HTTP/1.1, HTTP/2) + UDP 443 (HTTP/3) │
-│ Runtime API: localhost:9999                      │
-│ MCP Proxy: mcp.inouter.com → localhost:8000      │
-└──────────────┬───────────────────────────────────┘
-               │
-         ┌─────▼─────┐
-         │ MCP       │
-         │ Server    │
-         │ :8000     │
-         └───────────┘
+┌─ K8s Pod (MCP Server) ──────────┐    SSH/SCP    ┌─ HAProxy Host ──────────────────┐
+│                                  │ ───────────→  │                                 │
+│  /app/data/haproxy_mcp.db       │   Runtime API  │  HAProxy (host network)         │
+│  (SQLite - local working copy)  │ ───────────→   │  TCP 80/443 + UDP 443 (HTTP/3)  │
+│                                  │   :9999       │  Runtime API: localhost:9999     │
+│  MCP Server :8000               │               │                                 │
+│  Transport: streamable-http     │   SCP sync    │  /opt/haproxy/conf/             │
+│                                  │ ←──────────→  │    haproxy_mcp.db (persistent)  │
+│                                  │   map sync    │    domains.map                  │
+│                                  │ ───────────→  │    wildcards.map                │
+└──────────────────────────────────┘               └─────────────────────────────────┘
 ```

+### Data Flow
+- **SQLite DB** (`haproxy_mcp.db`): Single source of truth for domains, servers, certificates
+- **Map files** (`domains.map`, `wildcards.map`): Generated from SQLite, HAProxy reads directly
+- **Runtime API**: Real-time HAProxy configuration (map entries, server addresses)
+
 ### Domain Routing Flow
 ```
 Request → HAProxy Frontend
@@ -33,7 +37,7 @@ domains.map lookup (Host header)
    ↓
 pool_N backend (N = 1-100)
    ↓
-Server (from servers.json, auto-restored on startup)
+Server (from SQLite DB, auto-restored on startup)
 ```

 ## Services
@@ -58,7 +62,7 @@ journalctl -u haproxy-mcp -f    # Logs
 - Transport: `streamable-http`
 - Internal: `http://localhost:8000/mcp`
 - External: `https://mcp.inouter.com/mcp` (via HAProxy)
- **Auto-restore:** Servers restored from `servers.json` on startup
+- **Auto-restore:** Servers restored from SQLite DB on startup

 ### Environment Variables
 | Variable | Default | Description |
@@ -69,7 +73,10 @@ journalctl -u haproxy-mcp -f    # Logs
 | `HAPROXY_PORT` | `9999` | HAProxy Runtime API port |
 | `HAPROXY_STATE_FILE` | `/opt/haproxy/data/servers.state` | State file path |
 | `HAPROXY_MAP_FILE` | `/opt/haproxy/conf/domains.map` | Map file path |
-| `HAPROXY_SERVERS_FILE` | `/opt/haproxy/conf/servers.json` | Servers config path |
+| `HAPROXY_DB_FILE` | `/opt/haproxy/conf/haproxy_mcp.db` | Local SQLite DB path |
+| `HAPROXY_REMOTE_DB_FILE` | `/opt/haproxy/conf/haproxy_mcp.db` | Remote SQLite DB path (SCP target) |
+| `HAPROXY_SERVERS_FILE` | `/opt/haproxy/conf/servers.json` | Legacy servers config (migration only) |
+| `HAPROXY_CERTS_FILE` | `/opt/haproxy/conf/certificates.json` | Legacy certs config (migration only) |
 | `HAPROXY_POOL_COUNT` | `100` | Number of pool backends |
 | `HAPROXY_MAX_SLOTS` | `10` | Max servers per pool |
 | `HAPROXY_CONTAINER` | `haproxy` | Podman container name |
@@ -78,10 +85,10 @@ journalctl -u haproxy-mcp -f    # Logs
 ## Zero-Reload Domain Management

 ### How It Works
-1. **domains.map**: Maps domain → pool backend (e.g., `example.com pool_1`)
-2. **Pool backends**: 100 pre-configured backends (`pool_1` to `pool_100`)
-3. **Runtime API**: Add/remove map entries and configure servers without reload
-4. **servers.json**: Persistent server configuration, auto-restored on MCP startup
+1. **SQLite DB** (`haproxy_mcp.db`): Single source of truth for all persistent data
+2. **domains.map**: Generated from SQLite, maps domain → pool backend
+3. **Pool backends**: 100 pre-configured backends (`pool_1` to `pool_100`)
+4. **Runtime API**: Add/remove map entries and configure servers without reload

 ### Adding a Domain

@@ -94,11 +101,12 @@ haproxy_add_domain("api.example.com", "10.0.0.1", 8080)
 ```

 This will:
-1. Find available pool (e.g., `pool_5`)
-2. Add to `domains.map`: `example.com pool_5`
-3. Update HAProxy runtime map: `add map ... example.com pool_5`
-4. Configure server in pool via Runtime API (HTTP only)
-5. Save to `servers.json` for persistence
+1. Find available pool (e.g., `pool_5`) via SQLite query
+2. Save to SQLite DB (domain, backend, server)
+3. Sync `domains.map` from DB (for HAProxy file-based lookup)
+4. Update HAProxy runtime map: `add map ... example.com pool_5`
+5. Configure server in pool via Runtime API (HTTP only)
+6. Upload DB to remote host via SCP (persistence)

 **No HAProxy reload required!**

@@ -120,7 +128,7 @@ haproxy_add_domain("api.example.com", share_with="example.com")
 **Benefits:**
 - Saves pool slots (100 pools can serve unlimited domains)
 - Multiple domains → same backend servers
- Shared domains stored as `_shares` reference in `servers.json`
+- Shared domains stored as `shares_with` in SQLite domains table

 **Behavior:**
 - Shared domains cannot specify `ip` (use existing servers)
@@ -146,9 +154,12 @@ haproxy_add_servers("api.example.com", '[
 ### Files
 | File | Purpose |
 |------|---------|
-| `domains.map` | Domain → Pool mapping |
-| `servers.json` | Server IP/port persistence |
+| `haproxy_mcp.db` | SQLite DB — single source of truth (domains, servers, certs) |
+| `domains.map` | Domain → Pool mapping (generated from DB, HAProxy reads directly) |
+| `wildcards.map` | Wildcard domain mapping (generated from DB) |
 | `haproxy.cfg` | Pool backend definitions (static) |
+| `servers.json` | Legacy — used only for initial migration to SQLite |
+| `certificates.json` | Legacy — used only for initial migration to SQLite |

 ## SSL/TLS Certificates

@@ -243,7 +254,7 @@ Returns overall system status:
  "components": {
    "mcp": {"status": "ok"},
    "haproxy": {"status": "ok", "version": "3.3.2", "uptime_sec": 3600},
-    "config_files": {"status": "ok", "files": {"map_file": "ok", "servers_file": "ok"}},
+    "config_files": {"status": "ok", "files": {"map_file": "ok", "db_file": "ok"}},
    "container": {"status": "ok", "state": "running"}
  }
 }
@@ -342,15 +353,17 @@ api.example.com → pool_6
  └─ pool_6_1 (slot 1)
 ```

-### Persistence
- **domains.map**: Survives HAProxy restart
- **servers.json**: Auto-restored by MCP on startup
- No manual save required - `haproxy_add_server` auto-saves
+### Persistence (SQLite)
+- **SQLite DB** (`haproxy_mcp.db`): All domains, servers, certificates stored in single file
+- **Remote sync**: DB uploaded to HAProxy host via SCP after every write
+- **Startup restore**: DB downloaded from remote on pod start, servers restored via Runtime API
+- **Migration**: First startup auto-migrates from legacy JSON/map files (one-time, idempotent)
+- **WAL mode**: Write-Ahead Logging for concurrent access, checkpointed before SCP upload

 ### Safety Features
- **Atomic file writes**: Temp file + rename prevents corruption
- **File locking**: Prevents race conditions on concurrent operations
- **Disk-first pattern**: Config saved before HAProxy update, rollback on failure
+- **Atomic file writes**: Temp file + rename prevents corruption (map files)
+- **SQLite transactions**: All DB writes are atomic
+- **DB-first pattern**: Config saved to SQLite before HAProxy update, rollback on failure
 - **Command validation**: HAProxy responses checked for errors
 - **Input validation**: Domain format, IP (v4/v6), port range, slot limits
 - **Bulk limits**: Max 10 servers per bulk add, 10KB JSON size limit
@@ -365,7 +378,7 @@ api.example.com → pool_6
 - **Runtime API**: Certificates loaded/updated without HAProxy reload
  - `new ssl cert` → `set ssl cert` → `commit ssl cert`
  - No connection drops during certificate changes
- **Persistence**: `certificates.json` stores domain list
+- **Persistence**: Certificate domains stored in SQLite `certificates` table
 - **Auto-restore**: Certificates reloaded into HAProxy on MCP startup

 ## HAProxy Runtime API
@@ -397,10 +410,12 @@ echo "set server pool_1/pool_1_1 state ready" | nc localhost 9999
 ├── haproxy_mcp/         # MCP server package (streamable-http)
 │   ├── server.py        # Main entry point
 │   ├── config.py        # Configuration and constants
+│   ├── db.py            # SQLite database (single source of truth)
+│   ├── ssh_ops.py       # SSH/SCP remote execution
 │   ├── exceptions.py    # Exception classes
 │   ├── validation.py    # Input validation
 │   ├── haproxy_client.py # HAProxy Runtime API client
-│   ├── file_ops.py      # File I/O operations
+│   ├── file_ops.py      # File I/O (delegates to db.py)
 │   ├── utils.py         # Parsing utilities
 │   └── tools/           # MCP tools (29 total)
 │       ├── domains.py   # Domain management (3 tools)
@@ -411,9 +426,11 @@ echo "set server pool_1/pool_1_1 state ready" | nc localhost 9999
 │       └── certificates.py # Certificate management (7 tools)
 ├── conf/
 │   ├── haproxy.cfg      # Main HAProxy config (100 pool backends)
-│   ├── domains.map      # Domain → Pool mapping
-│   ├── servers.json     # Server persistence (auto-managed)
-│   ├── certificates.json # Certificate domain list (auto-managed)
+│   ├── haproxy_mcp.db   # SQLite DB (single source of truth)
+│   ├── domains.map      # Domain → Pool mapping (generated from DB)
+│   ├── wildcards.map    # Wildcard domain mapping (generated from DB)
+│   ├── servers.json     # Legacy (migration source only)
+│   ├── certificates.json # Legacy (migration source only)
 │   └── mcp-token.env    # Bearer token for MCP auth
 ├── certs/               # SSL/TLS certificates (HAProxy PEM format)
 ├── data/                # Legacy state files
@@ -433,8 +450,9 @@ echo "set server pool_1/pool_1_1 state ready" | nc localhost 9999
 | File | Purpose |
 |------|---------|
 | `/opt/haproxy/conf/haproxy.cfg` | HAProxy main config |
-| `/opt/haproxy/conf/domains.map` | Domain routing map |
-| `/opt/haproxy/conf/servers.json` | Server persistence |
+| `/opt/haproxy/conf/haproxy_mcp.db` | SQLite DB (single source of truth) |
+| `/opt/haproxy/conf/domains.map` | Domain routing map (generated from DB) |
+| `/opt/haproxy/conf/wildcards.map` | Wildcard domain map (generated from DB) |
 | `/opt/haproxy/certs/*.pem` | SSL certificates (auto-loaded) |
 | `/etc/containers/systemd/haproxy.container` | Podman Quadlet |
 | `/etc/systemd/system/haproxy-mcp.service` | MCP service |
@@ -451,13 +469,27 @@ echo "set server pool_1/pool_1_1 state ready" | nc localhost 9999
   ↓
 3. HAProxy loads domains.map (domain → pool mapping)
   ↓
-4. systemd starts haproxy-mcp.service
+4. systemd starts haproxy-mcp.service (or K8s pod starts)
   ↓
-5. MCP server reads servers.json & certificates.json
+5. init_db(): Download SQLite DB from remote host via SCP
+   (first time: migrate from legacy JSON/map files → create DB → upload)
   ↓
-6. MCP server restores servers via Runtime API
+6. MCP server restores servers from DB via Runtime API
   ↓
-7. MCP server loads certificates via Runtime API (zero-downtime)
+7. MCP server loads certificates from DB via Runtime API (zero-downtime)
   ↓
 8. Ready to serve traffic
 ```
+
+### Write Flow (runtime)
+```
+MCP tool call (add domain/server/cert)
+   ↓
+Write to local SQLite DB
+   ↓
+Sync map files to remote (domains only)
+   ↓
+Update HAProxy via Runtime API
+   ↓
+Upload DB to remote host via SCP (persistence)
+```