haproxy-mcp/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

HAProxy Dynamic Load Balancer Management Suite - a system for runtime management of HAProxy without restarts. Provides MCP interface for dynamic domain/server management. Frontend supports HTTP/HTTPS/HTTP3, backends use HTTP only (SSL termination at HAProxy).

Key Feature: Zero-reload domain management using HAProxy map files and pool backends.

Architecture

┌─ K8s Pod (MCP Server) ──────────┐    SSH/SCP    ┌─ HAProxy Host ──────────────────┐
│                                  │ ───────────→  │                                 │
│  /app/data/haproxy_mcp.db       │   Runtime API  │  HAProxy (host network)         │
│  (SQLite - local working copy)  │ ───────────→   │  TCP 80/443 + UDP 443 (HTTP/3)  │
│                                  │   :9999       │  Runtime API: localhost:9999     │
│  MCP Server :8000               │               │                                 │
│  Transport: streamable-http     │   SCP sync    │  /opt/haproxy/conf/             │
│                                  │ ←──────────→  │    haproxy_mcp.db (persistent)  │
│                                  │   map sync    │    domains.map                  │
│                                  │ ───────────→  │    wildcards.map                │
└──────────────────────────────────┘               └─────────────────────────────────┘

Data Flow

• SQLite DB (haproxy_mcp.db): Single source of truth for domains, servers, certificates
• Map files (domains.map, wildcards.map): Generated from SQLite, HAProxy reads directly
• Runtime API: Real-time HAProxy configuration (map entries, server addresses)

Domain Routing Flow

Request → HAProxy Frontend
    ↓
domains.map lookup (Host header)
    ↓
pool_N backend (N = 1-100)
    ↓
Server (from SQLite DB, auto-restored on startup)

Services

HAProxy (Podman Quadlet)

systemctl start haproxy      # Start
systemctl stop haproxy       # Stop
systemctl restart haproxy    # Restart
systemctl status haproxy     # Status

• Quadlet config: /etc/containers/systemd/haproxy.container
• Network mode: host (direct access to host services)

MCP Server (systemd)

systemctl start haproxy-mcp     # Start
systemctl stop haproxy-mcp      # Stop
systemctl restart haproxy-mcp   # Restart
journalctl -u haproxy-mcp -f    # Logs

• Transport: streamable-http
• Internal: http://localhost:8000/mcp
• External: https://mcp.inouter.com/mcp (via HAProxy)
• Auto-restore: Servers restored from SQLite DB on startup

Environment Variables

Variable	Default	Description
MCP_HOST	0.0.0.0	MCP server bind host
MCP_PORT	8000	MCP server port
HAPROXY_HOST	localhost	HAProxy Runtime API host
HAPROXY_PORT	9999	HAProxy Runtime API port
HAPROXY_STATE_FILE	/opt/haproxy/data/servers.state	State file path
HAPROXY_MAP_FILE	/opt/haproxy/conf/domains.map	Map file path
HAPROXY_DB_FILE	/opt/haproxy/conf/haproxy_mcp.db	Local SQLite DB path
HAPROXY_REMOTE_DB_FILE	/opt/haproxy/conf/haproxy_mcp.db	Remote SQLite DB path (SCP target)
HAPROXY_SERVERS_FILE	/opt/haproxy/conf/servers.json	Legacy servers config (migration only)
HAPROXY_CERTS_FILE	/opt/haproxy/conf/certificates.json	Legacy certs config (migration only)
HAPROXY_CUSTOM_TLDS	it.com	Custom multi-part TLDs (comma-separated) for subdomain detection
HAPROXY_POOL_COUNT	100	Number of pool backends
HAPROXY_MAX_SLOTS	10	Max servers per pool
HAPROXY_CONTAINER	haproxy	Podman container name
LOG_LEVEL	INFO	Logging level (DEBUG/INFO/WARNING/ERROR)

Zero-Reload Domain Management

How It Works

1. SQLite DB (haproxy_mcp.db): Single source of truth for all persistent data
2. domains.map: Generated from SQLite, maps domain → pool backend
3. Pool backends: 100 pre-configured backends (pool_1 to pool_100)
4. Runtime API: Add/remove map entries and configure servers without reload

Adding a Domain

haproxy_add_domain("example.com", "10.0.0.1")
# Creates: pool_N_1 → 10.0.0.1:80

haproxy_add_domain("api.example.com", "10.0.0.1", 8080)
# Creates: pool_N_1 → 10.0.0.1:8080

This will:

1. Find available pool (e.g., pool_5) via SQLite query
2. Save to SQLite DB (domain, backend, server)
3. Sync domains.map from DB (for HAProxy file-based lookup)
4. Update HAProxy runtime map: add map ... example.com pool_5
5. Configure server in pool via Runtime API (HTTP only)
6. Upload DB to remote host via SCP (persistence)

No HAProxy reload required!

Pool Sharing

Multiple domains can share the same pool/backend servers using share_with:

# First domain gets its own pool
haproxy_add_domain("example.com", "10.0.0.1")
# Creates: example.com → pool_5

# Additional domains share the same pool
haproxy_add_domain("www.example.com", share_with="example.com")
haproxy_add_domain("api.example.com", share_with="example.com")
# Both use: pool_5 (same backend servers)

Benefits:

• Saves pool slots (100 pools can serve unlimited domains)
• Multiple domains → same backend servers
• Shared domains stored as shares_with in SQLite domains table

Behavior:

• Shared domains cannot specify ip (use existing servers)
• Removing a shared domain keeps servers intact
• Removing the last domain using a pool clears the servers

Backend Configuration

• Backends always use HTTP (port 80 or custom)
• SSL/TLS termination happens at HAProxy frontend
• Each pool has 10 server slots (pool_N_1 to pool_N_10)
• IPv6 supported: Both IPv4 and IPv6 addresses accepted

Bulk Server Operations

# Add multiple servers at once
haproxy_add_servers("api.example.com", '[
  {"slot": 1, "ip": "10.0.0.1", "http_port": 80},
  {"slot": 2, "ip": "10.0.0.2", "http_port": 80},
  {"slot": 3, "ip": "2001:db8::1", "http_port": 80}
]')

Files

File	Purpose
haproxy_mcp.db	SQLite DB — single source of truth (domains, servers, certs)
domains.map	Domain → Pool mapping (generated from DB, HAProxy reads directly)
wildcards.map	Wildcard domain mapping (generated from DB)
haproxy.cfg	Pool backend definitions (static)
servers.json	Legacy — used only for initial migration to SQLite
certificates.json	Legacy — used only for initial migration to SQLite

SSL/TLS Certificates

Current Setup

Item	Value
ACME Client	acme.sh (Google Trust Services CA)
Cert Directory	/opt/haproxy/certs/ (auto-loaded by HAProxy)
acme.sh Home	~/.acme.sh/

How It Works

1. HAProxy binds with crt /etc/haproxy/certs/ (directory, not file)
2. All .pem files in directory are auto-loaded
3. SNI selects correct certificate based on domain
4. acme.sh deploy hook auto-creates combined PEM on issue/renewal

Adding New Certificate

# Issue certificate (Google Trust Services)
~/.acme.sh/acme.sh --issue \
  --dns dns_cf \
  -d "newdomain.com" -d "*.newdomain.com" \
  --reloadcmd "cat ~/.acme.sh/newdomain.com_ecc/fullchain.cer ~/.acme.sh/newdomain.com_ecc/newdomain.com.key > /opt/haproxy/certs/newdomain.com.pem && podman exec haproxy kill -USR2 1"

Certificate Commands

# List certificates
~/.acme.sh/acme.sh --list

# Renew all
~/.acme.sh/acme.sh --cron

# Force renew specific domain
~/.acme.sh/acme.sh --renew -d "domain.com" --force

# Check loaded certs in HAProxy
ls -la /opt/haproxy/certs/

Cloudflare Credentials

• File: ~/.secrets/cloudflare.ini
• Format: dns_cloudflare_api_token = TOKEN
• Also export: export CF_Token="your_token"

HTTP/3 (QUIC)

Frontend supports all protocols:

TCP 443: HTTP/1.1, HTTP/2
UDP 443: HTTP/3 (QUIC)

Clients receive alt-svc: h3=":443"; ma=86400 header for HTTP/3 upgrade.

MCP Security

HAProxy proxies MCP with Bearer Token authentication (configured in frontend ACL).

Access Rules

Source	Token Required
Tailscale (100.64.0.0/10)	No
External (CF Worker, etc.)	Yes

Token Location

• File: /opt/haproxy/conf/mcp-token.env

Connection Examples

Tailscale (no auth):

claude mcp add --transport http haproxy http://100.108.39.107:8000/mcp

External (with auth):

curl -X POST https://mcp.inouter.com/mcp \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}},"id":0}'

Health Check

System Health (haproxy_health)

Returns overall system status:

{
  "status": "healthy",
  "components": {
    "mcp": {"status": "ok"},
    "haproxy": {"status": "ok", "version": "3.3.2", "uptime_sec": 3600},
    "config_files": {"status": "ok", "files": {"map_file": "ok", "db_file": "ok"}},
    "container": {"status": "ok", "state": "running"}
  }
}

Domain Health (haproxy_domain_health)

Returns backend server status for a specific domain:

{
  "domain": "api.example.com",
  "backend": "pool_3",
  "status": "healthy",
  "servers": [
    {"name": "pool_3_1", "addr": "10.0.0.1:80", "status": "UP", "check_status": "L4OK"}
  ],
  "healthy_count": 1,
  "total_count": 1
}

Status values: healthy (all UP), degraded (partial UP), down (all DOWN), no_servers

MCP Tools (30 total)

Domain Management

Tool	Description
haproxy_list_domains	List domains (use include_wildcards=true for wildcards)
haproxy_add_domain	Add domain to pool (no reload), supports share_with for pool sharing
haproxy_remove_domain	Remove domain from pool (no reload)
haproxy_cleanup_wildcards	Remove unnecessary wildcard entries for subdomain domains

Server Management

Tool	Description
haproxy_list_servers	List servers for a domain
haproxy_add_server	Add server to slot (1-10 or 0=auto), auto-saved
haproxy_add_servers	Bulk add servers (JSON array), auto-saved
haproxy_remove_server	Remove server from slot, auto-saved
haproxy_set_server_state	Set state: ready/drain/maint
haproxy_set_server_weight	Set weight (0-256)
haproxy_set_domain_state	Set all servers of domain to ready/drain/maint
haproxy_wait_drain	Wait for connections to drain (timeout configurable)

Health Check

Tool	Description
haproxy_health	System health (MCP, HAProxy, config files)
haproxy_domain_health	Domain-specific health with server status
haproxy_get_server_health	Get UP/DOWN/MAINT status for all servers

Monitoring

Tool	Description
haproxy_stats	HAProxy statistics
haproxy_backends	List backends
haproxy_list_frontends	List frontends with status
haproxy_get_connections	Active connections per server

Configuration

Tool	Description
haproxy_reload	Validate, reload config, auto-restore servers
haproxy_check_config	Validate config syntax
haproxy_save_state	Save server state to disk (legacy)
haproxy_restore_state	Restore state from disk (legacy)

Certificate Management (Zero-Downtime via Runtime API)

Tool	Description
haproxy_list_certs	List all certificates with expiry info
haproxy_cert_info	Get detailed certificate info (expiry, issuer, SANs)
haproxy_issue_cert	Issue new certificate via acme.sh + Cloudflare DNS
haproxy_renew_cert	Renew specific certificate (force option available)
haproxy_renew_all_certs	Renew all certificates due for renewal
haproxy_delete_cert	Delete certificate from acme.sh and HAProxy
haproxy_load_cert	Load/reload certificate into HAProxy (manual trigger)

Key Conventions

Pool-Based Routing

• Domains map to pools: example.com → pool_N
• 100 pools available: pool_1 to pool_100
• Each pool has 10 server slots

Server Naming

Pool backends use simple naming: pool_N_1 to pool_N_10

Examples:

example.com → pool_5
  └─ pool_5_1 (slot 1)
  └─ pool_5_2 (slot 2)
  └─ ... up to pool_5_10

api.example.com → pool_6
  └─ pool_6_1 (slot 1)

Persistence (SQLite)

• SQLite DB (haproxy_mcp.db): All domains, servers, certificates stored in single file
• Remote sync: DB uploaded to HAProxy host via SCP after every write
• Startup restore: DB downloaded from remote on pod start, servers restored via Runtime API
• Migration: First startup auto-migrates from legacy JSON/map files (one-time, idempotent)
• WAL mode: Write-Ahead Logging for concurrent access, checkpointed before SCP upload

Safety Features

• Atomic file writes: Temp file + rename prevents corruption (map files)
• SQLite transactions: All DB writes are atomic
• DB-first pattern: Config saved to SQLite before HAProxy update, rollback on failure
• Command validation: HAProxy responses checked for errors
• Input validation: Domain format, IP (v4/v6), port range, slot limits
• Bulk limits: Max 10 servers per bulk add, 10KB JSON size limit

Performance Optimization

• Command batching: Multiple HAProxy commands sent in single TCP connection
  • Server config (addr + state): 1 connection instead of 2
  • Startup restore: All servers restored in 1 connection (was 2×N for N servers)
  • Example: 7 servers restored = 1 connection (was 14 connections)

Zero-Downtime Certificate Management

• Runtime API: Certificates loaded/updated without HAProxy reload
  • new ssl cert → set ssl cert → commit ssl cert
  • No connection drops during certificate changes
• Persistence: Certificate domains stored in SQLite certificates table
• Auto-restore: Certificates reloaded into HAProxy on MCP startup

HAProxy Runtime API

# Show domain mappings
echo "show map /usr/local/etc/haproxy/domains.map" | nc localhost 9999

# Add domain mapping (runtime only)
echo "add map /usr/local/etc/haproxy/domains.map example.com pool_5" | nc localhost 9999

# Show server state
echo "show servers state" | nc localhost 9999

# Show backends
echo "show backend" | nc localhost 9999

# Set server address
echo "set server pool_1/pool_1_1 addr 10.0.0.1 port 80" | nc localhost 9999

# Enable server
echo "set server pool_1/pool_1_1 state ready" | nc localhost 9999

Directory Structure

/opt/haproxy/
├── haproxy_mcp/         # MCP server package (streamable-http)
│   ├── server.py        # Main entry point
│   ├── config.py        # Configuration and constants
│   ├── db.py            # SQLite database (single source of truth)
│   ├── ssh_ops.py       # SSH/SCP remote execution
│   ├── exceptions.py    # Exception classes
│   ├── validation.py    # Input validation
│   ├── haproxy_client.py # HAProxy Runtime API client
│   ├── file_ops.py      # File I/O (delegates to db.py)
│   ├── utils.py         # Parsing utilities
│   └── tools/           # MCP tools (29 total)
│       ├── domains.py   # Domain management (3 tools)
│       ├── servers.py   # Server management (7 tools)
│       ├── health.py    # Health checks (3 tools)
│       ├── monitoring.py # Monitoring (4 tools)
│       ├── configuration.py # Config management (4 tools)
│       └── certificates.py # Certificate management (7 tools)
├── conf/
│   ├── haproxy.cfg      # Main HAProxy config (100 pool backends)
│   ├── haproxy_mcp.db   # SQLite DB (single source of truth)
│   ├── domains.map      # Domain → Pool mapping (generated from DB)
│   ├── wildcards.map    # Wildcard domain mapping (generated from DB)
│   ├── servers.json     # Legacy (migration source only)
│   ├── certificates.json # Legacy (migration source only)
│   └── mcp-token.env    # Bearer token for MCP auth
├── certs/               # SSL/TLS certificates (HAProxy PEM format)
├── data/                # Legacy state files
├── scripts/             # Utility scripts
└── run/                 # Runtime socket files

~/.acme.sh/
├── *_ecc/               # Certificate directories (one per domain)
└── acme.sh              # ACME client script

/etc/containers/systemd/
└── haproxy.container    # Podman Quadlet config

Configuration Files

File	Purpose
/opt/haproxy/conf/haproxy.cfg	HAProxy main config
/opt/haproxy/conf/haproxy_mcp.db	SQLite DB (single source of truth)
/opt/haproxy/conf/domains.map	Domain routing map (generated from DB)
/opt/haproxy/conf/wildcards.map	Wildcard domain map (generated from DB)
/opt/haproxy/certs/*.pem	SSL certificates (auto-loaded)
/etc/containers/systemd/haproxy.container	Podman Quadlet
/etc/systemd/system/haproxy-mcp.service	MCP service
/opt/haproxy/conf/mcp-token.env	MCP auth token
~/.acme.sh/	acme.sh certificates and config
~/.secrets/cloudflare.ini	Cloudflare API token

Startup Sequence

1. systemd starts haproxy.service
   ↓
2. HAProxy loads haproxy.cfg (100 pool backends)
   ↓
3. HAProxy loads domains.map (domain → pool mapping)
   ↓
4. systemd starts haproxy-mcp.service (or K8s pod starts)
   ↓
5. init_db(): Download SQLite DB from remote host via SCP
   (first time: migrate from legacy JSON/map files → create DB → upload)
   ↓
6. MCP server restores servers from DB via Runtime API
   ↓
7. MCP server loads certificates from DB via Runtime API (zero-downtime)
   ↓
8. Ready to serve traffic

Write Flow (runtime)

MCP tool call (add domain/server/cert)
   ↓
Write to local SQLite DB
   ↓
Sync map files to remote (domains only)
   ↓
Update HAProxy via Runtime API
   ↓
Upload DB to remote host via SCP (persistence)