11 Commits

Author SHA1 Message Date
kappa
12fd3b5e8f Store SQLite DB on remote host via SCP for persistence
Instead of syncing JSON files back, the SQLite DB itself is now
the persistent store on the remote HAProxy host:
- Startup: download remote DB via SCP (skip migration if exists)
- After writes: upload local DB via SCP (WAL checkpoint first)
- JSON sync removed (sync_servers_json, sync_certs_json deleted)

New functions:
- ssh_ops: remote_download_file(), remote_upload_file() via SCP
- db: sync_db_to_remote(), _try_download_remote_db()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:46:36 +09:00
kappa
b86ba5d994 Sync servers.json and certificates.json on every change
SQLite DB in the K8s pod is ephemeral (no PV). On pod restart,
migrate_from_json() re-reads the remote JSON files. Without syncing
back, any server/cert changes after migration would be lost.

Now every server/cert write to SQLite also writes the corresponding
JSON file to the remote host, keeping them in sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:32:29 +09:00
kappa
cf554f3f89 refactor: migrate data storage from JSON/map files to SQLite
Replace servers.json, certificates.json, and map file parsing with
SQLite (WAL mode) as single source of truth. HAProxy map files are
now generated from SQLite via sync_map_files().

Key changes:
- Add db.py with schema, connection management, and JSON migration
- Add DB_FILE config constant
- Delegate file_ops.py functions to db.py
- Refactor domains.py to use file_ops instead of direct list manipulation
- Fix subprocess.TimeoutExpired not caught (doesn't inherit TimeoutError)
- Add DB health check in health.py
- Init DB on startup in server.py and __main__.py
- Update all 359 tests to use SQLite-backed functions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:07:29 +09:00
kappa
e40d69a1b1 feat: Add SSH remote execution for HAProxy on remote host
MCP server can now manage HAProxy running on a remote host via SSH.
When SSH_HOST env var is set, all file I/O and subprocess commands
(podman, acme.sh, openssl) are routed through SSH instead of local exec.

- Add ssh_ops.py module with remote_exec, run_command, file I/O helpers
- Modify file_ops.py to support remote reads/writes via SSH
- Update all tools (domains, certificates, health, configuration) for SSH
- Fix domains.py: replace direct fcntl usage with file_lock context manager
- Add openssh-client to Docker image for SSH connectivity
- Update k8s deployment with SSH env vars and SSH key secret mount

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:56:54 +09:00
kaffa
4a411202d3 feat: Add pool sharing for domains
Allow multiple domains to share the same backend pool using share_with parameter.
This saves pool slots when domains point to the same servers.

- Add share_with parameter to haproxy_add_domain
- Add helper functions for shared domain management
- Protect shared pools from being cleared on domain removal
- Update documentation with pool sharing examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 00:34:22 +09:00
kaffa
95ab0583b3 docs: Simplify verbose docstrings
- save_map_file(): 55 lines → 12 lines
- is_legacy_backend(): 42 lines → 11 lines
- _check_response_for_errors(): 40 lines → 9 lines

Keep essential info, remove redundant examples and explanations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:25:45 +09:00
kaffa
06ab47aca8 refactor: Extract large functions, improve exception handling, remove duplicates
## Large function extraction
- servers.py: Extract 8 _impl functions from register_server_tools (449 lines)
- certificates.py: Extract 7 _impl functions from register_certificate_tools (386 lines)
- MCP tool wrappers now delegate to module-level implementation functions

## Exception handling improvements
- Replace 11 broad `except Exception` with specific types
- health.py: (OSError, subprocess.SubprocessError)
- configuration.py: (HaproxyError, IOError, OSError, ValueError)
- servers.py: (IOError, OSError, ValueError)
- certificates.py: FileNotFoundError, (subprocess.SubprocessError, OSError)

## Duplicate code extraction
- Add parse_servers_state() to utils.py (replaces 4 duplicate parsers)
- Add disable_server_slot() to utils.py (replaces duplicate patterns)
- Update health.py, servers.py, domains.py to use new helpers

## Other improvements
- Add TypedDict types in file_ops.py and health.py
- Set file permissions (0o600) for sensitive files
- Update tests to use specific exception types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:23:51 +09:00
kaffa
6bcfee519c refactor: Improve code quality, error handling, and test coverage
- Add file_lock context manager to eliminate duplicate locking patterns
- Add ValidationError, ConfigurationError, CertificateError exceptions
- Improve rollback logic in haproxy_add_servers (track successful ops only)
- Decompose haproxy_add_domain into smaller helper functions
- Consolidate certificate constants (CERTS_DIR, ACME_HOME) to config.py
- Enhance docstrings for internal functions and magic numbers
- Add pytest framework with 48 new tests (269 -> 317 total)
- Increase test coverage from 76% to 86%
  - servers.py: 58% -> 82%
  - certificates.py: 67% -> 86%
  - configuration.py: 69% -> 94%

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:50:00 +09:00
kaffa
18ce812920 perf: Implement 2-stage map routing for faster domain lookup
Split domain routing into two stages for improved performance:
- Stage 1: map_str for exact domains (O(log n) using ebtree)
- Stage 2: map_dom for wildcards only (O(n) but small set)

Wildcards now stored in separate wildcards.map file.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 11:46:04 +09:00
kaffa
6ced2b42d4 refactor: Move certificate config functions to file_ops.py
- Move load_certs_config, save_certs_config, add_cert_to_config,
  remove_cert_from_config from certificates.py to file_ops.py
- Add CERTS_FILE constant to config.py
- Add file locking for certificate config operations (was missing)
- Consistent pattern with servers.json handling

certificates.py: 543 → 503 lines
file_ops.py: 263 → 337 lines

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 04:26:55 +00:00
kaffa
7bee373684 refactor: Modularize MCP server with command batching
- Split monolithic mcp/server.py (1874 lines) into haproxy_mcp/ package:
  - config.py: Configuration constants and environment variables
  - exceptions.py: Custom exception classes
  - validation.py: Input validation functions
  - haproxy_client.py: HAProxy Runtime API client with batch support
  - file_ops.py: Atomic file operations with locking
  - utils.py: CSV parsing utilities
  - tools/: MCP tools organized by function
    - domains.py: Domain management (3 tools)
    - servers.py: Server management (7 tools)
    - health.py: Health checks (3 tools)
    - monitoring.py: Monitoring (4 tools)
    - configuration.py: Config management (4 tools)

- Add haproxy_cmd_batch() for sending multiple commands in single TCP connection
- Optimize server operations: 1 connection instead of 2 per server
- Optimize startup restore: All servers in 1 connection (was 2×N)
- Update type hints to Python 3.9+ style (built-in generics)
- Remove unused imports and functions
- Update CLAUDE.md with new structure and performance notes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 03:50:42 +00:00