73 Commits

Author SHA1 Message Date
7bb4f9e6c1 chore: anvil.it.com → inouter.com
Some checks failed
CI/CD / build-and-deploy (push) Failing after 7s
2026-03-27 16:18:35 +00:00
e3024432f9 chore: anvil.it.com → inouter.com
Some checks failed
CI/CD / build-and-deploy (push) Has been cancelled
2026-03-27 16:18:32 +00:00
57ff3dc4fa chore: anvil.it.com → inouter.com
Some checks failed
CI/CD / build-and-deploy (push) Has been cancelled
2026-03-27 16:18:26 +00:00
51fd477179 chore: anvil.it.com → inouter.com
Some checks failed
CI/CD / build-and-deploy (push) Has been cancelled
2026-03-27 16:18:21 +00:00
9585f480be chore: anvil.it.com → inouter.com
Some checks failed
CI/CD / build-and-deploy (push) Has been cancelled
2026-03-27 16:18:19 +00:00
e688f21405 chore: anvil.it.com → inouter.com
Some checks failed
CI/CD / build-and-deploy (push) Has been cancelled
2026-03-27 16:18:15 +00:00
kaffa
cf44ef0c2e chore: remove hardcoded namespace for Argo CD 2026-03-12 13:43:36 +09:00
kaffa
98d308768d chore: remove hardcoded namespace for Argo CD 2026-03-12 13:43:32 +09:00
kaffa
5c0af11735 Fix list_certs to use PEM files + openssl instead of acme.sh
acme.sh --list failed in uv/systemd context. Now scans certs directory
directly and uses openssl for cert details + HAProxy show ssl cert API
for loaded status.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 14:40:11 +09:00
kaffa
97af0b49e2 Update haproxy config and clean up domain mappings
- Add X-Forwarded-For support for real client IP detection (BunnyCDN)
- Change MCP domain from mcp.inouter.com to haproxy.inouter.com
- Remove unused wildcard domains (bench, mcp, ssh)
- Add nocodb.inouter.com wildcard mapping
- Ignore runtime files (*.db, cdn-ips.lst) in .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 11:07:32 +09:00
kappa
dd690aedcb Update CLAUDE.md: 30 tools, CUSTOM_TLDS env, cleanup_wildcards
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:22:39 +09:00
kappa
2e22a5d5a8 Add haproxy_cleanup_wildcards tool to remove subdomain wildcard entries
New db_remove_wildcard() for surgical wildcard-only deletion.
New haproxy_cleanup_wildcards tool scans all wildcard entries and removes
those belonging to subdomains (e.g., *.nocodb.inouter.com) while keeping
base domain wildcards (e.g., *.anvil.it.com).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:36:59 +09:00
kappa
170c48e257 Detect subdomains structurally to skip wildcard entries without certs
Add CUSTOM_TLDS config (HAPROXY_CUSTOM_TLDS env, default: "it.com")
and _get_base_domain() for eTLD+1 detection. _check_subdomain now uses
three layers: registered domains, certificate domains, and structural
analysis. This ensures nocodb.inouter.com never gets a *.nocodb wildcard
entry even when inouter.com has no cert or registration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:34:57 +09:00
kappa
81737bb256 Skip wildcard entry for subdomains of certificate domains
_check_subdomain now also checks certificate domains from DB, not just
registered domains. This prevents adding useless wildcard map entries
like *.nocodb.inouter.com when inouter.com already has a wildcard cert
that only covers one level deep (*.inouter.com).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:31:58 +09:00
kappa
c490ee8673 Change cert renewal period from 30 to 60 days after issuance
acme.sh default --days is 30, which triggers renewal 60 days before
expiry. Setting --days 60 aligns with the common practice of renewing
30 days before the 90-day certificate expires.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:19:11 +09:00
kappa
e0dd3807c6 Hide empty MAINT slots from get_server_health output by default
Reduces MCP response from ~12.3k tokens (~1000 lines) to only active servers.
Empty pool slots (MAINT with no health check) are filtered by default.
Added show_all parameter to optionally show all slots.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:58:16 +09:00
kappa
0084b99f05 Hide empty server slots (0.0.0.0) from list_servers output
Only show servers with actual IP addresses. Empty slots are
HAProxy pre-allocated placeholders that add noise to the output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:13:25 +09:00
kappa
1be615be99 Update CLAUDE.md for SQLite architecture
Reflects the JSON → SQLite migration: architecture diagram now shows
K8s pod + SSH/SCP remote sync, data flow documentation, updated
file descriptions, startup sequence, and write flow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 12:03:50 +09:00
kappa
12fd3b5e8f Store SQLite DB on remote host via SCP for persistence
Instead of syncing JSON files back, the SQLite DB itself is now
the persistent store on the remote HAProxy host:
- Startup: download remote DB via SCP (skip migration if exists)
- After writes: upload local DB via SCP (WAL checkpoint first)
- JSON sync removed (sync_servers_json, sync_certs_json deleted)

New functions:
- ssh_ops: remote_download_file(), remote_upload_file() via SCP
- db: sync_db_to_remote(), _try_download_remote_db()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:46:36 +09:00
kappa
b86ba5d994 Sync servers.json and certificates.json on every change
SQLite DB in the K8s pod is ephemeral (no PV). On pod restart,
migrate_from_json() re-reads the remote JSON files. Without syncing
back, any server/cert changes after migration would be lost.

Now every server/cert write to SQLite also writes the corresponding
JSON file to the remote host, keeping them in sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:32:29 +09:00
kappa
0803d7db92 Fix health check: DB file is always local, not remote
In REMOTE_MODE the SQLite database lives inside the MCP container,
not on the remote HAProxy host. Check db_file with os.path.exists
instead of remote_file_exists.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:21:08 +09:00
kappa
e228fc02fb Fix K8s deployment crash: ensure DB directory exists on startup
init_db() now creates the parent directory for the SQLite database
file before connecting. Also sets HAPROXY_DB_FILE in the K8s
deployment to /app/data/ since the container doesn't have
/opt/haproxy/conf/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:18:37 +09:00
kappa
cf554f3f89 refactor: migrate data storage from JSON/map files to SQLite
Replace servers.json, certificates.json, and map file parsing with
SQLite (WAL mode) as single source of truth. HAProxy map files are
now generated from SQLite via sync_map_files().

Key changes:
- Add db.py with schema, connection management, and JSON migration
- Add DB_FILE config constant
- Delegate file_ops.py functions to db.py
- Refactor domains.py to use file_ops instead of direct list manipulation
- Fix subprocess.TimeoutExpired not caught (doesn't inherit TimeoutError)
- Add DB health check in health.py
- Init DB on startup in server.py and __main__.py
- Update all 359 tests to use SQLite-backed functions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 11:07:29 +09:00
kappa
05bff61b85 fix: Use Runtime API for zero-downtime cert reload in issue_cert reloadcmd
Previously issue_cert saved a reloadcmd that only combined PEM files,
so cron renewals would not reload certs into HAProxy. Now the reloadcmd
uses the Runtime API (set/commit ssl cert) for zero-downtime reload,
matching the existing domain reloadcmd configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 01:11:56 +09:00
kappa
46d57d651a fix: Skip shared domains during server restore to avoid invalid slot warning
Shared domains (with _shares key) have no server slots to restore,
so they should be skipped early in the restore loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 01:00:41 +09:00
kappa
0f244b5c43 chore: Set revisionHistoryLimit to 2 to reduce old replicasets
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:26:39 +09:00
kappa
ae691e557c fix: SSH compatibility with fish shell on remote host
- Remove bash -c from remote_exec (pass command as single SSH arg)
- Fix remote_write_file to pass bash -c script as single quoted string
- Add LogLevel=ERROR to suppress SSH warning messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:17:45 +09:00
kappa
98e55ab1a5 fix: Force bash for SSH commands and suppress known_hosts warnings
- Add UserKnownHostsFile=/dev/null to prevent write errors on read-only .ssh
- Wrap all SSH commands with 'bash -c' for fish shell compatibility on remote

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:12:18 +09:00
kappa
98d2054d6c ci: Re-trigger build after SSH key secret creation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:08:32 +09:00
kappa
ec569ca4be fix: Update HAPROXY_HOST and SSH_HOST to linode-osaka Tailscale IP
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 23:00:30 +09:00
kappa
e40d69a1b1 feat: Add SSH remote execution for HAProxy on remote host
MCP server can now manage HAProxy running on a remote host via SSH.
When SSH_HOST env var is set, all file I/O and subprocess commands
(podman, acme.sh, openssl) are routed through SSH instead of local exec.

- Add ssh_ops.py module with remote_exec, run_command, file I/O helpers
- Modify file_ops.py to support remote reads/writes via SSH
- Update all tools (domains, certificates, health, configuration) for SSH
- Fix domains.py: replace direct fcntl usage with file_lock context manager
- Add openssh-client to Docker image for SSH connectivity
- Update k8s deployment with SSH env vars and SSH key secret mount

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:56:54 +09:00
kappa
ca3975c94c chore: Add dockerignore to gitignore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:34:00 +09:00
kappa
178531ed8a ci: Simplify workflow for act_runner Docker environment
Remove buildx action and test job, use direct docker build/push
and mounted kubectl for k8s deployment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:32:16 +09:00
kappa
fd056368a1 fix: Use tcpSocket probe instead of httpGet for MCP endpoint
MCP streamable-http transport returns 406 on GET requests.
Switch to tcpSocket probe for readiness/liveness checks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:26:30 +09:00
kappa
fb3a674eb8 feat: Add CI/CD pipeline with Docker build and K8s deployment
Add Dockerfile (multi-stage, python:3.11-slim + uv), K8s manifests
(Deployment + Service), and extend CI workflow with build-push-deploy
stages targeting Gitea registry and K3s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:19:32 +09:00
kaffa
07fdce0839 docs: Update README with comprehensive project documentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 15:26:11 +09:00
kaffa
da533f407a feat: Add CrowdSec logging, rate limiting, and fix MCP parameter defaults
- Add real client IP detection (CF-Connecting-IP / src fallback) to both frontends
- Add per-IP rate limiting (429) using real IP for Cloudflare compatibility
- Add CrowdSec syslog forwarding with custom log format
- Add httplog option for detailed HTTP logging
- Fix Python-level defaults on MCP tool parameters to match Field(default=X)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 00:22:39 +09:00
kaffa
4a411202d3 feat: Add pool sharing for domains
Allow multiple domains to share the same backend pool using share_with parameter.
This saves pool slots when domains point to the same servers.

- Add share_with parameter to haproxy_add_domain
- Add helper functions for shared domain management
- Protect shared pools from being cleared on domain removal
- Update documentation with pool sharing examples

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 00:34:22 +09:00
kaffa
95ab0583b3 docs: Simplify verbose docstrings
- save_map_file(): 55 lines → 12 lines
- is_legacy_backend(): 42 lines → 11 lines
- _check_response_for_errors(): 40 lines → 9 lines

Keep essential info, remove redundant examples and explanations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:25:45 +09:00
kaffa
06ab47aca8 refactor: Extract large functions, improve exception handling, remove duplicates
## Large function extraction
- servers.py: Extract 8 _impl functions from register_server_tools (449 lines)
- certificates.py: Extract 7 _impl functions from register_certificate_tools (386 lines)
- MCP tool wrappers now delegate to module-level implementation functions

## Exception handling improvements
- Replace 11 broad `except Exception` with specific types
- health.py: (OSError, subprocess.SubprocessError)
- configuration.py: (HaproxyError, IOError, OSError, ValueError)
- servers.py: (IOError, OSError, ValueError)
- certificates.py: FileNotFoundError, (subprocess.SubprocessError, OSError)

## Duplicate code extraction
- Add parse_servers_state() to utils.py (replaces 4 duplicate parsers)
- Add disable_server_slot() to utils.py (replaces duplicate patterns)
- Update health.py, servers.py, domains.py to use new helpers

## Other improvements
- Add TypedDict types in file_ops.py and health.py
- Set file permissions (0o600) for sensitive files
- Update tests to use specific exception types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:23:51 +09:00
kaffa
e66c5ddc7f chore: Add test artifacts to .gitignore
- .coverage
- htmlcov/
- .pytest_cache/

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:50:39 +09:00
kaffa
6bcfee519c refactor: Improve code quality, error handling, and test coverage
- Add file_lock context manager to eliminate duplicate locking patterns
- Add ValidationError, ConfigurationError, CertificateError exceptions
- Improve rollback logic in haproxy_add_servers (track successful ops only)
- Decompose haproxy_add_domain into smaller helper functions
- Consolidate certificate constants (CERTS_DIR, ACME_HOME) to config.py
- Enhance docstrings for internal functions and magic numbers
- Add pytest framework with 48 new tests (269 -> 317 total)
- Increase test coverage from 76% to 86%
  - servers.py: 58% -> 82%
  - certificates.py: 67% -> 86%
  - configuration.py: 69% -> 94%

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:50:00 +09:00
kaffa
18ce812920 perf: Implement 2-stage map routing for faster domain lookup
Split domain routing into two stages for improved performance:
- Stage 1: map_str for exact domains (O(log n) using ebtree)
- Stage 2: map_dom for wildcards only (O(n) but small set)

Wildcards now stored in separate wildcards.map file.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 11:46:04 +09:00
kaffa
46c86b62f2 fix: HAProxy batch commands and improve routing/subdomain handling
- Fix haproxy_cmd_batch to send each command on separate connection
  (HAProxy Runtime API only processes first command on single connection)
- HTTP frontend now routes to backends instead of redirecting to HTTPS
- Add subdomain detection to avoid duplicate wildcard entries
- Add reload verification with retry logic
- Optimize SSL: TLS 1.3 ciphersuites, extended session lifetime
- Add CPU steal monitoring script

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 11:46:04 +09:00
kaffa
95aecccb03 Add Gitea Actions CI workflow 2026-02-03 11:38:36 +09:00
kaffa
bf4b8ae635 Add README 2026-02-03 11:37:56 +09:00
kaffa
bda3ead099 Add MIT LICENSE 2026-02-03 11:20:34 +09:00
kaffa
02a17a62b4 docs: Add WAF integration docs and ignore lock files
- Add YARA guide, WAF integration docs, and webshell rules
- Ignore *.lock files in .gitignore

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 04:34:49 +00:00
kaffa
f835b04695 chore: Remove unused scripts directory
- Remove legacy restore-state.sh (MCP server handles state via servers.json)
- api/ was separate git repo, already removed from filesystem

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 04:31:55 +00:00
kaffa
6ced2b42d4 refactor: Move certificate config functions to file_ops.py
- Move load_certs_config, save_certs_config, add_cert_to_config,
  remove_cert_from_config from certificates.py to file_ops.py
- Add CERTS_FILE constant to config.py
- Add file locking for certificate config operations (was missing)
- Consistent pattern with servers.json handling

certificates.py: 543 → 503 lines
file_ops.py: 263 → 337 lines

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 04:26:55 +00:00