diff --git a/infra/infra-hosts.md b/infra/infra-hosts.md index a44e7ab..cbd6357 100644 --- a/infra/infra-hosts.md +++ b/infra/infra-hosts.md @@ -34,7 +34,16 @@ tags: [infra, network, kr-zone, openwrt] 주요 네임스페이스: anvil, apisix, argocd, cert-manager, db, gitea, ironclad, kroki, longhorn-system, openmemory, registry, system-upgrade, tools, vault -트래픽 흐름: 외부 → OpenWrt HAProxy (80/443) → APISIX hostNetwork (9080/9443, 3노드) → K3s 서비스 → pods +Traefik: DaemonSet (3노드), LoadBalancer 192.168.9.134/135/214:80,443 + +APISIX: Deployment (replicas 3, hostNetwork, strategy Recreate), 3노드 :9080/:9443 바인딩 + +트래픽 흐름: 외부 → OpenWrt HAProxy (TCP 80/443) + nftables (UDP 443) → APISIX hostNetwork (9080/9443, 3노드) → K3s 서비스 → pods + +### TCP 튜닝 (3노드 공통) + +- BBR 혼잡 알고리즘 + fq qdisc (`/etc/sysctl.d/99-bbr.conf`, `/etc/modules-load.d/bbr.conf`) +- tcp_fin_timeout=15, tcp_fastopen=3, netdev_max_backlog=8192, tcp_max_syn_backlog=8192, tcp_keepalive_time=600, ip_local_port_range=1024-65535 (`/etc/sysctl.d/99-tcp-tuning.conf`) ## 서비스 위치 @@ -107,23 +116,32 @@ tags: [infra, network, kr-zone, openwrt] 인터넷 └── OpenWrt 라우터 (공인 IP: 220.120.65.245, 내부: 192.168.9.1) ├── incus-kr1 (192.168.9.214) ← K3s control-plane - ├── incus-kr2 (192.168.9.135, eno1 DHCP 192.168.9.140) ← K3s control-plane + ├── incus-kr2 (192.168.9.135, br-uplink 고정) ← K3s control-plane └── incus-hp2 (192.168.9.134) ← K3s control-plane -외부 트래픽 흐름: +외부 트래픽 흐름 (TCP): 인터넷 → OpenWrt HAProxy(:80/:443) → APISIX hostNetwork(:9080/:9443, 3노드 roundrobin) → K3s svc → pods +외부 트래픽 흐름 (QUIC/HTTP3): + 인터넷 → OpenWrt nftables DNAT(UDP :443) → APISIX hostNetwork(:9443, numgen mod 3 roundrobin) → K3s svc → pods + 내부 트래픽 흐름: LAN → K3s/Traefik(192.168.9.x:80/443) → pods 직접 ``` - **게이트웨이**: OpenWrt 라우터 (root@192.168.9.1, SSH user: root) -- **OpenWrt HAProxy** (hp2 단독이 아닌 3노드 로드밸런싱): +- **OpenWrt HAProxy** (TCP, mode tcp, 3노드 로드밸런싱): - frontend ft_http :80 → backend bk_apisix_http (roundrobin) - frontend ft_https :443 → backend bk_apisix_https (roundrobin) - backend 서버: hp2(192.168.9.134:9080/9443), kr1(192.168.9.214:9080/9443), kr2(192.168.9.135:9080/9443) - stats: :9999 (admin/admin) -- **APISIX etcd**: apisix-etcd (incus 컨테이너, 10.179.99.101) +- **OpenWrt QUIC/HTTP3** (UDP, nftables DNAT): + - UDP 443 → numgen inc mod 3 → hp2/kr1/kr2 :9443 roundrobin + - CDN IP 필터도 UDP 443에 적용됨 +- **K3s datastore**: Incus etcd 3노드 클러스터 (192.168.9.214, 192.168.9.135, 192.168.9.134) + - `/registry/` — K3s 클러스터 백엔드 스토어 + - `/patroni/nocodb-cluster` — NocoDB PostgreSQL HA +- **APISIX etcd** (K3s 내부): apisix-etcd StatefulSet (ClusterIP 10.43.20.100:2379, prefix `/apisix`) - **OVN 네트워크**: ovn1 (10.165.246.0/24) — hp2↔kr2 간 오버레이 - **CDN IP 필터**: BunnyCDN + Cloudflare IP만 80/443 허용, 그 외 WAN 차단 - 스크립트: `/etc/cdn-filter-update.sh`