From 1dcf2f448e7f0f4e71dd6decdccc4d9cb632d489 Mon Sep 17 00:00:00 2001 From: kappa Date: Sun, 15 Mar 2026 13:13:54 +0900 Subject: [PATCH] Initial commit: CrowdSec BunnyCDN bouncer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - sync/bouncer.py: CrowdSec LAPI → bloom filter → BunnyCDN Edge Script - edge/middleware.ts: BunnyCDN edge middleware with bloom filter + Turnstile CAPTCHA - README.md: architecture and deployment docs --- README.md | 167 ++++++++++------- edge/middleware.ts | 344 +++++++++++++++++++++++++++++++++++ sync/bouncer.py | 411 ++++++++++++++++++++++++++++++++++++++++++ sync/requirements.txt | 1 + 4 files changed, 855 insertions(+), 68 deletions(-) create mode 100644 edge/middleware.ts create mode 100644 sync/bouncer.py create mode 100644 sync/requirements.txt diff --git a/README.md b/README.md index 2001ce9..d826b31 100644 --- a/README.md +++ b/README.md @@ -1,101 +1,132 @@ -# CrowdSec Bunny Bouncer +# CrowdSec BunnyCDN Bouncer -[CrowdSec](https://crowdsec.net) bouncer for [Bunny CDN](https://bunny.net). CrowdSec의 차단 목록을 Bunny CDN Edge Script와 연동하여 악성 IP를 CDN 엣지에서 차단합니다. +CrowdSec LAPI에서 탐지한 악성 IP를 BunnyCDN 에지에서 선제 차단하는 bouncer. -## Architecture +## 아키텍처 ``` -CrowdSec LAPI ──stream──▶ bouncer.py ──libSQL──▶ Bunny Database - │ - Edge Script (index.ts) - onOriginRequest 에서 - DB 조회 → 403 / pass +[CrowdSec LAPI (jp1, 10.253.100.240:8080)] + ↓ 크론잡 (3분 간격) + ↓ /v1/decisions/stream API + ↓ +[bouncer.py] + → IP 목록 → Bloom filter 생성 + → BunnyCDN Compute API로 Edge Script 코드 업데이트 + ↓ +[BunnyCDN Edge Script (middleware.ts)] + → X-Real-Ip 헤더에서 클라이언트 IP 추출 + → Bloom filter로 IP 매칭 + → 매칭되면 Turnstile CAPTCHA 챌린지 + → 캡차 통과하면 4시간 검증 (verified_ips DB) + → Bloom filter에 없으면 오리진으로 통과 ``` -- **bouncer.py** — CrowdSec LAPI Streaming API를 폴링하여 차단 결정을 Bunny Database(libSQL HTTP API)에 동기화 -- **edge-script/index.ts** — Bunny CDN Edge Script 미들웨어. 모든 요청에 대해 DB를 조회하고 차단 IP면 403 응답 -- **setup.py** — Edge Script 코드를 Bunny CDN에 업로드 및 퍼블리시 -- **monitor.py** — Edge Script 사용량, CDN 트래픽/에러, WAF 이벤트 로그 모니터링 +## 구성 요소 -## Features +### sync/bouncer.py -- CrowdSec Streaming API 기반 실시간 동기화 (기본 60초 간격) -- CAPI(Community Blocklist) 포함 27,000+ IP 처리 -- Edge Script 인메모리 캐시 (정상 IP 5분, 차단 IP 1분 TTL) -- Fail-open 설계 — DB 장애 시 요청 허용 -- 6시간 주기 전체 재동기화 -- 헬스체크 파일 기반 컨테이너 모니터링 +CrowdSec LAPI → BunnyCDN Edge Script 동기화 스크립트. -## Prerequisites +- CrowdSec LAPI stream endpoint에서 ban 결정 fetch +- IP 목록을 bloom filter (FNV-1a 해시)로 변환 +- BunnyCDN Compute API로 Edge Script 코드 내 `BLOOM_B64` 상수 교체 +- 스크립트 퍼블리시 -- CrowdSec LAPI 접속 가능 (bouncer API 키 필요) -- Bunny CDN 계정 + Pull Zone -- Bunny Database (libSQL) -- Bunny Edge Scripting 활성화 -- Python 3.12+ -- Docker/Podman (운영 환경) +### edge/middleware.ts -## Quick Start +BunnyCDN Edge Script 미들웨어 (현재 inouter 풀존에 배포됨, script ID: 64811). -### 1. Bunny Database 및 Edge Script 준비 +- `@bunny.net/edgescript-sdk` 사용 +- Bloom filter 기반 IP 차단 +- Cloudflare Turnstile CAPTCHA로 false positive 대응 +- LibSQL DB로 verified IP 4시간 캐싱 +- HMAC 서명 쿠키로 세션 유지 +- Clean IP 캐시 (네거티브 캐시, 최대 50K) -Bunny 대시보드에서: -1. Database 생성 → URL과 Token 기록 -2. Edge Scripting에서 스크립트 생성 → Script ID 기록 -3. Edge Script에 `BUNNY_DATABASE_URL`, `BUNNY_DATABASE_AUTH_TOKEN` 환경변수 설정 -4. Edge Script를 Pull Zone에 연결 +## 환경 변수 -### 2. 환경변수 설정 +### bouncer.py + +| 변수 | 설명 | 기본값 | +|------|------|--------| +| `CROWDSEC_LAPI_URL` | CrowdSec LAPI URL | `http://10.253.100.240:8080` | +| `CROWDSEC_BOUNCER_KEY` | Bouncer API key | (required) | +| `BUNNY_API_KEY` | BunnyCDN account API key | (required) | +| `BUNNY_SCRIPT_ID` | Edge Script ID | `64811` | +| `STATE_FILE` | 상태 파일 경로 | `/var/lib/crowdsec-bouncer/state.json` | + +### Edge Script Variables + +| 변수 | 설명 | +|------|------| +| `TURNSTILE_SITE_KEY` | Cloudflare Turnstile site key | +| `TURNSTILE_SECRET_KEY` | Cloudflare Turnstile secret key | +| `CACHE_MODE` | 캐시 모드 (auto) | +| `BUNNY_DATABASE_URL` | LibSQL database URL | +| `BUNNY_DATABASE_AUTH_TOKEN` | LibSQL auth token | + +## 배포 + +### 1. CrowdSec bouncer 등록 ```bash -cp .env.example .env -# .env 파일 편집 +# jp1 crowdsec 컨테이너에서 +cscli bouncers add bunny-cdn-bouncer +# 출력된 API key를 Vault에 저장 ``` -| 변수 | 필수 | 설명 | -|------|------|------| -| `CROWDSEC_LAPI_URL` | O | CrowdSec LAPI 주소 | -| `CROWDSEC_LAPI_KEY` | O | Bouncer API 키 | -| `BUNNY_DB_URL` | O | Bunny Database libSQL URL | -| `BUNNY_DB_TOKEN` | O | Bunny Database 인증 토큰 | -| `SYNC_INTERVAL` | | 동기화 간격 초 (기본: 60) | -| `INCLUDE_CAPI` | | CAPI 차단 목록 포함 (기본: true) | -| `FULL_RESYNC_INTERVAL` | | 전체 재동기화 간격 초 (기본: 21600) | -| `LOG_LEVEL` | | 로그 레벨 (기본: INFO) | -| `BUNNY_API_KEY` | | Bunny API 키 (setup.py, monitor.py용) | -| `BUNNY_SCRIPT_ID` | | Edge Script ID (setup.py용) | - -### 3. Edge Script 배포 +### 2. Vault에 시크릿 저장 ```bash -python setup.py +vault kv put secret/infra/crowdsec-bunny-bouncer \ + bouncer_key= ``` -### 4. Bouncer 실행 +### 3. Incus 컨테이너에 배포 ```bash -# Docker/Podman -podman compose up -d - -# 또는 직접 실행 -pip install -r requirements.txt -python bouncer.py +# jp1 infra-tool 컨테이너 +pip3 install -r sync/requirements.txt +cp sync/bouncer.py /opt/crowdsec-bouncer/bouncer.py ``` -## Monitoring +### 4. Cron 설정 ```bash -# 기본 (최근 30일) -python monitor.py - -# 기간 지정 -python monitor.py 7 +# /etc/cron.d/crowdsec-bunny-bouncer +*/3 * * * * root CROWDSEC_BOUNCER_KEY="..." BUNNY_API_KEY="..." /usr/bin/python3 /opt/crowdsec-bouncer/bouncer.py >> /var/log/crowdsec-bouncer.log 2>&1 +# 매시 정각: 전체 동기화 (startup mode) +3 * * * * root CROWDSEC_BOUNCER_KEY="..." BUNNY_API_KEY="..." /usr/bin/python3 /opt/crowdsec-bouncer/bouncer.py --startup >> /var/log/crowdsec-bouncer.log 2>&1 ``` -출력 항목: -- **Edge Script 사용량** — 월간 요청 수, CPU 시간, 무료 한도(25M) 대비 비율 -- **CDN 통계** — 총 요청, 캐시 히트율, 대역폭, 3xx/4xx/5xx 에러 -- **WAF / Bunny Shield** — Shield 상태, WAF 룰 현황, 이벤트 로그 요약 (공격 유형, IP, 국가별) +## 테스트 + +```bash +# 테스트 IP 추가 +ssh incus-jp1 "incus exec crowdsec -- cscli decisions add -i 198.51.100.1 -d 10m -R 'test'" + +# 동기화 실행 +python3 sync/bouncer.py --startup -v + +# 테스트 IP 제거 +ssh incus-jp1 "incus exec crowdsec -- cscli decisions delete -i 198.51.100.1" +python3 sync/bouncer.py -v +``` + +## 풀존 적용 현황 + +| 풀존 | ID | 미들웨어 | 상태 | +|------|-----|---------|------| +| inouter | 5316471 | 64811 | 적용됨 | +| actions | 5330178 | - | 미적용 | + +## Bloom Filter 세부 + +- 해시 함수: FNV-1a (32-bit) double hashing +- False positive rate: 0.1% +- 헤더: 8 bytes (m: uint32 LE, k: uint32 LE) +- 페이로드: bit array +- 버전: MD5 해시 앞 16자 (변경 감지용) ## Edge Script Pricing diff --git a/edge/middleware.ts b/edge/middleware.ts new file mode 100644 index 0000000..0309899 --- /dev/null +++ b/edge/middleware.ts @@ -0,0 +1,344 @@ +import { createClient } from "@libsql/client/web"; +import * as BunnySDK from "@bunny.net/edgescript-sdk"; +import process from "node:process"; + +// --------------------------------------------------------------------------- +// Embedded bloom filter (updated by bouncer.py via BunnyCDN API) +// --------------------------------------------------------------------------- + +const BLOOM_B64 = "tBgBAAcAAACOmJnIe7yJjDGrkCYuyIrKiuKIK6Mqvqfgpgq6/4rKnvY83wqOrMQvKAxKCr6tqYPJmHitHs6ieIujmZuyMiqq+skqzKfk3viOFjrnqD6uKrqoDv2Iro8vnC2xibGYiCuJiqmDn7w5mtorg4qWSLpnqxps66Jfqpqqmgzifou2i4KIC7ql7F4b9vq5rJg+q64d6k6xa6gS6KOoAgqi0bHGI7rj7Jq6AkauwX9RvMhFtIi/u/iki3rqov246wsKrJMmmMx2+Yq6mSIPqBCGSZqqnsizcsoOKTqrr2go+oigRZq7uLLojJXKI3VLauVqgDvq6ooju4OIxo4Irj4d5qqokLYY6IQsgt+qiG7ciomarku8KteAm+MogbwIuLi8uusRq8Tkrpq6MtRJ6aKYqpv4uUwuuctSkjAIokHsg5iLKPryC2KLj4Lqv4yrqrpw7giinlcKxZ6a6YwP6yj+r7TiOslIgTmb2LqcruOmgrSpuaqtPKmKk75oCn6P6wuNTzOIzqmM7Aou7qwqqI/ui0xROyoOn/C6rtgMUKfq5CoNiaxc6baq6nJuv/voJSzq74ErosDpmwC/mLoEIKaqrKiYvccqMowjsq7ciF6aT/iICCCoh3oqrOFoLpTqox6OilzaCo6curDsp4rE6o/JjY8eroivCpwogpqpyruHDA3qh+BQaqkhhe/jLqX4/gXpMISJijzYnquYu//KIK7iy4qMiOsaDSL7f8a0TuC+P5M7yPqNOIaDqqubOgH8nrK/6QmqqCDAqKKmQCeLjO854qqgKbK0quHW+Ci5gMCK+oKYyj2KjqoYip+67WnaCDnorZOLiOir+qkoM4ifHuyjugjCD7qvaYqqqM6oqFyhsrz2Tvjgyab39ArOMp6rBmqI66Kijm7yyIouqva7+ZKoqXnUqK8tKi0ug+xADuozs6ug0vvoirGnNjn5v62Ij0rKvu04qp2MZqHCubCKjo64YSOSar+DaujKhrqKikKAieaoPM6UqrjsuYDqraiJfEqwjaooqj7N5q2qoIzonPR/LqjdqOnOuuIOqnqnoO7KCMxgrhyeqhyoi5WKLp6vD+YKnOb7n60DTqii9qP8LrCYkXriaoCS6IgaqCECuy+Kn567naXu/u/IlpSyGjapl6uk7q4q2q3IK7jKqyo2+WVoCAsKz6luuJ6fS8iCCcqs6P6q3t+oqMi7MpJ66tqcs+qc6Iyv66OayrrO4bequstDi5Kq7QiLKLwqna7wvimOaK+q7ejujWvIptnJyppnYOnuYPqJ3sPziYuoj5wKnrOOGKm9rjs6ZKgqqouh4rnoLDiv7mi8pu+Ihtnj/7iPitqqmGmki9v7jMgf64q4g+0pmrmspK3zrAiIOziLsKCsq+j4OTh064QyipC9QxywePCquQ7kmaOIqpIIjU4pmRCbmrjpG6qrmo4Xt46J6kiKwISquDCoawO8toIqssrpmnIhLrrqqkjeqqoZuLvjtsynquqI2Y+JsDgqvq6y7kuouamI6uyE4e3//wql7qGZWiCquYTmDcwL4Krpqr6+rqurCozJzAGqyvojf+hIvqqjKgKBmKgO56mYFgrw/5KbIUwuqpyPpbiimVq6SwiuYm3J6+HuK4FyOu776airqKWqicmeqLrI83TuScibCYokqdEYomvvf4zhSqAo+iyYANwkuqyzCoiKuQkJy/4I45OpmWKrv0zgd+LwIL+mna4I6KCiq+MJ577cqOmIi6g6jcmGjujK5gx07vrcuw27jO8sbLxqmoqh+qfNvuur6q+6zjmap4gagvJcq8isj/+sgt3IuoGOgrjK6AEtqLqm7DuaK8C6+qu4irOKxxarrIvp6ghBs8qqqtKvKQ7zkaiqvsiKqoijoqS5sRWJ+VpIyagerCk4qIiAKezq+DLq7q/aa7ynqwyJ2EeK2rNKp5yJj6Z4yIqwmcqKKwKYntif+pyAuqqfIKpqtGhopqCqTaWN68+aubNAsT8yaalzkjq+6EKLj7WIP8y5G7hL+7KOCD4Y6uqoKq/oGGFKAsmartvoyOt4SzA/0AjrenpgiBtaginI7I6Y+pI7uJGo2KKYoocWg6qK4x6gyEjpqv2a6KTq7Tgo2q/YEIjMuMVKjROpzGkPzMrJiHJp4ECYL6hO7BvnLooAuGjCwqszmkiWLp1G+5ojr+K6mgi7wraAZz2a/I264qrJiYqEYNglKODB6Jw7roosw4GyijlvmoqSjIPo772E6u6gGbqJoBUu8UwK48ox6Sjo/I7qmNqmGYyqaaqrqQ6AiJK6iqCbxo/qGorOhyiqjBIu0BqpkY2JqSGQaEa6+OGaAW2eiwiqqiuqaqSL2uIsm76bPqmknJuK+k6k4KGM6n6HOY9tjWPIDyno6zq4h4bOwNyfw8KPULvnDGsGg4iDKLrI6wf7qciuj76o8O65hrirqJkcCKyIyLbJvAiqLoisqSK76hkhit5U7i3+iqH9kruCj5mXsioZsY7+WSiAqvyNgBsl6jzorMgobCSlhZ6mr2KOwoC+iyv25PGuaMPAfs+Pu7LTqijqirqvKaKs+eLPwuqoaKWFjpOu6ayKrqDqKbSOoqzqaTDuS8sqqkCtOKujn1RaoJu/jtpKq+A6goALh+6qn7i2+r6JxYy2thiaKkn76/i5rZ0562qoevtzuyqwawqv6siWy3IOuSzqCynvopi7Geg66imQ4zqomQgKuM6O26gLqLBbjmqUowwJAB9tipyMupDqdhppIyLMnrqvi6+P63ju+hrrK8qSUnuGmo00S83OxrwkkcFziWesflp0kM6UaIkkqueSMuap3BHJrgwutlzLjvjqi8gc7YKuKij4GYqjPLLoeJjqiG74CKjLCyNgaKW6IAaKIsyCuOMujumClp+Kmd0lbCKjiOMhiWi+suySrfmArYjByCgsaoru6BaOgKjDqo1JoK0wR2IIlirou7ruk9+YqIOOyIvTrqCrCDuwgrjKuQJ5+4oi3pi7rB/Qa4iqqjCYGtoKn6tg6Qj/q86L+JkEHoyKItglarilinDMtAKImKi4t+qptr7umPqEspe4uoqYqjgep8gMTtzKy8/pkqzuoqgqi6qgrtYoirL62jwzudvvLegaHqkuxIiWk/JUPj5uqpDaLo6fpiqo7iLIKrSUTuYBfXTouY9nuq6LOdq9GALsDr7tAlawTNQoru7l4izLDq+4Lxyq26G4avrGsKuMKheD/qObikwue7rkPBaKKC6b7/86moiKaIqoSomAKLtKDKuA7IskiQhKsgjMmKGI8LkIihrqOL+7D6opiAuvoE6aiBm7oo+I7KfoOogoS6yok+rPrunQzJuqq5moyLmaK8qJliitqUrMnpKOIPz/LZKCoqACgoSFvmhqvirqoLDM4p4LuoCKgPn9cuwCyu/s6LoVrq7ZnyKGmyzyqureqL3YKYY3PrHIKbeOrtuviQ2Ji75Qv+ndi05ihIjqjIdIpPXx4g6Kk4WrrhyMqPyZoty05ntLiq6sOKoN3pI6cpuLesS/jSeoKsoPKY3yBYgEoI/r/o+Vg96qBUqJaHkKYssqKpip7CKQyKgaj4KEqaAuGcOcwonvSuxCpUuACdnTrt5brLl2qWa/Ka6ioryviSvtgrbXjoICyjOLP+qgYuCLgJ+buBKToo6Kzh++ooaFnP6WzyyAh4ySt6KKDqKurmi4tho5vVCvuC5sgzM6n+iqztusg8jLqoiCOs8gLb/h7JugOp7I0wTKiIv/q6RAI6mEgL46skS5LKwisHs6I7yCzbnOpeqCp6yjoMTaXkKCu8qNjIqLqCvg2KoPzoiOLyub7IjOqmCKqQ4KoYCiqhu639Dqn2SJPISCLomq6juBe++DgXwPGPiJ4Hy6vveoRx8bqM6PrPoSq7GNyP2AvDaguYysqrnqAMeKbGuHa4XHe+Szro6dSq6IiazKyKLBj1fQGazWuWgL7ras6RSPpIqxiIC5vNvqJOm6CIquuopo8caJxJyq6NHIu+hgpoPLc4iOU8uNDqjtzuig2nztKN7v+b++29gKowr/6xicngLePticsmKmmqymgPjYnISoJug4oqO6Cqls3q2PD8nguMk6nOmTKu7Yis1/iiCHmq6/qqDJu8gqwLruCKq4rS063OhuqPnnLm68+Ma47saoK6rD6x7t/spai97ezK44riD6T5myjansq10PpYUtLIjIn/F48aGsLYqLkLuHjAEuiyCpu0o4IYq9vinhMv+QqsqTrMrppUKda/KKqYjJsoq+ri7o9HaoAC/IkqudnDJgwlpanBzu3ei+gKuuvK53qO92OQpqKmqqaqMoWcmZq4CljmyIicno7/oNCo6BRq4ujXqLoqiKuojuzp6pShMuoIpauKg8yVC5Daiq6ous5yqEab4KIMstmSqkDaoAuqUqLiMBusQ22DZlPyOqbO8pufuqnooptZ/OrbqD2fDqKNSi/bmCqK+GiZrsi2qYqYgc6Qaqo+DKuagpKum6r0y5+sny7p4ibQjImIr+O6esyqkoiriEuiOMOp5OgCiqGIg73IxSihaxAKOya6pbIo4aqI8+ytrq74hLC7oBvLLozOCzvLYe+ZKTOPYY7it0ujiaE2BiqIG7OoaTGvHsTZloLM7guC/mhuypdjpty6mqK7+q6mQswtqizI44g66uCuiqqNtI5euGuMHoJcjAzcipq8kvqv7IqqqsCaKh88M6vy8uzK+oqK75qfmKqgKOir0KqOmwHMatrrroC4qYvpqpaNy8iQ2/bJqH4MiLGsgJ7a6arOaPrL6ogIK7Bu6oxD79vQqCAijeszmCouJO61j6mOfanqCxGjDZyKIc/kquCAgopIKrIvnyDm6IosiA/cuusSt7w4lMq6LiIf26j6qvUImuiK2IrooqrrGqBqeiqK04GrojgiKgvPvrS8OWoqq6i8reICoYL+MC+JoDjqnqqgyKKoultsliegiK54gIq5IPguyg6zAgLqnK2KrrqIuLwg5ObX7u+q6LCukBr7pF6qhoosJsooZDkYGuNU6oqIqiuqjOE6qKyZSOwvCqovCIqo7qiF654OiICpII2Iw4+qDmno4ci5iqrqngzK6WubCgOAWIgMjl6u4Y6oaogLDouW9a8VjA5YXOqgoauaroqCnKtS3Aj5Kbq+6om+gujruVrOvq+OYbp8CZkLEoy6HFq8AjgqjqpMK/JbqiIjarjIjeG2uOmpak+EqYi7qVmogDry7OuyY0aO+eip7qhoqAyi44WYiI9i6+yroLGoqOoP0oivyq7/qyseaeiiHuartIYbC2JUSsCro2I+24zXomaiiQjqSnL9PqCOqmgq/ayK2QKixmMoRLnCumG4gKegKqUrnvquLrjtariKpNfvrpSqsyvJ4J9o35/uoSguBcyiqpLiUzKwyAuMmQD8QOinqhoACm7GiTaHqr65CZrmzqrruo7vIocapSPVOw8uqp+qOLIZxrm6qMqrKLrqp4jtuy+yEwDY4soutyzC6JkIwavsFY+JmkLq++rOy8uhog8bDy0Ki2v3q+fSqajriyiKKrt/7Yg6K0DtjimFrxmLoq4vM1hqifuetkzroXaElYjk8uqcq6lzyqgsk6776LvxqNSKauyDKimooIsu4qOcIC3JyHqKwOsOcw4s6x+JgE8s6KuMrIeEqopqmeq7zoyK3OixaOPqqKuQi6igOeqdrtqomZbyDYaLqr67LzOoorqrr4+rpl2kqisusVqMnhjw6kiuLCASxRs66oOokuLaqFuOaAIEqsq7ICuvANo/r/GyPGMmjKIq6iv4RqMo69y7OkQ839i4r+PGiuuqsAi6vquss0OIuurNY8vpfpx4Sj9CqChs6bi4cqzZKMZ6+6yzqwyO+mMo6vgunKjaeImou/uNOhv5iqK+0kkqt5nePauWinCr5KGoCo2ZkBiC6ZqkYMDp6kg6zgq++Eqo0bUipqx6MNgLEe/rP773uggfSInLLM3aqH6MiyiQ5xOeSq3Ki7qviqLqrs/QmzL6o4RPqo+8C8uabIuXkqbElqCErZCirQgK86i4bY+jvC/2CLrJqrq97qjrLgkuODGI5NWfg9yTgIPlsayIL4veqAL7vEqD+IzuxODKKo+PC69IiYz64zYR6qrG6wiK3ruCo5Ce6ix6rvC4yeyaWoO7gaoaVtDYbqDK6IuIpEoo+uuq68rvAKl6ngihgc4Jqp6pmagQo2vL72vkj42J04zmqYvc4NyPgD6tzq4ezozMzryzqtCqK7q4aN7P1Iug6SJG7IqK2vybicrSLKSzvv3SqBboI4wIugp/HMrPzY0rKHiuD+rZSKK/+6jqqOSuHEwK/6/46WmO7un6Aou8me/pusqA7B6pj6moqJbKvzALiJqoQ8iu6qiQMvtrkCuWT7nrLiuq6YoKj4tqieq+ALoTIEgWmWDuyQzCY4mKo+lKK5briir6YoY6uey4iaFqjqrSDz1CgZlboAnRLpqmq6YsDIygOeDIouBqC5wynqi7wIqju6+Ku2IpjtAIiNArKDCrSpq0uqt4u/CrD+JNzqu2ietgJwhi2OwiyIiYfqmJAsByDoqCi46A8aigDq7rr8rxma6ahwLJzqs4I5vSScm76yPu2ieqqrsi6oBMMNhspsyJjhisuPhPrsKLRqoh6poI6IKoRmq1o0iG4xiLKHqorRqK6vmix9aobyGoqO+64rnqo46oERvcrqwimu2Luv/qog6LOO5nQItbqvzKioH2ureLAumAr0rrLLMneoiwinMaDGpQkoD7u9iN78r8E8q7JimQMYJZvJizbgw6756tkOAeLMK76pzCHaKwaCjK/qXMDu8rqfZLtMiendLqjN16vLrqirqEKqSyAasqnLoL0KKo5KicL56RDeP9PjjY+ozqqhmC5rmoaKit3JvTvooi8tICj+H+LrkCuyyup+CuAA0Kq8OsuouwuL6Koq/LiKrikiav7pogirL6mhv9ktxrhS8tNprrqaKqJLgS6ogqqk14iLpqjIr22ood/Ay5jKigqryoAqWoHpkmzzuCruqro6u7CbGr0uhoxMuQytnmk6ld+6abu8Q+syqTg4KtKrpR/pikpaqpvojzQKTkDmhDqoeaGn775pKYuoiugCYoIOC6aum6m6yiIooKhNp6P+6LjJiS6Qpeno8qqQuqIATmgA6GqmqImK6vi+jYo7nJqj7+vLoOO7qp66LOwc2t4C5q6mI566Jaja6o7ajZKwzLqkq6j8OMorW6uqqILzbKeq6XorGpE4yKw50q7F2xi83snEvbq8GoGqEKiIieT7S6pkuujQwIuqq6vq/rn6w0oI8uOykarMmErp7Cv4ivigi5BqaLggSorAe6ifjB7iqLmmygqUmgrAsWrqCSzuoHuPfsgA+xCEC4w6mZsXL6goCh4r7Qo7zMKsabuFqLTJ+Bo+qqiOqPnqI/n+y4K4oKqqLgueiqW8qo66S+hqIf/IPtOoyxm6ma6Qu4rrplaOKv2g0m+s6Oi/3FmrqZT6KoI4kKo3KEhuqcgCuhLi+6spAoqqBkOLq5qS0osoy6+g2wpvOZqA5qiquXuL3pupOrou6ZICEojJijiruKoOm0qomLLpoOvKS+i6M6Qqriialmr3oYoeqaIo4Dyq6hgUKq4qiyileMz4jPXpmWq6lsqmqK6pp6y9zKi4+mQqwZiqamqa8yaqeK6Pmom6pODzsmroLjywu6BahPOqANlIAuQTigiKgughOiSm2B4rstgYycnqjSZOgDKaqmqu4nxe7NikT55OKJjji+rLrjCJyJjLqiamYKqMgtKv4gOibCOKiQDa4vKOiykrvqi/dvn5WI8QuOjLGk5pv6yugjj6qhgYiE6+apqgoGi75Ar7pIv9ursrKgraov2iS46k8KuUjy6KCQGijP+BoqtZm98sia8qy26okpoLKK4ugKsODFFcUqmKqcu4jjmL2KYDu8L97+hqWruby52KGu6AC7omrr2qnMkROmuurAe+DsrEuhZtqKWBorqqCqN8qbOaquto4oOGiCjGKuuogtggIVAjLq6b+ur4zOP+mipbniH+pOAC2s1/qqnAIK47ixqggc7G/L+4oumiubroKK+SyaCIiKHqWLvlw9i6J9G71wHui47XtaCkZtC/KeuqPayDCqjK0qYoj3+w9y5ar96e7k0CGCZciIyKjIKqokL0isrfqq+oajsIKhobDrCp2F3tlpgKi/giqqdbhqiqqCq7up5qrGukOu5gysrrmgm5jiHbRMwahIoZo6iKhB54Qpji+rq/CjwBTOK/JpvYmoC4CLMwCeovZYrOYSbdio++YPuSirVIpBumLpO5+Bmp6CniGBKxqFsqO6guCsLwjO765JWYiuKqqImB+4sCg6iWHyrvmMyC7JglcGzYg+gEesmJronqirGGMrzAaAhrv2xoIELCoAmqmsgyv8V7oQzPTmD6Ruv3GfqCpIzNroLDS/G2jC8mrq+6ubpoHM2lSO2prB7W6Kim6bJJzryMorotP8kkiqv+rb7v867k9L244u+sZKivKssup9m97CA7mghQcenuK5qMrGqkgIHrq7wtCqvEXu7yKKNYmoqokqy7wnA7gpoOAgKu/a5/yO64B9m6vITLmswjiTmKqKqKGP5o+8na4ppji88q6Khq7w6cqmSmXIApuLKuLwLqiKmsx+gfyqyq6eyoigqf8pjiq83iweGvjyLzAqosnLso8r/E6qoOhbco3KKb6pqMaL7Om6sE86uZaowcr5vITsstuqizCx6p0IqpMolqagCoTOoQ6i6BjG4KudfotLnNhexKbeT6rvbN/LPril/Jw6WE0+KIir+vqSpQ+JuyKzy1pEutqZionqWNKrkiLIg2goir/sqJkKirkp6OOniq7Ovnur1Ir/7qaLqo90CCiOlzqviazrqqdzG4Ooh95qfYqoKRpMGgo2CUEaymqrym8G7UuhbunKqG2JiM46lvr+uIrNOMjKK6KCr4634qs7yoOKygSyDojqMn+oG8i4mQjJkZhkCFIrqj2DqcqbfCpK2gP0GoiI+vAnKB2bzqmo6OrQzJlNlGwr6NuY+LKYhIiA/oyAvWjDKCjat4ypjS9jND+oCjsaz6upgKou6JvI2bgmm8q4KK+otqjtww4tCGmgcoIAKg3Cp6P/Dg/a8OCp+6sv46qIs6jqcqH6uSr4L7uOzKqY7oiYp5iiur6KI7WWCJCJt27ACpa/Fp9JUP4/6Raoor+pwJyiurrXu6y4qinrwol7jr7A9t5m6oqt6IuKisoqmL2s6vop3u4iKEghqm+uK9Cyhukq0ay/2lhM+D1RirmIOgoBdOK6mG6j3mIOybLbuwqOIr6gzKhpWrCzyY2v7kyqlbqpg4yvjpvnfS4PmG7p5qy7GAgjQqI4opjZOagqrcYEZuCmaqrubuIOQbgoHgyMzigECErqiPzoiLr5KOuqmLB6K46ny7qNGZqqOq4Lo5NKvoi4xKi+L6q7qiKuojCSCCuMTouobBKaqqpajOqC/lBqztjNnYrybK6SCsDvq+nIGgqpPLpqiuRDnRi5rlAE+qqI74iL6opqH60BjomsytySdjDJrJkaqiiMasiiuowMq+DNXroJkmzKiB6P7Iiv6RgvGLjvqooJCubrC4qwu6q4VIiopti/qs+u99LvCi4K78OsmY2L6uirzZXRAurMzgU/qhiAoguKqKLq44iqosiYqgpCmpuKsumK+qwOqrmvzurqSeyYtbcbp+3r9mhdKqKu8VKs36/qpossP0xg62btKUYqT62r844uDqjq3Ay/243JXqsFilj8+qLKvuesGMCrKvparoh6mK+oYbvLn/+iEMAf6GmOkx5i6a6hgGB7CpCaoWq4BFfT46jsi364mejiMBqYhisrohKpdIKpAvWefpKJo46ImQnp9rq6vf4smfyBmbxoooZomauKrs5jOKwMKNgirLwqsv2rvPqPKaiYmIYqirPb3OvOgaqpQ6caPsgsA6irrqPM4ur56MK62hZKiCjaD4kKrp2jmyC7TFjikjglyC6p/KEPhqGIMoPO2OiuhsjivAyqOIuKsLAJXBiDguL4guKljrgu7rz6iestzMHuOgLwPqqMq6iwgAtjs4gIiAmjrKoo/mq0Qq2aqqpJqIgLqL/Tm4SqKfjuQuqqqgLY87i4jq65fEgypAm5rKmKqBrdDoKiqiqrFOmvgL/qihqJKq9gCsuqKdoqPl6ZaJpqrPi4nnyMrtyKOTmauq69o4ibGqKvP/C6vBrirLq5kGq+re4q2e5y0MpTprCbuOmjMZqqj4HAjvSaxiuNaIa5qn1LKqz6q9yr2Os8u6KrCbGo2Lucn9wrOrsoiJ6KuYiKhUKjtDg5YMqADQirueqVqqnoihKBLpO7zSDuKBgPq/hivaLs+igOMinPhorCIq7q844IL6oMYej6poWptauu/vCvasn8pkq24ODiLm7aySLrsa9jGKmhgkvRCjgdvyzrqnjxucor4yuS6WtpKob6uQvpSyy4auiYg9qbKIw4igh7Slrssoxp45ir64Grm5K+HKaoKn5uKgvo65i5hqhIXahm6oDYqNgJvLOMiopE+IsM+76hrOimes6IiuqOc7mqUo7cqei6yPvCF6TpO6QjfOYcpu96EofIKmCIN4Nonhj+C4rIqLjTqzOSh4jo6bPquIic5I2p2Dha7jiPpP6IfDI7qjTOa6iYiqie4q4ISMOK4fKriG4uvjuBik7rO4WoUIr+qC+KuKqtiC4LKiiIqwyqy6IzDf7JiA6sztBI7SjqOviW/oKLviyou476b5mli75B/Yo8p4EqurapwYae7syOjrpLLmqo2ooOoiawyquCCtObCGxoutGKBMrmIwormcKOLp+96rmB6pehnPmMwlJDAuos4qBqqIO+3t0v4qKYiIGI6CuML+rlqoi4uyosKr8gpijAu5H5orloyOLKKvjjy47g2/qK7Jiq+QpqHIyFo6Ku05jPDciMOL6ICua7yqisD/9q0M6JGIyPpzzIjTinajjpn6qZIe680s4in4O374IbrIigI6hRnaqprum56b9jhM6kq3luqPiJe9O6ys9M5asq8Mv4ojac76iwhTrsj8roqq4+zanzhfSIqoGKiPrunjJ4KAqwC8l8Qkksqu3ggKo6EbyuirksyqcJq4qqDqPpKPbtBqq9mq7rqb+gC5QDU77qjOluagqjiomrDq5ggrBoI9LireEOssSLqDxrqe5wiUKLig1MiLHyuJDYi67uv7/M7HKtKImujSiGKOanqogXoq2urnrEyAIjnbaG7IsYeUALLugDroqs6uteot4q0ioYuKrIctqsyIqN4IHoKsmKtZr4tIh7noLFiiiIhqvLYwuDIux7aioasqoqSosChcoOmMcJztzOMLv4Z2qomADyqqInD7muqUQcyJjNDIpaqYXSnqirKJuypaLqYgIZiOuJjZmzr3as7rDTCKZqqjuJ0gOt31qRwDo6/9pKyYzwyoeHw4YgmoyCiqDcqcgosDkqbckqisuvqKoDxKqKq0xthPjveQAfqpj+Ti4tJKqgiepboczOm6rjrqUQqgz+nSj2vK3bqKrqguyag2BAKKNTpooq637x8Gqoikmwoiio7bqCQM7TCjqpPJiLqL2gqoqiLNzIhLDGico+6q+v5TKK16mIqrnNmq2soKiSqgz85gXuj7reuLqp6D7ymq2J6GCIqgusz6raBOKpC4KJo07cyKsoCbuapCmKDvuoyqqmqqTarL+qv+NbprqIhv6J+bjuCyikgKy2yMm2rDy8KqolqwoOzvLarI7FGC6ovzuuSNaoICej4Mb6rRqO7ckcAMLr6Snq24Seos3ovfr64ukKvsCijE9qS9y6q4mL4KrLuCLuYKzOEs+fCqqbzMuhPruIqqyE1PT75nKtGE0KioosUavIK6LItLjtoSD6uZGLopFhsqKv4uofBqrkwqosjKa7mzj02Ii4aqrpnKjqmyHbsNgPgKEK6unLnm34sutaAL"; +const BLOOM_VERSION = "efb9e3f964c0f417"; + +// --------------------------------------------------------------------------- +// Database client (verified_ips only) +// --------------------------------------------------------------------------- + +const db = createClient({ + url: process.env.BUNNY_DATABASE_URL!, + authToken: process.env.BUNNY_DATABASE_AUTH_TOKEN!, +}); + +// --------------------------------------------------------------------------- +// Turnstile config +// --------------------------------------------------------------------------- + +const TURNSTILE_SITE_KEY = process.env.TURNSTILE_SITE_KEY || ""; +const TURNSTILE_SECRET_KEY = process.env.TURNSTILE_SECRET_KEY || ""; + +// --------------------------------------------------------------------------- +// Bloom filter (FNV-1a — must match bouncer.py) +// --------------------------------------------------------------------------- + +function base64ToBytes(b64: string): Uint8Array { + const bin = atob(b64); + const bytes = new Uint8Array(bin.length); + for (let i = 0; i < bin.length; i++) { + bytes[i] = bin.charCodeAt(i); + } + return bytes; +} + +let bloomBits: Uint8Array | null = null; +let bloomM = 0; +let bloomK = 0; + +if (BLOOM_B64 && !BLOOM_B64.startsWith("__")) { + const raw = base64ToBytes(BLOOM_B64); + bloomM = raw[0] | (raw[1] << 8) | (raw[2] << 16) | (raw[3] << 24); + bloomK = raw[4] | (raw[5] << 8) | (raw[6] << 16) | (raw[7] << 24); + bloomBits = raw.slice(8); + console.log( + `Bloom filter loaded: version=${BLOOM_VERSION}, m=${bloomM}, k=${bloomK}, ${bloomBits.length} bytes`, + ); +} + +function fnv1a32(data: Uint8Array): number { + let h = 0x811c9dc5; + for (let i = 0; i < data.length; i++) { + h ^= data[i]; + h = Math.imul(h, 0x01000193) >>> 0; + } + return h; +} + +function isBlocked(ip: string): boolean { + if (!bloomBits) return false; + const enc = new TextEncoder(); + const ipBytes = enc.encode(ip); + const ipBytesFF = new Uint8Array(ipBytes.length + 1); + ipBytesFF.set(ipBytes); + ipBytesFF[ipBytes.length] = 0xff; + const h1 = fnv1a32(ipBytes); + const h2 = fnv1a32(ipBytesFF); + for (let i = 0; i < bloomK; i++) { + const pos = (h1 + i * h2) % bloomM; + if ((bloomBits[pos >> 3] & (1 << (pos & 7))) === 0) return false; + } + return true; +} + +// --------------------------------------------------------------------------- +// Clean IP cache (negative cache — bloom에 없는 정상 IP) +// --------------------------------------------------------------------------- + +const cleanIpCache = new Set(); +const CLEAN_CACHE_MAX = 50_000; + +function isCleanCached(ip: string): boolean { + return cleanIpCache.has(ip); +} + +function addCleanCache(ip: string): void { + if (cleanIpCache.size >= CLEAN_CACHE_MAX) cleanIpCache.clear(); + cleanIpCache.add(ip); +} + +// --------------------------------------------------------------------------- +// Verified IP cache (memory only, DB는 캡차 검증 시에만) +// --------------------------------------------------------------------------- + +const verifiedCache = new Map(); +const VERIFIED_TTL_MS = 14_400_000; +const VERIFIED_CACHE_MAX = 10_000; + +function cacheEvict(m: Map, maxSize: number): void { + if (m.size < maxSize) return; + const now = Date.now(); + for (const [key, val] of m) { + if (val <= now) m.delete(key); + } + if (m.size >= maxSize) m.clear(); +} + +// --------------------------------------------------------------------------- +// HMAC cookie signing (Web Crypto API) +// --------------------------------------------------------------------------- + +async function hmacSign(data: string, secret: string): Promise { + const enc = new TextEncoder(); + const key = await crypto.subtle.importKey( + "raw", + enc.encode(secret), + { name: "HMAC", hash: "SHA-256" }, + false, + ["sign"], + ); + const sig = await crypto.subtle.sign("HMAC", key, enc.encode(data)); + return Array.from(new Uint8Array(sig)) + .map((b) => b.toString(16).padStart(2, "0")) + .join(""); +} + +// --------------------------------------------------------------------------- +// Cookie helper +// --------------------------------------------------------------------------- + +function getCookie(request: Request, name: string): string | null { + const header = request.headers.get("cookie") || ""; + const match = header.match(new RegExp(`(?:^|;\\s*)${name}=([^;]*)`)); + return match ? decodeURIComponent(match[1]) : null; +} + +// --------------------------------------------------------------------------- +// Verified IP check (memory cache → cookie → DB) +// --------------------------------------------------------------------------- + +async function isVerified(ip: string, request: Request): Promise { + const cachedExp = verifiedCache.get(ip); + if (cachedExp && cachedExp > Date.now()) return true; + if (cachedExp) verifiedCache.delete(ip); + + const cookie = getCookie(request, "_cs_verified"); + if (cookie) { + const parts = cookie.split(":"); + if (parts.length === 3) { + const [cookieIp, ts, sig] = parts; + if (cookieIp === ip) { + const expected = await hmacSign(`${cookieIp}:${ts}`, TURNSTILE_SECRET_KEY); + if (expected === sig) { + const timestamp = parseInt(ts, 10); + if (Date.now() - timestamp < VERIFIED_TTL_MS) { + verifiedCache.set(ip, timestamp + VERIFIED_TTL_MS); + return true; + } + } + } + } + } + + try { + const result = await db.execute({ + sql: "SELECT 1 FROM verified_ips WHERE ip = ? AND expires_at > datetime('now')", + args: [ip], + }); + if (result.rows.length > 0) { + verifiedCache.set(ip, Date.now() + VERIFIED_TTL_MS); + return true; + } + } catch (err) { + console.error("Verified IP lookup failed:", err); + } + + return false; +} + +// --------------------------------------------------------------------------- +// Turnstile server-side verification +// --------------------------------------------------------------------------- + +async function verifyTurnstile(token: string, ip: string): Promise { + try { + const resp = await fetch( + "https://challenges.cloudflare.com/turnstile/v0/siteverify", + { + method: "POST", + headers: { "Content-Type": "application/x-www-form-urlencoded" }, + body: `secret=${encodeURIComponent(TURNSTILE_SECRET_KEY)}&response=${encodeURIComponent(token)}&remoteip=${encodeURIComponent(ip)}`, + }, + ); + const data = (await resp.json()) as { success: boolean }; + return data.success === true; + } catch (err) { + console.error("Turnstile verification failed:", err); + return false; + } +} + +// --------------------------------------------------------------------------- +// Store verified IP +// --------------------------------------------------------------------------- + +async function storeVerified(ip: string): Promise { + try { + await db.execute({ + sql: "INSERT OR REPLACE INTO verified_ips (ip, expires_at) VALUES (?, datetime('now', '+4 hours'))", + args: [ip], + }); + cacheEvict(verifiedCache, VERIFIED_CACHE_MAX); + verifiedCache.set(ip, Date.now() + VERIFIED_TTL_MS); + } catch (err) { + console.error("Failed to store verified IP:", err); + } +} + +// --------------------------------------------------------------------------- +// CAPTCHA HTML page +// --------------------------------------------------------------------------- + +function escapeHtml(s: string): string { + return s.replace(/&/g, "&").replace(//g, ">").replace(/"/g, """); +} + +function captchaPage(originalPath: string, error?: string): Response { + const safeRedirect = escapeHtml(originalPath); + const html = ` + + + + + 보안 확인 + + + + +
+

보안 확인이 필요합니다

+

계속하려면 아래 확인을 완료해주세요.

+ ${error ? `
${escapeHtml(error)}
` : ""} +
+ +
+
+
+ + +`; + + return new Response(html, { + status: 403, + headers: { + "Content-Type": "text/html; charset=utf-8", + "Cache-Control": "no-store", + }, + }); +} + +// --------------------------------------------------------------------------- +// CAPTCHA verify handler +// --------------------------------------------------------------------------- + +async function handleCaptchaVerify( + request: Request, + ip: string, +): Promise { + const body = await request.text(); + const params = new URLSearchParams(body); + const token = params.get("cf-turnstile-response") || ""; + const redirect = params.get("redirect") || "/"; + + if (!token || !(await verifyTurnstile(token, ip))) { + return captchaPage(redirect, "확인에 실패했습니다. 다시 시도해주세요."); + } + + await storeVerified(ip); + + const ts = Date.now().toString(); + const sig = await hmacSign(`${ip}:${ts}`, TURNSTILE_SECRET_KEY); + const cookieValue = encodeURIComponent(`${ip}:${ts}:${sig}`); + const safeRedirect = escapeHtml(redirect); + + return new Response( + `

확인 완료. 이동 중…

`, + { + status: 200, + headers: { + "Content-Type": "text/html; charset=utf-8", + "Set-Cookie": `_cs_verified=${cookieValue}; Path=/; HttpOnly; Secure; SameSite=Lax; Max-Age=14400`, + "Cache-Control": "no-store", + }, + }, + ); +} + +// --------------------------------------------------------------------------- +// Middleware +// --------------------------------------------------------------------------- + +BunnySDK.net.http + .servePullZone() + .onOriginRequest(async (ctx) => { + const ip = ctx.request.headers.get("X-Real-Ip"); + if (!ip) return ctx.request; + + if (isCleanCached(ip)) return ctx.request; + + if (isBlocked(ip)) { + if (!TURNSTILE_SITE_KEY || !TURNSTILE_SECRET_KEY) { + return new Response("Forbidden", { status: 403 }); + } + + if (await isVerified(ip, ctx.request)) { + return ctx.request; + } + + const url = new URL(ctx.request.url); + + if (url.pathname === "/__captcha/verify" && ctx.request.method === "POST") { + return handleCaptchaVerify(ctx.request, ip); + } + + return captchaPage(url.pathname + url.search); + } + + addCleanCache(ip); + return ctx.request; + }); + diff --git a/sync/bouncer.py b/sync/bouncer.py new file mode 100644 index 0000000..9e8ad66 --- /dev/null +++ b/sync/bouncer.py @@ -0,0 +1,411 @@ +#!/usr/bin/env python3 +""" +CrowdSec → BunnyCDN Bouncer Sync + +CrowdSec LAPI에서 ban 결정을 가져와 bloom filter로 변환 후 +BunnyCDN Edge Script 코드에 임베딩하여 배포. + +Usage: + python3 bouncer.py # 스트림 모드 (delta) + python3 bouncer.py --startup # 전체 동기화 + python3 bouncer.py --dry-run # 변경 없이 출력만 + +Environment: + CROWDSEC_LAPI_URL CrowdSec LAPI URL (default: http://10.253.100.240:8080) + CROWDSEC_BOUNCER_KEY Bouncer API key + BUNNY_API_KEY BunnyCDN account API key + BUNNY_SCRIPT_ID Edge Script ID (default: 64811) +""" + +import argparse +import base64 +import hashlib +import json +import logging +import math +import os +import re +import struct +import sys +import time +from ipaddress import IPv4Network, ip_address +from typing import Optional + +import requests + +logging.basicConfig( + level=logging.INFO, + format="%(asctime)s [%(levelname)s] %(message)s", +) +log = logging.getLogger("bouncer") + +# --------------------------------------------------------------------------- +# Config +# --------------------------------------------------------------------------- + +CROWDSEC_LAPI_URL = os.environ.get("CROWDSEC_LAPI_URL", "http://10.253.100.240:8080") +CROWDSEC_BOUNCER_KEY = os.environ.get("CROWDSEC_BOUNCER_KEY", "") +BUNNY_API_KEY = os.environ.get("BUNNY_API_KEY", "") +BUNNY_SCRIPT_ID = int(os.environ.get("BUNNY_SCRIPT_ID", "64811")) + +# Bloom filter parameters +BLOOM_FP_RATE = 0.001 # 0.1% false positive rate +BLOOM_MIN_ITEMS = 100 # minimum expected items (avoid tiny filters) + +# State file for stream cursor +STATE_FILE = os.environ.get("STATE_FILE", "/var/lib/crowdsec-bouncer/state.json") + +# --------------------------------------------------------------------------- +# Bloom filter (FNV-1a — must match edge script) +# --------------------------------------------------------------------------- + + +def fnv1a32(data: bytes) -> int: + h = 0x811C9DC5 + for b in data: + h ^= b + h = (h * 0x01000193) & 0xFFFFFFFF + return h + + +class BloomFilter: + def __init__(self, expected_items: int, fp_rate: float = 0.001): + n = max(expected_items, BLOOM_MIN_ITEMS) + self.m = max(self._optimal_m(n, fp_rate), 64) # bits + self.k = max(self._optimal_k(self.m, n), 1) # hash functions + byte_count = (self.m + 7) // 8 + self.bits = bytearray(byte_count) + + @staticmethod + def _optimal_m(n: int, p: float) -> int: + return int(-n * math.log(p) / (math.log(2) ** 2)) + + @staticmethod + def _optimal_k(m: int, n: int) -> int: + return max(int((m / n) * math.log(2)), 1) + + def add(self, item: str) -> None: + ip_bytes = item.encode("utf-8") + ip_bytes_ff = ip_bytes + b"\xff" + h1 = fnv1a32(ip_bytes) + h2 = fnv1a32(ip_bytes_ff) + for i in range(self.k): + pos = (h1 + i * h2) % self.m + self.bits[pos >> 3] |= 1 << (pos & 7) + + def to_base64(self) -> str: + # Header: 4 bytes m (little-endian), 4 bytes k (little-endian) + header = struct.pack(" str: + return hashlib.md5(self.bits).hexdigest()[:16] + + +# --------------------------------------------------------------------------- +# CIDR expansion +# --------------------------------------------------------------------------- + + +def expand_cidr(value: str) -> list[str]: + """Expand a CIDR range to individual IPs (max /16).""" + try: + net = IPv4Network(value, strict=False) + if net.prefixlen < 16: + log.warning("Skipping too-large range: %s", value) + return [] + if net.prefixlen == 32: + return [str(net.network_address)] + return [str(ip) for ip in net.hosts()] + except ValueError: + # Not a CIDR, treat as single IP + try: + ip_address(value) + return [value] + except ValueError: + log.warning("Invalid IP/CIDR: %s", value) + return [] + + +# --------------------------------------------------------------------------- +# CrowdSec LAPI client +# --------------------------------------------------------------------------- + + +def load_state() -> Optional[str]: + """Load last stream cursor timestamp.""" + try: + with open(STATE_FILE) as f: + data = json.load(f) + return data.get("last_pull") + except (FileNotFoundError, json.JSONDecodeError): + return None + + +def save_state(timestamp: str) -> None: + """Save stream cursor timestamp.""" + os.makedirs(os.path.dirname(STATE_FILE), exist_ok=True) + with open(STATE_FILE, "w") as f: + json.dump({"last_pull": timestamp}, f) + + +def fetch_decisions(startup: bool = False) -> tuple[list[str], list[str]]: + """Fetch decisions from CrowdSec LAPI stream endpoint. + + Returns (new_ips, deleted_ips). + """ + url = f"{CROWDSEC_LAPI_URL}/v1/decisions/stream" + params = {"startup": "true"} if startup else {} + + if not startup: + last_pull = load_state() + if last_pull: + # For non-startup, we only care about changes since last pull + pass + + headers = {"X-Api-Key": CROWDSEC_BOUNCER_KEY} + + try: + resp = requests.get(url, headers=headers, params=params, timeout=30) + resp.raise_for_status() + except requests.RequestException as e: + log.error("Failed to fetch decisions: %s", e) + return [], [] + + data = resp.json() + new_ips = [] + deleted_ips = [] + + for decision in data.get("new") or []: + value = decision.get("value", "") + if decision.get("type", "").lower() == "ban": + new_ips.extend(expand_cidr(value)) + + for decision in data.get("deleted") or []: + value = decision.get("value", "") + deleted_ips.extend(expand_cidr(value)) + + save_state(time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())) + return new_ips, deleted_ips + + +def fetch_all_active_decisions() -> list[str]: + """Fetch all currently active ban decisions.""" + url = f"{CROWDSEC_LAPI_URL}/v1/decisions" + headers = {"X-Api-Key": CROWDSEC_BOUNCER_KEY} + + try: + resp = requests.get(url, headers=headers, params={"type": "ban"}, timeout=30) + resp.raise_for_status() + except requests.RequestException as e: + log.error("Failed to fetch all decisions: %s", e) + return [] + + decisions = resp.json() or [] + ips = [] + for d in decisions: + value = d.get("value", "") + ips.extend(expand_cidr(value)) + return ips + + +# --------------------------------------------------------------------------- +# IP state management +# --------------------------------------------------------------------------- + + +class IPState: + """Manages the set of currently blocked IPs.""" + + def __init__(self, state_file: str): + self._file = state_file.replace("state.json", "blocked_ips.json") + self.ips: set[str] = set() + self._load() + + def _load(self) -> None: + try: + with open(self._file) as f: + data = json.load(f) + self.ips = set(data.get("ips", [])) + except (FileNotFoundError, json.JSONDecodeError): + self.ips = set() + + def save(self) -> None: + os.makedirs(os.path.dirname(self._file), exist_ok=True) + with open(self._file, "w") as f: + json.dump({"ips": sorted(self.ips)}, f) + + def apply_delta(self, new_ips: list[str], deleted_ips: list[str]) -> bool: + """Apply changes and return True if the set changed.""" + before = len(self.ips) + self.ips.update(new_ips) + self.ips -= set(deleted_ips) + after = len(self.ips) + changed = before != after or bool(new_ips) or bool(deleted_ips) + if changed: + self.save() + return changed + + def full_sync(self, all_ips: list[str]) -> bool: + """Full replacement. Returns True if changed.""" + new_set = set(all_ips) + if new_set == self.ips: + return False + self.ips = new_set + self.save() + return True + + def build_bloom(self) -> BloomFilter: + bf = BloomFilter(len(self.ips)) + for ip in self.ips: + bf.add(ip) + return bf + + +# --------------------------------------------------------------------------- +# BunnyCDN Edge Script updater +# --------------------------------------------------------------------------- + +BLOOM_PATTERN = re.compile( + r'(const BLOOM_B64\s*=\s*")([^"]*)(";)', + re.DOTALL, +) +VERSION_PATTERN = re.compile( + r'(const BLOOM_VERSION\s*=\s*")([^"]*)(";)', +) + + +def get_current_script() -> Optional[str]: + """Get current Edge Script source code.""" + url = f"https://api.bunny.net/compute/script/{BUNNY_SCRIPT_ID}/code" + headers = {"AccessKey": BUNNY_API_KEY} + + try: + resp = requests.get(url, headers=headers, timeout=30) + resp.raise_for_status() + return resp.json().get("Code", "") + except requests.RequestException as e: + log.error("Failed to get script: %s", e) + return None + + +def update_script(code: str) -> bool: + """Update Edge Script source code and publish.""" + url = f"https://api.bunny.net/compute/script/{BUNNY_SCRIPT_ID}/code" + headers = { + "AccessKey": BUNNY_API_KEY, + "Content-Type": "application/json", + } + + try: + resp = requests.post(url, headers=headers, json={"Code": code}, timeout=60) + resp.raise_for_status() + except requests.RequestException as e: + log.error("Failed to update script: %s", e) + return False + + # Publish + pub_url = f"https://api.bunny.net/compute/script/{BUNNY_SCRIPT_ID}/publish" + try: + resp = requests.post(pub_url, headers=headers, json={}, timeout=60) + resp.raise_for_status() + log.info("Script published successfully") + return True + except requests.RequestException as e: + log.error("Failed to publish script: %s", e) + return False + + +def embed_bloom_in_script(code: str, bloom: BloomFilter) -> Optional[str]: + """Replace BLOOM_B64 and BLOOM_VERSION in script code.""" + b64 = bloom.to_base64() + version = bloom.version_hash() + + new_code = BLOOM_PATTERN.sub(rf'\g<1>{b64}\g<3>', code) + if new_code == code: + log.error("Could not find BLOOM_B64 in script") + return None + + new_code = VERSION_PATTERN.sub(rf'\g<1>{version}\g<3>', new_code) + return new_code + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + + +def main() -> int: + parser = argparse.ArgumentParser(description="CrowdSec → BunnyCDN Bouncer") + parser.add_argument("--startup", action="store_true", help="Full sync (startup mode)") + parser.add_argument("--dry-run", action="store_true", help="Don't deploy changes") + parser.add_argument("--verbose", "-v", action="store_true") + args = parser.parse_args() + + if args.verbose: + logging.getLogger().setLevel(logging.DEBUG) + + if not CROWDSEC_BOUNCER_KEY: + log.error("CROWDSEC_BOUNCER_KEY not set") + return 1 + if not BUNNY_API_KEY: + log.error("BUNNY_API_KEY not set") + return 1 + + state = IPState(STATE_FILE) + log.info("Current blocked IPs: %d", len(state.ips)) + + if args.startup: + log.info("Full sync mode (startup)") + new_ips, _ = fetch_decisions(startup=True) + # startup=true returns all active decisions in "new" + changed = state.full_sync(new_ips) + else: + log.info("Delta sync mode") + new_ips, deleted_ips = fetch_decisions(startup=False) + if new_ips: + log.info("New bans: %d IPs", len(new_ips)) + if deleted_ips: + log.info("Removed bans: %d IPs", len(deleted_ips)) + changed = state.apply_delta(new_ips, deleted_ips) + + log.info("Total blocked IPs after sync: %d", len(state.ips)) + + if not changed: + log.info("No changes — skipping deploy") + return 0 + + bloom = state.build_bloom() + b64 = bloom.to_base64() + version = bloom.version_hash() + log.info("Bloom filter: m=%d, k=%d, size=%d bytes, version=%s", + bloom.m, bloom.k, len(b64), version) + + if args.dry_run: + log.info("[DRY RUN] Would update script with %d IPs", len(state.ips)) + log.info("[DRY RUN] Bloom b64 length: %d chars", len(b64)) + return 0 + + code = get_current_script() + if code is None: + return 1 + + new_code = embed_bloom_in_script(code, bloom) + if new_code is None: + return 1 + + # Check if bloom actually changed + if new_code == code: + log.info("Bloom filter unchanged — skipping deploy") + return 0 + + if update_script(new_code): + log.info("Successfully deployed bloom filter with %d IPs", len(state.ips)) + return 0 + + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/sync/requirements.txt b/sync/requirements.txt new file mode 100644 index 0000000..0eb8cae --- /dev/null +++ b/sync/requirements.txt @@ -0,0 +1 @@ +requests>=2.31.0