nginx-proxy-manager/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

This is a utility collection for managing nginx-proxy-manager log streaming to Cloudflare R2 storage. The project contains bash scripts and configuration helpers for setting up real-time log monitoring and upload systems, particularly designed for integration with CrowdSec security analysis.

Key Components

Log Upload Utilities (upload_log_file_fixed.sh)

Core functionality for reliable file uploads to Cloudflare R2 using rclone:

• upload_log_file() - Robust upload function with retry logic and proper error handling
• upload_log_file_minimal() - Simplified version focusing on exit code checking
• monitor_and_upload_logs() - Continuous monitoring loop for log directory surveillance
• test_rclone_upload() - Validation function for rclone configuration testing

Key Implementation Details:

• Uses rclone exit codes (0=success) rather than parsing output text for reliability
• Implements exponential backoff retry mechanism with configurable attempts
• Includes timeout protection (300s default) to prevent hanging operations
• Captures both stdout/stderr for detailed error reporting when failures occur

Error Handling Patterns (error_handling_comparison.sh)

Documentation and examples showing:

• Common anti-patterns when working with rclone output parsing
• Why checking exit codes is more reliable than parsing "Transferred: X/Y" messages
• Best practices for monitoring loop stability (don't exit on individual upload failures)

Remote Server Integration

The scripts are designed to work with a remote nginx-proxy-manager setup running in Podman containers on Debian systems. The typical deployment includes:

• System tuning: Kernel parameters optimized for container workloads and proxy traffic
• Log streaming: Real-time extraction from Podman containers and systemd journals
• R2 integration: Direct upload of uncompressed log files for CrowdSec consumption
• Service automation: systemd user services for continuous operation

Development Commands

Testing rclone Configuration

# Test basic upload functionality
./upload_log_file_fixed.sh

# Source the functions for interactive testing
source upload_log_file_fixed.sh
upload_log_file "/path/to/logfile" "cloudflare-r2:bucket/path"

Remote Server Management

When working with the remote server deployment:

# Test log streaming script
ssh user@server "/home/user/scripts/npm-log-streamer.sh test"

# Manual log sync
ssh user@server "/home/user/scripts/npm-log-streamer.sh sync"

# Check service status
ssh user@server "systemctl --user status npm-log-streamer.service"

# View streaming logs
ssh user@server "journalctl --user -u npm-log-streamer.service -f"

Cloudflare R2 Operations

# List uploaded files
rclone ls cloudflare-r2:npm-logs/

# Test connection
rclone lsd cloudflare-r2:

# Manual file upload
rclone copy localfile.log cloudflare-r2:npm-logs/path/

Architecture Considerations

Reliability Design

• Fail-safe monitoring: Individual upload failures don't terminate the monitoring service
• Retry mechanisms: Built-in exponential backoff for transient network issues
• Timeout handling: Prevents indefinite hangs on network problems
• Resource cleanup: Automatic cleanup of temporary files and connections

Integration Points

• Podman containers: Log extraction from running nginx-proxy-manager containers
• systemd integration: User-level services for automatic startup and restart
• CrowdSec compatibility: Uncompressed log files uploaded in real-time for security analysis
• R2 bucket organization: Hierarchical structure with hostname/date organization

Performance Characteristics

• Upload frequency: 1-minute intervals for real-time log availability
• Batch processing: Multiple log files processed in single sync cycle
• Memory efficiency: Streaming operations avoid large memory buffers
• Network optimization: Configurable retry counts and timeout values

Troubleshooting

Common Issues

• TLS handshake failures: Usually indicate incorrect R2 credentials or endpoint configuration
• Exit code 126: Typically permissions issues with script execution
• Service restart loops: Often caused by missing dependencies or configuration errors

Debugging Commands

# Check rclone configuration
rclone config show cloudflare-r2

# Test direct R2 connection
rclone lsd cloudflare-r2: --verbose

# Verify systemd service configuration
systemctl --user status npm-log-streamer.service --no-pager

# Check recent service logs
journalctl --user -u npm-log-streamer.service --since "1 hour ago"