A production-ready hybrid load balancer implementation in Go that supports both Layer 4 (TCP) and Layer 7 (HTTP) load balancing with multiple algorithms, health checking, and monitoring.
- Layer 4 (TCP) Load Balancing: Direct TCP connection forwarding with bidirectional proxying
- Layer 7 (HTTP) Load Balancing: HTTP-aware reverse proxy with header inspection and manipulation
- Multiple Algorithms: Round-robin, least connections, weighted round-robin
- Health Checking: Active health monitoring with automatic backend failover
- Graceful Shutdown: Clean shutdown handling for all components
- Management API: RESTful API for statistics and monitoring
- Thread-safe backend pool management
- Connection tracking and statistics
- Configurable health check intervals
- Support for backend weights
- Real-time metrics and statistics
- Demo backend servers for testing
┌─────────────┐
│ Client │
└──────┬──────┘
│
├─────────────────┬──────────────────┐
│ │ │
┌──────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐
│ L4 Balancer │ │ L7 Balancer │ │ Management │
│ :9000 │ │ :8000 │ │ API :9090 │
└──────┬──────┘ └──────┬──────┘ └─────────────┘
│ │
│ ┌────────────┼────────────┐
│ │ │ │
┌──▼────▼──┐ ┌─────▼─────┐ ┌─▼──────────┐
│ Backend │ │ Backend │ │ Backend │
│ :8081 │ │ :8082 │ │ :8083 │
└──────────┘ └───────────┘ └────────────┘
.
├── cmd/
│ ├── loadbalancer/main.go # Main load balancer entry point
│ └── backend/main.go # Demo backend servers
├── pkg/
│ ├── backend/
│ │ └── pool.go # Backend pool management
│ ├── health/
│ │ └── checker.go # Health checking logic
│ └── lb/
│ ├── balancer.go # Balancer interface
│ ├── algorithms.go # Load balancing algorithms
│ ├── l4.go # TCP load balancer
│ └── l7.go # HTTP load balancer
├── config.json # Configuration file
├── go.mod # Go module definition
└── README.md # This file
- Go 1.21 or higher
# Build the load balancer
go build -o loadbalancer cmd/loadbalancer/main.go
# Build the backend server
go build -o backend cmd/backend/main.goStart three demo backend servers with different configurations:
# Terminal 1: Backend 1 (Normal - 50ms delay)
go run cmd/backend/main.go -port 8081 -name "Backend-1" -delay 50
# Terminal 2: Backend 2 (Fast - 30ms delay)
go run cmd/backend/main.go -port 8082 -name "Backend-2" -delay 30
# Terminal 3: Backend 3 (Slow with failures - 100ms delay, 10% failure rate)
go run cmd/backend/main.go -port 8083 -name "Backend-3" -delay 100 -failure-rate 0.1# Terminal 4: Start the load balancer
go run cmd/loadbalancer/main.go
# Or with custom config:
go run cmd/loadbalancer/main.go -config config.json# Make HTTP requests (default round-robin)
curl http://localhost:8000/
# Test different endpoints
curl http://localhost:8000/api/data
curl http://localhost:8000/slow
# View L7 statistics
curl http://localhost:8000/stats# Test TCP load balancer (requires telnet or nc)
echo "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n" | nc localhost 9000# View combined statistics
curl http://localhost:9090/stats
# View L4-specific stats
curl http://localhost:9090/stats/l4
# View L7-specific stats
curl http://localhost:9090/stats/l7
# Health check
curl http://localhost:9090/healthConfiguration can be provided via JSON file:
{
"l4": {
"listen": ":9000",
"backends": [
{"address": "localhost:8081", "weight": 1},
{"address": "localhost:8082", "weight": 2},
{"address": "localhost:8083", "weight": 1}
],
"algorithm": "least-connections"
},
"l7": {
"listen": ":8000",
"backends": [
{"address": "http://localhost:8081", "weight": 1},
{"address": "http://localhost:8082", "weight": 2},
{"address": "http://localhost:8083", "weight": 1}
],
"algorithm": "round-robin"
},
"health": {
"interval": "5s",
"timeout": "2s"
}
}- round-robin: Distributes requests evenly across all backends
- least-connections: Routes to the backend with fewest active connections
- weighted-round-robin: Distributes based on backend weights
address: Backend server addressweight: Backend weight for weighted algorithms (higher = more traffic)
interval: Time between health checks (e.g., "5s", "1m")timeout: Health check timeout (e.g., "2s", "500ms")
Cycles through backends in order, distributing load evenly.
Request 1 → Backend 1
Request 2 → Backend 2
Request 3 → Backend 3
Request 4 → Backend 1
Routes traffic to the backend with the fewest active connections.
Backend 1: 5 connections ← New request goes here
Backend 2: 10 connections
Backend 3: 8 connections
Distributes traffic based on backend weights. A backend with weight 2 receives twice as much traffic as one with weight 1.
Weight 1 → 25% of traffic
Weight 2 → 50% of traffic
Weight 1 → 25% of traffic
The health checker periodically monitors backend health:
- L4 (TCP): Attempts TCP connection to each backend
- L7 (HTTP): Sends HTTP GET requests to
/healthendpoint (falls back to/)
Backends are automatically:
- Removed from rotation when they fail health checks
- Restored when they become healthy again
Health check status changes are logged:
[Health] Backend localhost:8081 is now DOWN
[Health] Backend localhost:8081 is now UP
{
"total_requests": 150,
"failed_requests": 5,
"avg_response_time_ms": 45,
"requests_by_path": {
"/": 100,
"/api/data": 40,
"/slow": 10
},
"backends": [
{
"address": "http://localhost:8081",
"healthy": true,
"connections": 2,
"weight": 1
}
]
}{
"total_connections": 50,
"active_connections": 3,
"failed_connections": 2,
"bytes_transferred": 1048576,
"backends": {
"total": 3,
"healthy": 3
}
}Demo backend servers provide several endpoints:
GET /- Basic endpoint returning server infoGET /health- Health check endpointGET /stats- Server statisticsGET /api/data- API data endpoint with random dataGET /slow- Slow endpoint (3x normal delay)
go run cmd/backend/main.go [options]
Options:
-port int Server port (default 8081)
-name string Server name (default "Backend-<port>")
-delay int Response delay in milliseconds (default 100)
-failure-rate float Failure rate 0.0-1.0 (default 0.0)Layer 4 (TCP):
- Operates at the transport layer
- Routes based on IP addresses and ports
- No awareness of application protocol
- Lower latency, higher throughput
- Can handle any TCP traffic (HTTP, MySQL, Redis, etc.)
Layer 7 (HTTP):
- Operates at the application layer
- Routes based on HTTP content (headers, paths, cookies)
- Full protocol awareness
- Advanced routing (path-based, header-based)
- Can modify requests/responses
- Enables session persistence, SSL termination
- Goroutines: Lightweight threads for handling connections
- Mutexes: Protect shared state (backend health, statistics)
- Atomic Operations: Lock-free connection counting
- Channels: Graceful shutdown coordination
- Thread-Safe Pool: RWMutex for concurrent backend access
- Connection Tracking: Atomic counters for accurate least-connections
- Health Checks: Separate goroutines with configurable intervals
- Bidirectional Proxy: Concurrent io.Copy for L4 forwarding
- Graceful Shutdown: Context-based cancellation and WaitGroups
- Health checks automatically remove unhealthy backends
- Failed requests don't crash the load balancer
- Connection errors are logged and counted
- Graceful degradation when backends are unavailable
- Connection pooling for HTTP backends
- Efficient round-robin with atomic counter
- Lock-free operations where possible
- Minimal memory allocations
# Send 10 requests and observe distribution
for i in {1..10}; do
curl -s http://localhost:8000/ | jq .server
done# 1. Start all backends
# 2. Send requests (observe distribution)
# 3. Stop one backend (Ctrl+C)
# 4. Wait for health check (5s)
# 5. Continue sending requests (observe failover)# Send concurrent requests to see least-connections in action
seq 1 20 | xargs -P 10 -I {} curl -s http://localhost:8000/slow# Use Apache Bench or similar
ab -n 1000 -c 10 http://localhost:8000/
# Or with wrk
wrk -t 4 -c 100 -d 30s http://localhost:8000/For production use, consider adding:
- Configuration Management: Environment variables, config reloading
- Observability: Prometheus metrics, distributed tracing
- Security: TLS/SSL support, rate limiting, authentication
- Advanced Features: Session persistence, circuit breakers, retry logic
- Scalability: Dynamic backend registration, service discovery
- Logging: Structured logging with levels
- Testing: Unit tests, integration tests, load tests
This is a demo project for educational purposes.
Built as a learning project to understand load balancing concepts and Go concurrency patterns.