Deploying Real-Time WebSockets & Voice AI: Nginx, Daphne/Uvicorn, and Persistent Connections

The Shift from Ephemeral Requests to Persistent Streams

Traditional web applications operate on short-lived request-response lifecycles: a client issues an HTTP request, the WSGI worker processes the query within two hundred milliseconds, returns a response, and terminates the socket connection. In contrast, real-time WebSockets and conversational Voice AI agents (such as LiveKit WebRTC or OpenAI Realtime audio streams) maintain persistent, bidirectional connections that remain open for minutes or hours.

Deploying persistent streaming systems at scale introduces unique operational challenges: standard Nginx timeouts terminate in-flight calls, traditional WSGI workers cannot handle concurrent socket streaming, and operating system file descriptor limits can silently exhaust. Solving these issues requires mastering hybrid WSGI/ASGI architectures, Nginx WebSocket protocol upgrades, and Linux kernel socket tuning.

1. Hybrid WSGI and ASGI Process Separation

Attempting to handle persistent WebSockets on standard synchronous Gunicorn workers quickly locks up your web server. When three synchronous workers are tied up serving three persistent audio streams, all subsequent HTTP web requests queue indefinitely. Resilient architectures decouple traffic into dedicated process tiers:

• WSGI Tier (Gunicorn): Dedicated to rendering standard HTML pages, executing administrative actions, and serving standard REST endpoints.
• ASGI Tier (Uvicorn / Daphne): Dedicated exclusively to asynchronous WebSocket connections, audio buffer streaming, and WebRTC signaling.

2. Nginx WebSocket Protocol Upgrades & Buffer Bypass

Nginx requires explicit configuration directives to convert standard HTTP connections into bidirectional WebSocket streams and prevent intermediate buffers from corrupting real-time audio playback:

# Upstream definition for asynchronous ASGI cluster
upstream websocket_backend {
    server 127.0.0.1:8001;
    keepalive 32;
}

server {
    location /ws/ {
        proxy_pass http://websocket_backend;
        proxy_http_version 1.1;
        
        # Mandatory protocol upgrade headers
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # Prevent Nginx from terminating active voice calls on silence
        proxy_read_timeout 86400s;
        proxy_send_timeout 86400s;
        
        # Bypass proxy buffering for zero-latency audio streaming
        proxy_buffering off;
    }
}

"Disabling proxy_buffering is mandatory for real-time Voice AI. If buffering remains active, Nginx accumulates small audio frames until a buffer fills, destroying fluid human turn-taking with erratic latency jitter."

3. Linux Kernel File Descriptor & Socket Sizing

In Linux, every open network socket is treated as a file descriptor. Default system limits often restrict processes to 1,024 open descriptors, meaning your server will crash with Too many open files once concurrency spikes. Increase system limits in /etc/security/limits.conf and tune kernel TCP socket recycling in /etc/sysctl.conf:

# /etc/sysctl.conf: Tune network socket capacity
fs.file-max = 2097152
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

Key Takeaway

Real-time Voice AI and WebSockets require purpose-built infrastructure designed for persistence rather than rapid turnover. By decoupling ASGI streaming workers from WSGI processes, tuning Nginx buffer bypass rules, and sizing kernel socket limits, engineering teams deliver sub-second voice interactions that scale reliably.