SSE, proxy buffering, and long-polling fallbacks

The problem

Server-Sent Events are appealing because they keep a plain HTTP response open and stream events as text. The hard part is that every layer between the app and the browser has to preserve streaming behavior.

If a proxy buffers the response, the application may write events correctly while the client receives them late or in a burst. From the user's point of view, the feature is broken even though the server-side code looks reasonable.

That is why SSE work should start with infrastructure behavior, not only controller code. Streaming is an end-to-end property.

What SSE needs

An SSE response is a long-lived HTTP response with the text/event-stream content type. The server writes small chunks such as data: lines, and the client processes them as they arrive.

For that to work, the app must flush data, the web server must not wait for a complete response, the platform router must allow the connection to remain open, and any proxy or CDN in front must avoid buffering the stream.

The application also needs heartbeat messages. Without periodic bytes, idle timeouts can close connections that are healthy but quiet.

Where buffering enters

Buffering can appear in reverse proxies, app servers, middleware, compression layers, CDN configuration, and platform routers. Compression is a common surprise: a gzip layer may wait for enough data before emitting a compressed block, which defeats small event delivery.

Heroku adds another practical constraint: requests that do not send data within the platform's timeout rules are at risk, and long-running connections still consume dyno resources. Even when SSE works, it should be treated as a scarce connection, not a free background channel.

The test has to be real. A local curl test against Rails does not prove that the production route through proxies and routers streams correctly.

Long-polling fallback

Long polling is less elegant but often more tolerant. The client asks for updates since a known cursor. The server waits briefly. If something changes, it returns data. If nothing changes, it returns an empty response and the client asks again.

This model works through ordinary request-response infrastructure. It is easier to cache-bust, retry, inspect, and scale with common web tooling. The cost is more requests and slightly less immediate delivery.

For many product features, that trade-off is acceptable. Notifications, background job progress, and status refreshes often need reliable freshness more than perfect streaming purity.

The state model matters

The fallback only works if updates are modeled as state, not as unrepeatable whispers. Each client needs a cursor, version, timestamp, or event id. The server should be able to answer "what changed since this point?"

That state model also helps SSE. If a stream drops, the client can reconnect and resume. Without a replayable boundary, any network interruption becomes data loss.

The pragmatic design is to make streaming an optimization over a durable state model. Then SSE, long polling, and manual refresh all read the same underlying truth.

A Rails fallback shape

A durable implementation starts with a replayable event model. The browser can stream when SSE works, but it can also poll using the same cursor when a proxy, timeout, or browser condition makes streaming unreliable.

class EventsController < ApplicationController
  include ActionController::Live

  def stream
    response.headers["Content-Type"] = "text/event-stream"
    cursor = params[:after].to_i

    EventFeed.each_after(cursor) do |event|
      response.stream.write("id: #{event.id}\n")
      response.stream.write("data: #{event.payload.to_json}\n\n")
    end
  ensure
    response.stream.close
  end

  def poll
    events = EventFeed.after(params[:after].to_i).limit(100)
    render json: { events: events, next_cursor: events.last&.id }
  end
end

The important detail is that both endpoints share the same event feed. SSE is only the delivery optimization. The correctness property is that every client can ask for changes after a known point and receive a bounded, repeatable answer.

Production checks

A production check should prove streaming through the same path the browser will use. Test with the CDN, router, proxy, app server, compression settings, and timeout behavior in place. A local controller test only proves that the action can write chunks, not that chunks arrive at the client when deployed.

Useful checks are simple: send a heartbeat every few seconds, watch timestamps at the client, confirm compression is not coalescing events, and disconnect the client while the server is writing. The fallback poll endpoint should be tested with the same cursor values so reconnects and downgrades exercise one state model.

The same test should record what the application wrote and what the browser observed. If those timestamps diverge, the bug is in the delivery path, not necessarily in the controller. That distinction prevents a lot of wasted Rails-side debugging.

If the production path cannot stream reliably, the product can still be correct by preferring polling and reserving SSE for environments where it has been proven end to end. That is a product decision backed by deployed evidence, not a retreat from the streaming design.