Understanding Server-Sent Events

Server-Sent Events is a W3C standard for pushing data to browsers and mobile devices over HTTP.

Server-Sent Events (SSE) is part of the HTML5 standard. If you aren't familiar with SSE then a good primer is Stream Updates with Server-Sent Events from Google's HTML5 Rocks series. The Mozilla Developer Network has a technical explanation for SSE at Using server-sent events.

Creating and Closing an SSE Stream

The SSE standard defines an EventSource API that opens a connection to the server to begin receiving events (or data or messages) from it. The API includes specifying a handler so the client application can react whenever messages are received.

To create as SSE EventSource stream:

const stream = new EventSource("https://streaming.tenefit.cloud/stock-prices");

To close a stream:

stream.close();

Receiving Message Events

To add a message handler for new message events and do whatever you would like with the data, contained in the data field of the event:

stream.addEventListener(
  "message",
  function (event) {
    console.log("Received a message event:", event.data);
  },
  false,
);

Custom Event Types

The default event type used by SSE is message, and most messages from tenefit.cloud will be of that type, with the message data being the payload from the Kafka message.

However SSE allows custom event types. tenefit.cloud takes advantage of that to send messages of another event type: delete. These messages are for tombstone (null) messages, with the message data being the key that was deleted.

A message handler for delete event types looks the same as a regular one, but replace message with delete:

stream.addEventListener(
  "delete",
  function (event) {
    console.log("Received a delete event for key", event.data);
  },
  false,
);

Open Events

Add a message handler for when the stream connection is established:

stream.addEventListener(
  "open",
  function (event) {
    console.log("Stream is open");
  },
  false,
);

Error Events and Automatic Reconnect

If there is an error, the browser or SSE client will automatically attempt to reconnect. The reconnect is usually delayed a few seconds, although an SSE server can override the default and specify a different time.

The reconnect is transparent to your application; you do not have to write any code or take any action. However if you would like to know when a connection dropped and is trying to reconnect, you can add a handler:

stream.addEventListener(
  "error",
  function (event) {
    switch (event.target.readyState) {
      case EventSource.CONNECTING:
        console.log("Reconnecting...");
        break;
      case EventSource.CLOSED:
        console.log("Connection failed, will not reconnect");
        break;
    }
  },
  false,
);

If there is a connection failure, the client will not attempt to reconnect. This is usually because the server closed the connection in such a way as to tell the client not to reconnect. Generally, the client will always try to reconnect, though.

For example, if you are on a mobile device and you drive through a tunnel, the connection will terminate. The SSE client will continually attempt to reconnect. When you exit the tunnel, the SSE client's reconnect attempt will succeed, and data will resume streaming. This is all done automatically without the need for you to write any application code – it's defined by the SSE standard!

Moreover, the stream will continue from where it was interrupted thanks to Last-Event-Id, discussed in the next section.

Last-Event-Id

In important part of the Server-Sent Events standard is the ability for clients to automatically reconnect if the connection is interrupted. Furthermore,the data stream continues from the point it disconnected, so no messages are lost. This is all done transparently without the need for application code.

The way SSE achieves that is with Last-Event-Id. Each message from the server can include a unique Last-Event-Id. If a message does, and the connection is unexpectedly terminated, the client will attempt to reconnect. While reconnecting to the server, the client includes the most recent Last-Event-Id it received. With that information, the server knows where in the message stream to start sending messages to the client.

Thus, the client receives all of the same messages as if there was no interruption, and no messages are lost.

Being transparent to the client, you don't need to worry about Last-Event-Id in your application code. But if you'd like to know what it is for some reason, it's exposed in the client API in the lastEventId field:

stream.addEventListener(
  "message",
  function (event) {
    console.log(
      "Received a message event: [" + event.lastEventId + "]",
      event.data,
    );
  },
  false,
);

Multiple Streams

It's possible to have many streams open at the same time. Since the client connects using HTTP/2, the SSE streams are multiplexed over the same physical connection.

Here is an example with two streams open at the same time:

const googleStream = new EventSource(
  "https://streaming.tenefit.cloud/stock-prices/GOOG",
);
googleStream.addEventListener(
  "message",
  function (event) {
    // Update Google price on screen
  },
  false,
);
const appleStream = new EventSource(
  "https://streaming.tenefit.cloud/stock-prices/APPL",
);
appleStream.addEventListener(
  "message",
  function (event) {
    // Update Apple price on screen
  },
  false,
);

Note
Google Chrome browser has a built-in restriction that allows up to 256 streams per physical connection rather than honoring the maximum concurrent streams setting provided by the server during HTTP/2 handshake. In practice, that means any application which is expected to run in a Google Chrome browser should ensure it doesn’t create more than 256 concurrent open streams.

Polyfill

Some older or obstinate browsers do not have native support for Server-Sent Events. Because SSE is over HTTP, it is easy to polyfill in a Javascript application, and there are several open source polyfills available.

The following example shows how you simply need to add one extra line to your application. A modern browser will ignore the polyfill, it will only be used by browsers that don't natively support SSE:

<html>
  <head>
    <!-- Polyfill for older browsers without native support for the HTML5 EventSource API. -->
    <script src="https://cdn.polyfill.io/v2/polyfill.min.js?features=EventSource"></script>
  </head>
  <body>
    …
  </body>
</html>

Putting It All Together

See the Javascript Starter Template for the complete example that shows all of the above together.

#Creating and Closing an SSE Stream

#Receiving Message Events

#Custom Event Types

#Open Events

#Error Events and Automatic Reconnect

#Last-Event-Id

#Multiple Streams

#Polyfill

#Putting It All Together