One of my previous posts about communcation via a Telegram Bot reminded me that Structr also has support server-sent events using the EventSource interface. This enables us to push live updates to users with a few lines of JavaScript. In this post we will extend the example application from the previous post and create a small example with an auto-updating progress-bar using this pattern. With a little imagination we can adapt this basic example to more advanced scenarios. Let's get started...

What are server-sent events?

Server-sent events are a way to transport information to the client from the server directly, rather than having the client have to request data constantly (polling). This is done using the EventSource interface, which opens a persistent connection to an HTTP server, which in turn sends events in text/event-stream format. The connection remains open until closed by calling EventSource.close().

Almost all modern browsers support EventSource which makes it a perfect candiate to replace polling.

Current limitation

Currently Structr relies heavily on WebSocket. The Jetty webserver Structr ships with only recently released version 10 where HTTP/2 and WebSocket can be used together. Until Structr upgrades its dependency to Jetty 10, only 6 connections per user are possible. Opening 6 tabs would lead to the last tab not loading properly until one persistent connection is closed.

In practice this is not much of a hurdle though. First, don't overuse EventSource, a little bit goes a long way. Second, you can close the EventSource when it is not needed anymore. Third, the upgrade to Jetty 10 is also on the near-future roadmap for Structr so this will not be a problem soon.

Structr-specific functionality

To use use server-sent events in Structr we first need to enable the EventSourceServlet in the configuration tool in the key httpservice.servlets and save the configuration. If the servlet was not enabled before, we now need to either restart Structr or restart the HttpService in the Services tab in the configuration tool.

Once that servlet is active, we (administrators) can make connections to /structr/EventSource using the following client-side JavaScript code.

let source = new EventSource("/structr/EventSource", { withCredentials: true });

source.addEventListener('name-of-our-event', function(event) {
    // Listen to the event stream for the event "name-of-our-event"
    // this can be any name
    console.log(event.data);
});

To enable users to make such a connection, we also need to create a resource access grant with the signature _eventSource and allow the HTTP GET method for authenticated and/or public users.

From any scripting environment on the server we can now send events to all connected clients. We can either broadcast messages to all connected clients using the broadcast_event() function (documentation) or we can send messages to a specific user or group or collection of users/groups using the send_event() function (documentation).

Example usage

In our example we connect to the EventSource servlet and listen for events telling us about the import progress or the progress of a cleanup method. Updates are in both cases visualized using a progress bar.

On the server we know the current step of the process and can calculate the percentage of completion and send that information to the client. For simplicity we are using the broadcast_event() function but we could use the send_event() function as well and only send update events to the client which triggered the process.

Note: I did not really change the importMovieData method from the previous example project to make the progress bars behave more "natural". I simply added broadcast_event() function calls in there to update the users. In a more real-world scenario imports of bigger, more interconnected datasets would possibly first import the separate node types and then connect the nodes in a separate step, allowing for nicer progress updates, maybe even with multiple progress bars, one for every step.

I also added a deleteMovieData method so we have a second example and can observe the behavior multiple times. In this method the progress is not calculated depending on the number of nodes. It is simply divided into three discrete steps, each set to be 33%.

HTML & JavaScript

The markup is probably the least interesting part. We are reusing the previous TailwindCSS example application and add a new /data page where we can import and delete data. A simple button to start the import (or delete) process which executes a global schema method. Paired with a progress-bar (based on this article) which is basically an empty container which will be set to a percentage of the full width after every update.

The screenshots are both taken from the IMPORT_ACTION template underneath the /data page and control the import process. Very similar code can be found in the DELETE_ACTION template for the delete process.

The JavaScript event listener parses the JSON encoded event data and simply applies a CSS style to the progress bar which sets the width in percent. The message contained in the event data is displayed in the status container.

The Final Product

Having all the pieces together and clicking the "Start Import" button, we should see something like the following video.

If your progress bar scrolls by way too fast (because your server/computer is a lot faster than my docker setup), there is a $.sleep(100); statement in the schema method importMovieData which can be used to illustrate the effect better.

Example Application

As usual, I created a repository for the example application for you to play along and experiment yourself without any of the friction of the initial setup.

Important notes:

• This has been created with the most recent 4.0-SNAPSHOT version of structr and uses pretty recent functionality. Be sure to import it with at least this version, otherwise it will not work.
• The deployment process will overwrite your current application (if present), so be sure to do this after creating a backup or in a dedicated instance. Creating a throwaway instance for this is easiest via Docker - we have an excellent example Docker setup in this repository