Almost two years ago Phoenix LiveView was released. It proposed the revolutionary* streaming web applications model. It breaks from the common client side single page application model by running the application in the server instead of the client:

Streaming web applications model

*The concept is not precisely new, it is equivalent to what the good old computer terminals were doing. But everything old is new again, and thanks to the features of WebSocket and the high speed / low latency internet it is totally revolutionary. Last decade was the one of the client side frameworks and applications, this decade is going to be the one of streaming.

Elixir is Cool, But What About Node?

I have been late to the Phoenix LiveView party, and I just learnt about it a few weeks ago. The LiveView applications have to be written in Elixir and that is a big selling point to Elixir programmers. But I'm not one, so to play with streaming I could learn Elixir or I could try to implement my own in Node. A no-brainer.

SWAN: Streaming Web Applications with Node

The code involving the communication between server and client is straightforward:

Communications on the Client Side

  • A WebSoket connects to the server and reconnects after each disconnection.
  • Provides feedback to the user when connection is broken.
  • Retrieves a stream of updates from the server, usually DOM change requests, and exeutes them.
  • Listen to user events - like clicks and changes in form fields - and forwards them to the server.

An update from the server might look like this:

{
    type: "setTextContent",
    elementId: "country",
    value: "Spain"
}

And our client would execute the following:

document
    .getElementById("country")
    .textContent = "Spain";

The user events sent to the server might look like this:

{ type: "click", elementId: "deleteButton" }

In my prototype I'm automatically generating unique ids for all elements as it is probably the simpler way for client and server to reference them.

Communications on the Server Side

  • Minimal - possibly static - HTTP server to serve thin client.
  • A WebSocket server that streams updates to the clients. An update for a new client connection can create the full DOM.
  • The WebSocket server listen to user events sent by the clients. These events would potentially cause updates that would be them streamed to the clients.

So far for the communications part. Simple enough that a proof of concept can be developed in a couple of hours. Now lets take a look at the application:

Application Running on the Server

There are two conceptual approaches to running the application on the server:

In the first one the server pass the clients to the application and the application takes care to create a new view for each of the clients and send and receive updates to each of them.

In the second one the server creates an instance of Application per client and acts as a router for communication between clients and instances. The Application instances doesn't need to be aware about the client connection.

Both approaches are functionally equivalent. For my prototype I'm using the second approach, that is conceptually closer to client side SPAs.

Now we only need to create an application in some way that it knows what DOM updates needs to be requested and that is reactive to user events. That sounds quite familiar. And here it is where the advantage of using Node kicks in.

Working with Client Side Frameworks

Client side web frameworks and libraries do everything you need in your application. They know how and when to update the DOM, they know how to react to user events and most of them already work on Node. The changes needed to adapt them to streaming are absolutely minimal and should be straightforward.

For my prototype I have chosen my own toy framework, but any framework should work great.

As a side note, a side effect of running a client framework on the server is that you can ensure what JavaScript features are present unlike in the client. If you don't need JSX or you replace it by an alternative like the brilliant HTM then you might consider if Babel still sparks joy or not. And you might not need webpack anymore.

globalState and clientState

Moving the framework and application to the server creates new possibilities and new concepts. In the client a powerful concept used by many frameworks is the state. In the server the state becomes the clientState. The server adds the globalState.

The clientState is unique to each client while the globalState is shared by all clients. A change in the globalState will potentially cause updates in all clients.

There are other concepts related to state that I'm still exploring:

  • userState: a single user might run the application in multiple devices at the same time, so there is a one to many user to client connections relation.
  • groupState: it would be shared by the users that pertain to it. A group could contain other groups, becoming a tree.

*Erratum: In the video "localState.counter" should be "clientState.counter".

Other Clients / Users Awareness

Each of the different states can expose their clients, users and groups: globalState can expose all current client connections, groupState all the users in the group, userState all the clients related to the user and so on. This data can be used in the application.

Collaborative Components

With the multiple shared statuses and the other clients awareness mentioned before, collaborative applications and components emerge naturally.

This is the most revolutionary and mind-blowing property of streaming applications. Any previous simple single user application now can become multiuser and collaborative with little to no effort. This is huge. Interestingly enough, I haven't been able to find this fact mentioned anywhere, and making people aware of it is the main purpose of this post.

Just to be leave it totally clear: some components are not easy to make. For example, I was able to put together the proof of concept collaborative textarea in the example in only a few hours, but to make a robust one you would need to deal with crossbrowser issues - ouch! -, figure out how to manage undo/redo and use operational transformations - or a similar technology. A hard nut to crack.

However, given that collaboration is intrinsic to streaming, network effects are to be expected, the community and framework developers will release good basic collaborative components and they will become the building blocks for other components, also collaborative. For the application developer collaboration will be ready just out of the box.

Side Effects of SWAN

The current conversations about streaming web applications technology and running the application on the server doesn't seem to mention or recognise collaboration and give the biggest credit to some properties that I consider very important but just side effects:

  • No bundles, blazing fast first draw.
  • Libraries live and run in the server, 0 download cost for the client.
  • Reduced client-server communications increase data visualisation speed.
  • You are in node. You can call the database directly. Or execute code written in other languages. The sky is the limit.
  • By reducing the building process and avoiding transpilation, seeing changes while you are coding becomes immediate. I love this one.

So far this is all I have been experimenting with. Now lets talk about more possibilities:

Hybrid Components

In my experiments everything except for the thin client was executed server side. However it is possible to take a hybrid approach and run parts of the code in the client and some in the server.

Just a week ago React introduced Server Components. I would call their approach down-top and SWAN to be top-down, as they are adding server components to their client applications and components, and SWAN is a server application that could have hybrid client components. In the video they mention that they have plans for streaming, so I can imagine how one way or another the ends will meet.

Performance Concerns

I guess it is natural to have performance concerns about any new technology. I had my concerns until I remembered about Google Stadia. What Stadia does by streaming videogames at 4k is a lot of orders of magnitude more computationally and bandwidth expensive than the common web application. And it works great.

For the best results you will need low lag and so the server should not be too far from you. This is in fact true for all web applications, not just streaming ones.

As a side note: there is a neat trick to help hiding a couple of hundreds of miliseconds of lag that I learnt from Adobe Photoshop's Magic Wand tool. For very high resolution images, the magic wand tool can take a few hundred miliseconds to compute. However it feels immediate. What they do is to start computing the result when you press the mouse but they don't show it until you release it. Simple yet powerful, and can be used for streaming applications.

Component Ownership

Right now in my prototype for each client there is a full application instance running all its component instances. It is not too costly: it is just virtual DOM and state, but it adds up if you have big amounts of concurrent clients.

And idea I have been thinking about is the component ownership. Some components might render exactly the same for all users or a group of users, so they might be represented by a single component instance in the server's memory. And so a component could be owned by a single client, user, group or be global, owned by all.

This has very interesting implications for collaboration. Previously I have talked about collaboration via state, a shared component would allow collaboration via component. The difference is subtle, but in the first you modify data - that is the traditional approach -, in the second you are modifying the component itself. I'm still trying to wrap my head around it, right now the only difference I can imagine is that if an external actor modifies the shared component then the change will be reflected to everybody.

Hot Application Swap

Streaming should make possible the immediate hot swapping of applications to new versions. If the new version is state compatible then the server could maintain all clients state and just replace the application.

I can see how methods to properly hot swap while retaining state will appear. Right now I just have a very very light idea that would involve state blueprints.

Conclusion

SWAN is the most exciting technology in web development since the early web applications started appearing. It is not free from problems, but the advantages and possibilities are astounding. Unreasonable effective.