The Web Beyond the Edges of the Browser Window

Thank you so much Marc… Hi! I’m Niels and I just want to say up front that I am so honoured to be here today on this stage. For so many years I’ve say on these red chairs you are sitting at, lookup up at this stage, just in awe by the speakers and their talks and their creativity. Their madness and imagination. Sometimes by their obsessions that drive them.

Now, I don’t consider myself to be an artist. I build things. Out of curiosity and just for myself. Like Gavin Strange last year said: Less thinkering, more tinkering. It’s my own personal creative outlet. So, it’s a bit strange to be standing here and sharing what if built.

Last year at Beyond Tellerrand in Berlin, Astrid Bin closed the conference with a talk called Dork Patterns. She showed a list of dorky subjects that she obsessed about and I love that. And want to start a little tradition. I too get obsessed sometimes. And I really think that is a healthy thing. For me an obsession is often a period of focus and ideas and sometimes unexpected, and sometimes slightly unhinged results. It’s healthy to care about something – even though you may be the only one that cares. I call them obsessions, but it’s often the source of creativity. You can also call it inspiration.

I get obse…inspired by web standards, but also northern lights, lego, clocks, user agent strings, astro-photography, oscilloscopes, receipt printers, reverse engineering bluetooth devices…

Video’s about watch repair, evolution of germanic languages, DMX controlled stage lights and pyrotechnics, browser compatibility before 2000, the space shuttle challenger accident report, how the browser parses HTML, history of writing

Typography and getting annoyed that proper use of the EM dash now makes you look like an AI bot, making remote controlled lego cars, how barcodes and QR codes are encoded… and so much more

But mostly about the web and how I can use the web beyond the edges of the browser window. I fell in love with the web back in 1994 and that has pretty much been a constant in everything I create. But lately my obsession has been…

…clocks. And I am going to share a bit about that obsession and some other projects I created in the last years or two. The thing that connects all of these projects together? This conference. All of these projects are the direct result of talks I’ve seen here. Speakers that stood on this stage. Or they were programmed in the train on the way to Beyond Tellerrand. Sometimes even started right in the middle of a talk. Not that the talk was boring, but it somehow something in that talk triggered me and I just had to put that into code.

But my fascination with clocks really started with Vasilis van Gemert – who stood on this stage a couple of years ago, and his not so secret passion is clocks. And his gradient clocks really inspired me and dropped me head-first in a months long obsession with Raspberry Pis and touch screens. There was just one thing. For me personally a clock should be round. And I know the Rheinturm here in Dusseldorf very clearly shows a clock does not need to be round, but for me when I think of a clock it is always round. Btw, I love the CSS version of the Rheinturm that Gunnar Bittersman made – ask him to show it to you, he’s around here somewhere. So I went on a quest to find a round touch screen.

Turns out that is not the difficult part. Meet project Bad Clock.

This project consists of several components – first that round screen that is just perfect for a clock. Connected to it is a Raspberry Pi 5 – which means lots of computing power – which we are going to need. I had plans. Its runs Chrome. And everything is just a webpage. And of course some custom hardware, such as rotary encoder which you can use to wind the clock – something that a clock definitely needs

Meet our bad clock. It’s a perfect round screen. And thanks to an accelerometer it always stays upright. You can change the time by tapping these segments to turn them on and off until it forms that new number. It does have some issues though. The screen flickers sometimes. Just like there is some bad solder joint. And just randomly numbers can stop working. And sometimes it forgets to reset the clock at midnight, so it’ll keep counting 24, 25, 26, 27, 28…

One other issue is that our clock also does understand wintertime and summertime. Sometimes it will simply forget. Or it does it at the wrong moment. Not every country changes dst at the same moment. Or it will change it the wrong way. So instead of going forward, it’ll change the time backwards.

And sometimes the clock pretends to be a VCR… Nobody knows how to set the time on a VCR… So it just blinks 12:00. Apologies… I’ve turned 50 recently… A VCR is basically a DVD player that uses magnetic tape… Apologies again… A DVD is basically Netflix but you have to go to a physical store to borrow a shiny disk. Anyway…

The clock also contains a full Box2D simulation and everything you see on screen are in these simulated environments that have motors connected to the hands. And these environments also have gravity. And thanks to an accelerometer I can set that gravity in the same direction as real gravity is…. And you can set the time…

… by just touching the hands and moving them. And it determines the force you use and sometimes that is too much and the hands break off from the pivot point and start flying. Actually once in a while one of the hands will come loose on its own and drop down swinging on the pivot point. And if you don’t put it back within an hour or so it will just fall to the floor. Now, I put this on the internet… and got some great reactions… and also this…

Yeah… I mean…

I’ve got work to do. It doesn’t make any sense. Why would you? I still have this talk to write. I don’t have the time. I can’t possibly spend 10 hours and 17 minutes on a gravity simulation of beans… And I know that exact time, because…

I mean, I just had to do it… Right? I added 120 beans to the gravity model added a liquid simulation as well, including a method where beans and liquid would leave brown smudges on the glass whenever the beans are touching the glass.

It looks like this. Every bean is a circle with the bean oval randomly sized. That way the beans can overlap a bit and the distribution looks uneven – which is perfect. The hands can interact with the beans and you can even stir the beans and pick up individual beans. The clock now has a bean mode. And I don’t care about how ridiculous it is. It is my clock. I can do what I want.

And added a menu as well. Also in the gravity simulation. The buttons drop down from the top. Not very user friendly because the buttons are always in a different place and maybe even in a different order and even sometimes up-side down. But having that gravity makes it so tactile. It is so fun to play with this.

And finally this knob. Well…. You need to wind the clock. Every day you have to wind it up otherwise it will start running slower and slower to eventually just stop. But the tricky part is, you do not know how much to wind it. And if you wind it up to much…

… the forces on those gears just build up and up and they start slipping and the teeth just break off. This has probably been the most fun project I did in the last 10 years. I never understood the attraction people had to tamagotchi, but I think I get it now. And it was kind of an accident, that I got sucked into… because what I really wanted to do is something completely different.

For year I wanted to make a laser clock. Even since I seen a talk by Seb Lee Delisle talk about lasers 10 years ago I’ve been fascinated by laser projectors. But I knew nothing of how lasers work and if I could connect them to the web – Seb did explain some of the issues he ran into recreating the classic 1979 Asteroids game.

Now the original game used a special display system called the QuadraScan, which used a CRT, but instead of pixels it directly steered the electron beam to draw these vector shapes.

Now, before you get excited about me showing a laser clock… They are expensive. It’s not like my name is Seb Lee Delisle who has probably a couple of spare ones just stacked up behind couch.

But… that game of Asteroids kept lingering in the back of my head and I figured that an oscilloscope could function in the same way as that QuadraScan display.

And they are really cool too. I just love the aesthetic – it really feels you have wandered into the lab of a mad scientist with all of those buttons. And if you think of it. It literally is a small particle accelerator that fires electrons at about a quarter of the speed of light. And it is pointed right at your face. Only to be caught by a small layer of phosphor.

But of course we want to use web technology to create this clock. So this part of the talk is titled: “How I used CSS Animations to draw a clock on an oscilloscope”

Eh… wait… Actually, that is not really accurate.

How I used WebAudio to blow up an oscilloscope and almost caused a fire…. That would be more appropriate.

Now before I tell that story… I want to take a minute and let’s think about what a clock really is… I don’t mean philosophically…

It is a circle and three lines. And on the web we can use SVG for that. Four simple shapes… Vectors… just like that Asteroid game.

Now we need to draw these shapes on the scope. The face-melting electron beam is steered - or better deflected - by two electrically charged plates, called the x- and Y-plates. And by deflecting the beam vertically and horizontally we can trace the image that we want. And do that multiple times per second and our image appears.

So we need to generate two signals, one for X and one for Y and it needs to trace the image we want to draw. Let’s start with the clock face, the circle. This is the circle and to create this you need some high school math. Turns out my math professor was right all those years ago – I was going to need math later in life.

So this is the circle and if you look closely it actually starts to make sense. We just make a wave form directly from the X/Y coordinates of the circumference of the circle.

And we can make any shape with that method. Every shape can be expressed in that two channel signal.

Even triangles…. Which are three lines… and we need three lines for our clock. Just oriented slightly different.

So this is what our clock looks like. If you look closely at the red signal, we have a sine wave, that the circle, and three triangles, those are the clock hands that are drawn from the centre out and then back to the centre. That is one triangle. Then another back and forth for the second hand. And another one for the third. But how do we get those coordinates?

We start with our SVG, and we can use an API to get those coordinates along the length of each path. And we end up with two arrays with numbers. And if we plot those numbers in a graph, you can see the sinus wave again, and the three triangles.

And we do this 30 times per second… We transform the hands with CSS animations, so we get different wave every time we sample the shape and capture every frame of the animation. CSS animations on a oscilloscope.

So I built a web app that does exactly this. It has a small editor which you can use to edit your SVG and CSS and it will just inject that into the DOM and sample the geometry 30 times per second. And it outputs the waveforms using WebAudio to the oscilloscope.

We’ve connected the computer’s audio output to the X and Y channel of the scope

Let see how this looks on the scope… and now let’s set it to XY mode. And there it is…

And at this point the oscilloscope decided to… explode. Now this is just the aftermath… I was too busy finding the power plug, scrambling to find my phone to film it and at the same time a big column of smoke was rising up and sparks went flying everywhere. And then I realised — I should probably unplug my computer too. What if this thing sends a couple thousand volts back down the audio cable?

I was fine. Did not get electrocuted. My computer survived. But the scope was dead. Now what…

Redundancy!… I may have overreacted… But I am getting ahead of myself. My scope has exploded. I have a signal generator to finish, but no way to actually continue until I get my scope repaired or find a replacement.

And I find myself on the train to Beyond Tellerrand in Berlin last November. It’s a five hour ride and I have nothing to do… So I decide to built my own oscilloscope simulator.

But what I really want a 1980s oscilloscope with all of its faults and limitations. We also want to replicate how the phosphor works in a real scope. And what we really want is a physics simulation of the electron beam and how it is deflected by the electrostatic x and y plates. But I have to do it from memory. I only had the scope working for an hour or so when it exploded.

And this is where things go off the rails. I did a deep dive into how scopes work. Electromagnetic Force, Acceleration, Velocity, Damping, Euler integration of how velocity moves the beam. Euler integrations. Overshoot amplitude decay… And we haven’t even talked about Energy deposition based on beam velocity, The P31 phosphor physics, like phosphor saturation, phosphor persistence, Beam dwelling at direction changes causing brighter dots on the screen.

And I was about to calculate how the electrons are exciting the phosphor and then I realised… that I have a life.

I have a family and work and so many other projects. I actually don’t care about this. Apparently there was an end to the rabbit hole for me. It turns out I only care about how it looks. Does it look the same as a real scope? Yes. So great… And with the simulator in place, it did allow me to create some other generators. Fully created and tested on the simulator… and when my scope got repaired, they just worked.

And sometimes during these obsessions you just get sucked into another one. When working on DOOM on the scope I started thinking… I have this data… Could I use this to create DOOM in CSS? The answer is yes! Every wall, every floor, every barrel, every enemy is a div. Rotated into 3D space using CSS 3D transforms. You can play it on cssdoom.wtf… even works on you phone… not now… la-ter…

But not everything is about clocks…

I also created a CSS flamethrower.

On the train to Beyond Tellerrand I also saw this on AliExpress. It’s really cheap. AliExpress, cheap and flamethrower. Not great to hear those words in the same sentence…. But there was this button. And I bought it… I figured I had weeks before it arrived and plenty of time to prepare my wife. And when I arrived home, it was there. On my coffee table. I had some explaining to do. The project was actually something that I had on my shelf for at least a year or two. I wanted to control big stage lights using CSS.

And to do that we need a protocol called DMX… not this DMX…

This one.

And it is used by stage lights, smoke machines… and also flame throwers.

DMX controls these devices by channels. These are bytes in a big array of 512 values. Each device uses one or more channels. A light typically 3, one for red, green and blue. Sometimes more, for example for white or amber. Or pan and tilt for motorised lights. Each light has a starting channel that determines which part of that 512 channel array belongs to them. And we send that array 40 times per second to all devices. And the device just listens for changes to their channels.

So if we change one byte in that array… A light turns on.

When I envisioned this project about two years ago, I had the idea that I wanted CSS to run everything. The whole goal was to have CSS animations running on the lights.

I did built a whole ui that you can use to set up the devices, so every device will get an id in css, and you can set a color, just by editing CSS and setting the color property.

And I added a animation editor but everything you do…

Is just CSS. Each device type has a number of custom properties that you can use to control it. For lights you can use target the individual LED’s or just use the color property.

But everything runs as CSS. Even smoke machines.

But we also need inputs to trigger animations and scenes. I was absolutely amazed by Adam Argyle’s talk in Berlin last year about combining CSS with WebMIDI. The talk was not even finished before I started added WebMIDI support to this project as well.

So you can use a MIDI controller to trigger the animations. A slider becomes available as a custom property with a value from 0 to 100%.

And knobs too. And pressing a button sets a class on a container, which you can use to change values or trigger that animation.

But I added much more. Like gyroscopes that change pan and tilt of lights based on your gestures, or gamepads buttons and joysticks. And even WebAudio to change the lights based on the music that is playing. It really feels like this projects bring all Web APIs together puts it into CSS and let CSS do what it does best – even though the output is not in the browser itself.

A CSS controlled flamethrower… and this isn’t just theory. I just can’t show it here on stage.

And eventually it even turned into a controller for laser projectors… Yes. I bought a laser projector, and yes, you can connect it to the web thanks to WebUSB and you can draw using SVG and CSS. So let’s take a look!

Thank you Marc for 15 years of Beyond Tellerrand and 15 years of inspiration and creativity. Thank you for the obsessions and ideas that I got here. Thank you to all the speakers that gone before me and inspired me to no end. And thank you for making me feel like family for all of those years. Thank you Marc! Thank you.