# data[].attributes.blurb.html
This session explores hardware APIs like WebHID, WebUSB, and WebSerial, connecting browsers to cash drawers, customer displays, receipt printers, and barcode scanners. It touches on Project Fugu APIs like Idle Detection, Multi-screen Window Placement, and Shape Detection. A live demonstration showcases connecting hardware devices to web apps, based on a demo featured at Google I/O Connect 2023.

# data[].attributes.slidedeck.data[].slides[].desc.html
Making money with Project Fugu building a cash register with web technologies

# data[].attributes.slidedeck.data[].slides[].desc.html
Hi, I am Niels Leenheer. Thank you being here. It’s amazing to be back in Warsaw. I haven’t been here since, I think 2017 and this city has changed so much in the last 8 years. I just love to be back.   I am a Google Developer Expert for the web and…

# data[].attributes.slidedeck.data[].slides[].desc.html
Anyway. I am the co-founder and CTO of Salonhub.

# data[].attributes.slidedeck.data[].slides[].desc.html
And we make software for hair salons. You know the whole point of sale system and appointment booking system. Everything is based on web technology. Deployed on desktop using Electron, or on tablets using Capacitor, or just a plain Progressive Web App. Just look how happy these people are using our PWA. PWA’s are awesome.

# data[].attributes.slidedeck.data[].slides[].desc.html
So yeah… I don’t often talk about my job. What I usually talk about is WebBluetooth and programming Lego cars with JavaScript. You know, fun stu . And nobody wants to hear a 30 minute talk about the exiting world of hair styling. But today is an exception. No fun stu today. Just me venting about my day job.

# data[].attributes.slidedeck.data[].slides[].desc.html
Earlier this year I got this email from Thomas at Google. He’s part of the Chrome team and focuses on Project Fugu, a multi-vendor effort to make the web more capable. To add new API’s that allow the web to compete with native applications.

# data[].attributes.slidedeck.data[].slides[].desc.html
Our PWA uses quite a few API’s that are part of Project Fugu, including the API’s responsive for integrating with hardware devices, like receipt printers, customer displays and cash drawers. A couple of years ago that was simply not possible. But it is getting closer and closer. Thanks to Project Fugu and

# data[].attributes.slidedeck.data[].slides[].desc.html
The various hardware API’s like WebUSB, WebSerial, WebHID and more. And his question was… could we make a demo version of our product that shows off these API’s and… oh…

# data[].attributes.slidedeck.data[].slides[].desc.html
Could we do it in time for Google I/O Connect in Miami. In four weeks…

# data[].attributes.slidedeck.data[].slides[].desc.html
So… two weeks later…

# data[].attributes.slidedeck.data[].slides[].desc.html
We had a fully functioning demo that was completely browser based. Of course it wasn’t two weeks work. It was years in the making. In fact I started working on the receipt printer code for my HalfStack talk about WebBluetooth back in 2017.

# data[].attributes.slidedeck.data[].slides[].desc.html
We could scan barcodes to add products. Items are shown on the customer pole. The receipt is visible on the display. And when we pay we can print a paper receipt.

# data[].attributes.slidedeck.data[].slides[].desc.html
So we went to Miami.

# data[].attributes.slidedeck.data[].slides[].desc.html
After that we went to Amsterdam.

# data[].attributes.slidedeck.data[].slides[].desc.html
Thomas took our demo to Bengalore and in September it was shown in I/O Connect Shanghai. All and all an amazing experience.

# data[].attributes.slidedeck.data[].slides[].desc.html
So, who wants to see it? And afterwards I’ll tell you how we created it.

# data[].attributes.slidedeck.data[].slides[].desc.html
So, yeah. That was a little bit underwhelming maybe. It is just a cash register. You scan a product, you pay and you’ll get a receipt. You can see that every day, in pretty much every shop or store. But it is more complicated than you might think. So how did we build it? Like I said it was years in the making. I’ve been playing with this in some way or form for the last 10 years.

# data[].attributes.slidedeck.data[].slides[].desc.html
Receipt printer Let’s start with receipt printers. This is an essential part of a cash register. Nowadays most are USB, but they still sell serial or even parallel port versions.

# data[].attributes.slidedeck.data[].slides[].desc.html
But the one we’re using is a USB device. So we’re going to use WebUSB. The API allows us to request a device with specific filters. The user then gets a list of all devices that they can choose from. And the browser then gets access to the device that is chosen.

# data[].attributes.slidedeck.data[].slides[].desc.html
Then we have to open the device. Claim the interface and transfer out our data to the endpoint.

# data[].attributes.slidedeck.data[].slides[].desc.html
Yeah, this may require some extra explanation.

# data[].attributes.slidedeck.data[].slides[].desc.html
USB devices have one or more interfaces. This usually depends on functionality.

# data[].attributes.slidedeck.data[].slides[].desc.html
Most have one interface, but if you have a device that combines multiple functions, like a scanner and printer combination, you would usually have one interface for the printer and one for the scanner.

# data[].attributes.slidedeck.data[].slides[].desc.html
And each interface has one or more endpoints. Endpoints can be use to send data, or receive data. So in case of a printer you have two endpoints, one for sending what you want to print and one for receiving for example status information about how much paper you have.

# data[].attributes.slidedeck.data[].slides[].desc.html
So we claim the right interface. And we claim it, so we can use the device exclusively and there are no other applications that can interfere while we use it. And then we send the data using the proper endpoint.

# data[].attributes.slidedeck.data[].slides[].desc.html
And now we can talk to our printer, but we still have no idea about what to tell it. We need to learn their language first.

# data[].attributes.slidedeck.data[].slides[].desc.html
Let’s start really simple. Say we want to print this. Just two lines. One line in bold text. And then cut the receipt. If we were to do this with the web platform, this would be really, really simple.

# data[].attributes.slidedeck.data[].slides[].desc.html
Just npm install react. And tailwind. Obviously. Then have a good think about what build system you want to use. Add some div’s to the root and apply a whole bunch of css classes. Simple….

# data[].attributes.slidedeck.data[].slides[].desc.html
Okay, okay. Let’s use some proper HTML. And I am keeping it simple here. Just a couple of <br> and <b> tags. And that is it. But that is not how printers do it. The language printers use long predate HTML.

# data[].attributes.slidedeck.data[].slides[].desc.html
Let’s travel back to 1990. Epson is launching their rst receipt printer. Nowadays all receipt printers use thermal paper, but back then it used dot matrix technology. And Epson was famous for creating the MX-80 dot matrix printer back in 1980.

So they took a shortcut and basically used the ESC/P language from that dot matrix printer and adapted it a bit and called it ESC/POS. And that is what receipt printers have been using ever since.

# data[].attributes.slidedeck.data[].slides[].desc.html
And the language is basically 7 bit ASCII. And with ASCII we have all the printable characters you would ever need. 26 of them. A to Z. And you get numbers. Some special characters. But that is not all, you also have lowercase version of A to Z… Whoooowww. What more do you need… It is the American Standards Code for Information Interchange after all.

# data[].attributes.slidedeck.data[].slides[].desc.html
So we can just send this to the printer and it will work. But this is not entirely what we wanted. It’s not bold…

# data[].attributes.slidedeck.data[].slides[].desc.html
So let’s fix the bold part. We’re not using tags, like HTML, but we send a special escape character that signals to the printer that what follows is a command.

# data[].attributes.slidedeck.data[].slides[].desc.html
ESC E-for-emphasis. And 1 to turn emphasis on. That makes the text bold. And ESC E 0 is to turn it off.

# data[].attributes.slidedeck.data[].slides[].desc.html
And there are many commands for all kinds of formatting and commands for inserting things like barcodes and images. Too many to list here and too many to go into details. There is a manual.

# data[].attributes.slidedeck.data[].slides[].desc.html
We need to send a command for cutting the paper. GS V 0. But we’re not done yet.

# data[].attributes.slidedeck.data[].slides[].desc.html
First we need to initialise our printer. We forgot to do that. And by initialising we make sure the printer is back to the default state and we don’t have the styles of the previous user of the printer interfering with what we want to do.

# data[].attributes.slidedeck.data[].slides[].desc.html
But what about these letters? We don’t have Unicode. Just ASCII… Which is ne of the US, but really not for the rest of the world. Even a simple character like the Euro is not part of ASCII…

# data[].attributes.slidedeck.data[].slides[].desc.html
And even in the US they have IKEA…

# data[].attributes.slidedeck.data[].slides[].desc.html
So let’s go on a tangent and talk about internationalisation. In fact, this could a whole talk in itself. Internationalisation is so complex that we’re not even going to scratch the surface. But that is okay. Because this is not proper internationalisation, it is… Internationalisation before Unicode.

# data[].attributes.slidedeck.data[].slides[].desc.html
In other words… How can we sell computers to the rest of the world with the least amount of effort.

# data[].attributes.slidedeck.data[].slides[].desc.html
So welcome to…. Internationalisation before Unicode.

# data[].attributes.slidedeck.data[].slides[].desc.html
So ASCII is just 7 bits. That means just 128 different characters and we still have another 128 left over.

# data[].attributes.slidedeck.data[].slides[].desc.html
And for that we have codepages. We can swap out a different set of 128 characters that are in the upper range of an 8 bit byte.

# data[].attributes.slidedeck.data[].slides[].desc.html
It can be Western European, which contains the characters we want. Which is by the way exactly the reason why you should never test your software with that particular string.

# data[].attributes.slidedeck.data[].slides[].desc.html
Or Hebrew…

# data[].attributes.slidedeck.data[].slides[].desc.html
Or Greek…

# data[].attributes.slidedeck.data[].slides[].desc.html
Ukrainian, which has its own codepage, which is di erent from Russian, because Ukrainian uses number of different characters.

# data[].attributes.slidedeck.data[].slides[].desc.html
Or even Thai…

# data[].attributes.slidedeck.data[].slides[].desc.html
We have so many codepage. So….

# data[].attributes.slidedeck.data[].slides[].desc.html
Many codepages.

# data[].attributes.slidedeck.data[].slides[].desc.html
But we want codepage 858, which contains all the letters that have here.

# data[].attributes.slidedeck.data[].slides[].desc.html
So we start with a unicode string and then we lookup the byte value for each character in the codepage table. Luckily there is a library to do that. But unfortunately the codepages used by printers can be pretty obscure, so it only supports a fraction.

# data[].attributes.slidedeck.data[].slides[].desc.html
So I created a library called CodepageEncoder, specifically for receipt printers. And it even allows us to things like automatically choosing the right codepage and for the characters you want to print and switching on the fly when needed.

# data[].attributes.slidedeck.data[].slides[].desc.html
Of course, not every printer supports every codepage.

# data[].attributes.slidedeck.data[].slides[].desc.html
Some support even less.

# data[].attributes.slidedeck.data[].slides[].desc.html
Some support more, but use a different identifier than what Epson does. And Epson created ESC/POS, so why…. Why!? It is really annoying.

# data[].attributes.slidedeck.data[].slides[].desc.html
But it is not just cheap Chinese printers that do that…

# data[].attributes.slidedeck.data[].slides[].desc.html
But wait… what about cash drawers. As you can see with this model there is an integrated cash drawer. How do we open the cash drawer?

# data[].attributes.slidedeck.data[].slides[].desc.html
Well, all receipt printers have a DK port. You connect your cash drawer to that port and then you can tell the printer to send a pulse to that port and that activates the solenoid in the cash drawer and it opens. And with the combined printer cash drawer it works in the same way. So the cash drawer is not really a different device, it is a function of the receipt printer.

# data[].attributes.slidedeck.data[].slides[].desc.html
When I started this journey I really wished there was somebody who would that created a javascript library to make my life easier. And there were libraries, but mostly server side, badly written, php or python. Or JavaScript, but for Node. Not for the browser. So I created one.

# data[].attributes.slidedeck.data[].slides[].desc.html
And to get the receipt that we want we just need to do this. This is much easier and you can forget everything I just mentioned.

# data[].attributes.slidedeck.data[].slides[].desc.html
But wait… it can’t be that simple.

# data[].attributes.slidedeck.data[].slides[].desc.html
No of course not. I mentioned all printers support ESC/POS, but yeah… I lied.

# data[].attributes.slidedeck.data[].slides[].desc.html
Introducing StarPRINT. Exactly the same as ESC/POS, except that everything is different. Great!

# data[].attributes.slidedeck.data[].slides[].desc.html
Instead ESC @ we now need to initialise…

# data[].attributes.slidedeck.data[].slides[].desc.html
with ESC @ CAN… because…. We can?

# data[].attributes.slidedeck.data[].slides[].desc.html
ESC t turns into ESC GS t….

# data[].attributes.slidedeck.data[].slides[].desc.html
And instead of ESC Emphasis On and O . You now just have ESC Emphasis and ESC F for ehh…. F this.

# data[].attributes.slidedeck.data[].slides[].desc.html
So if you have a Star printer, EscPosEncoder is completely useless. You need something specifically for StarPRNT.

# data[].attributes.slidedeck.data[].slides[].desc.html
So I created a library called StarPrntEncoder….

# data[].attributes.slidedeck.data[].slides[].desc.html
But don’t use it… I’ll get back to that.

# data[].attributes.slidedeck.data[].slides[].desc.html
There were several points during this almost 5 year long journey that I just wanted to give up. There are no standards. And I haven’t even talked about di erent methods of image encoding. Barcode support that di ers between even models from the same manufacturer. Printers that use 203 dpi, or 180 dpi. How many characters can t on one line? Yes that can be 44, or 42. Or 35. Or 32. It depends. It’s a big mess.

# data[].attributes.slidedeck.data[].slides[].desc.html
This is how my office currently looks like. I’ve got more receipt printers than what is considered healthy. It’s starting to mess with me and and every shop or store that I visit I try to look for the model and make of their printer. Wondering if I should get one more.

# data[].attributes.slidedeck.data[].slides[].desc.html
If somebody says to you: “You don’t need another receipt printer. You already have more than enough receipt printers”. Just ignore them. You don’t need that kind of negativity in your life.

# data[].attributes.slidedeck.data[].slides[].desc.html
The result this madness is a library called ThermalPrinterEncoder that supports both ESC/POS and StarPRNT. It supports all the different resolutions and codepage mappings.

# data[].attributes.slidedeck.data[].slides[].desc.html
So if we want to print our original receipt. This is how we do it. And send the payload to the printer. Any receipt printer. StarPRNT or ESC/POS.

# data[].attributes.slidedeck.data[].slides[].desc.html
So if we want to print our original receipt. This is how we do it. And send the payload to the printer. Any receipt printer. StarPRNT or ESC/POS.

# data[].attributes.slidedeck.data[].slides[].desc.html
As a companion to ThermalPrinterEncoder, also I created a library that wraps WebUSB and allows you to simply connect to any supported printer and print. No need to worry about endpoints, interfaces.

# data[].attributes.slidedeck.data[].slides[].desc.html
And it is actually a set of libraries, with both WebSerial and WebBluetooth versions, that use the same API.

# data[].attributes.slidedeck.data[].slides[].desc.html
And that allows me to support all three protocols in our demo. The di erence is just which library is instantiated. So now we’re done with printers. We’re done with ESC/ POS and codepages. We can nally discuss some of the other devices.

# data[].attributes.slidedeck.data[].slides[].desc.html
Customer display fl Like this customer display. It is basically two lines of 20 vacuum uorescent characters that we use to show the current item added to the receipt and the price, so that the customer can check if the cashier added the right item for the right price.

# data[].attributes.slidedeck.data[].slides[].desc.html
What language do customer displays speak? So the important question. Do we need to learn another obscure language from the 80s? What language do customer displays speak?

# data[].attributes.slidedeck.data[].slides[].desc.html
Ehh… yeah. They speak printer. The next question would obviously be…. Why?

# data[].attributes.slidedeck.data[].slides[].desc.html
And the reason is again found in the 1980s… Because computers back then usually only had one serial port. So if you had two devices, you needed share that port. And to make that sharing work, both need to work together and speak the same language.

# data[].attributes.slidedeck.data[].slides[].desc.html
So when you initialise your printer slash display, both are active. Everything you print is also shown on the display. But you can send ESC = to turn o the printer, or turn o the display. When you turn it o , it will ignore everything it receives, until the next initialisation where you can make another choice.

# data[].attributes.slidedeck.data[].slides[].desc.html
So WebSerial. We request a port.

# data[].attributes.slidedeck.data[].slides[].desc.html
We open the port… specify the right speed. 10 Gigabit per second? 20? No… 9600… bits… per second.

# data[].attributes.slidedeck.data[].slides[].desc.html
We get our writable stream for sending data to the device… and we write our payload to the stream, effectively sending it to the display.

# data[].attributes.slidedeck.data[].slides[].desc.html
And that is it. Well… we still have to deal with codepages, still need to deal with ESC/POS….

# data[].attributes.slidedeck.data[].slides[].desc.html
But wait… This isn’t right. Our display is connected using USB. Why is it using WebSerial? And not WebUSB? Because of the 1980s…

# data[].attributes.slidedeck.data[].slides[].desc.html
And to see why, we need to look at USB classes.

# data[].attributes.slidedeck.data[].slides[].desc.html
USB defines a number of default classes. If you have a device that uses these default classes you don’t need to install a driver. It just works. ADC for audio. Human Input Device or HID for keyboards and mice. Mass storage for hard drives. And many more. And we have CDC for communication, which includes ACM - which is a modern replacement for legacy serial port communication.

# data[].attributes.slidedeck.data[].slides[].desc.html
Some of these classes have lower level API’s. Like WebHID for HID and WebSerial for legacy serial devices, Bluetooth based serial devices, but also the USB ACM class. And we have WebUSB for all other devices that do not t into these classes. So this is why we need WebSerial, even though it is USB, and not WebUSB.

# data[].attributes.slidedeck.data[].slides[].desc.html
But ACM is a replacement and not backwards compatible, so USB to serial port dongles don’t use ACM. In fact most USB serial devices don’t use ACM. They just took their old 1980s serial port device and added a custom Serial to USB chip from FTDI or Proli c with a custom driver.

# data[].attributes.slidedeck.data[].slides[].desc.html
And the driver creates a virtual serial port. And you use WebSerial for any type of serial port. Legacy, ACM or virtual. But… it is the custom class… so in theory you could write a WebUSB implementation… There is no documentation, so you would have to reverse engineer it. Besides, it would be completely useless… You can use WebSerial. It’s a big waste of time. But in theory you could.

# data[].attributes.slidedeck.data[].slides[].desc.html
Don’t use it.

# data[].attributes.slidedeck.data[].slides[].desc.html
So in conclusion. Almost nobody uses actual legacy serial ports anymore. But for some USB connected devices you still need to use WebSerial.

# data[].attributes.slidedeck.data[].slides[].desc.html
So I created a library that uses WebSerial to connect to that display and show some simple text. It will handle the codepages and ESC/POS encoding for you.

# data[].attributes.slidedeck.data[].slides[].desc.html
Barcode scanner And that brings us to barcode scanners. And that is really easy. We do not need to do anything to support barcode scanners, because

# data[].attributes.slidedeck.data[].slides[].desc.html
Barcode scanners are just keyboards.

# data[].attributes.slidedeck.data[].slides[].desc.html
When you scan a barcode they just literally type out the numbers of the barcode. So you could use keyboard events. Of course you need to gure out what belongs to the barcode and where it starts and ends, but that could be solved. Well, maybe not perfect, but good enough… Next…

# data[].attributes.slidedeck.data[].slides[].desc.html
Oh, but a barcode scanner is a HID device… what if we could use WebHID to get the actual barcodes instead of the keystrokes.

# data[].attributes.slidedeck.data[].slides[].desc.html
We request the device. The user then selects the barcode scanner.

# data[].attributes.slidedeck.data[].slides[].desc.html
And we actually get back a list of “devices”, because most HID devices are composite devices. This barcode scanner, also. It is a keyboard… like we just said. And a barcode scanner. So we need to gure out which devices is the barcode scanner part, and not the keyboard part.

# data[].attributes.slidedeck.data[].slides[].desc.html
Then we need to tell it to turn off the keyboard emulation and use plain HID reports instead.

# data[].attributes.slidedeck.data[].slides[].desc.html
And listen for the input reports.

# data[].attributes.slidedeck.data[].slides[].desc.html
And then we get this back.

# data[].attributes.slidedeck.data[].slides[].desc.html
And we just need to parse this and extract the value of the barcode.

# data[].attributes.slidedeck.data[].slides[].desc.html
But you don’t need to do any of that, because I have this library for you.

# data[].attributes.slidedeck.data[].slides[].desc.html
And if you have a serial based scanner, I also have a similar library that uses WebSerial instead.

# data[].attributes.slidedeck.data[].slides[].desc.html
And that is our final device. Still think that demo was underwhelming? The demo made it look easy. And that is great. It is supposed to be easy for the end user. But is still a huge effort to get this working.

# data[].attributes.slidedeck.data[].slides[].desc.html
We’ve always used web technologies for our app. But integrating with Point of Sale devices was always a struggle. We needed wrappers or middleware. But now all the puzzle pieces are in place. But this is what I love about Project Fugu. It really allows us to move towards a future where we don’t need Electron for our application.

# data[].attributes.slidedeck.data[].slides[].desc.html
But Project Fugu is not just about cash registers or hardware. It is about all kinds of API’s to make the web more capable and making sure that PWA’s can compete with native applications. Be it on mobile or on desktop. And for us that really works. And it might work for you too.

# data[].attributes.slidedeck.data[].slides[].desc.html
Thank you!