Sixel Support in Windows Terminal
Image from Microsoft
Hey, it’s been a while. Recently, my work-life balance has come to resemble the lower circles of hell; violence, treachery, and betrayal are the order of the day. In order to cope (make it worse) — I’ve been working on a cool side project that hopefully will (probably not) get me some cred in the embedded systems space; a Game Boy emulator. It’s going awful, but over the past couple of months I’ve dug up some interesting things I wanted to write about.
Today’s topic is Sixel support in Windows Terminal.
For those uninitiated, Windows Terminal is an open source UWP app from Microsoft, released at some point around 2019. It now ships in Windows 11 as the default terminal emulator, while the legacy terminal sits quietly in the corner, waiting for you to right-click and “Run as Administrator”, so it can rear its ugly head once more. Running UWPs as admin is still a secret jutsu, I guess.
What’s a Sixel?
If you’re like me, you were probably today-years-old when you found out that most modern Linux terminal emulators can display bitmaps. That’s right! Pixels? In your ASCII-based retro-futuristic terminal? It’s more likely than you think.
Leave my custom neofetch ASCII art alone, you monster!
— Some guy reading this, probably
There’s actually plenty of different ways to display images in the terminal. If you’re using Kitty, you can use the icat command, for example. But today we don’t care about those, so here’s a stackoverflow link for you.
Why?
To be honest, I don’t know. It sounds cool, and I could sure think of a handful of cooool ideas (not foreshadowing). The lack of widespread support probably hinders adoption, which is probably why I’ve never heard of it until now.
How Does It Work?
I don’t really know. Aside from printing some fancy control characters, I’m not sure what the terminal emulator does to display the image. Judging from the grumpy maintainer comments I’ve seen on various terminal emulator repos, I’m guessing it’s not a trivial task.
Support
I found a website that lists terminal emulators that support Sixel graphics. However, I’m disappointed to report that (at the time of writing) it’s out of date, and the owner hasn’t accepted any PRs for a while.
Windows Terminal
Back in February, the team behind Windows Terminal pushed Sixel support to the release version of the app. This means that you can now display images in your terminal, just like a cool Arch Linux user.
Who Cares?
While working on my previously mentioned side project, I found myself in a very particular situation. You see, while I’m waiting for Power BI to get its act together, I’ve been working on this emulator over SSH. This is because our workstations are locked down tighter than my odds of finding a more fulfilling job. As such, I need to somehow display the 160x144 Game Boy screen on my terminal.
Previously, I had been using ASCII blocks to represent pixels on the LCD. This actually worked really well; the original Game Boy has 4 colors, and there are 4 shades of gray in the ASCII block spectrum (█, ▓, ▒, ░). Simply double that up for a square pixel, and you have a pretty good abstraction for a 2-bit display.
Now you might be thinking, “Cole, that’s a lot of screen space for a single pixel.” And you’d be correct; I had to zoom out to ~5% to see the whole screen. There are other issues, of course, such as I/O being too laggy to send 160x144x2 (Copilot says that’s 46,080) characters, 60 times a second.
At the time, I thought, “damn, it’d be cool if I could just send the image over the wire and have the terminal display it.” And then I remembered that I’m not a wizard, and that’s not how terminals work. Turns out I was wrong, because I came across the release notes for Windows Terminal, saw the latest release from a day earlier, and immediately booted up the Microsoft Store. (That’s right, I’ve been putting off this post for over a month.)
That Sounds Cool, Tell Us More
I found another library on GitHub that can convert the native Image object in Go to a Sixel representation. Did I use it? No, of course not—I got GPT o1 to rewrite it to be more efficient (it was too slow). I wrote a small “menu box” package and used it to draw the screen in the terminal.
func DrawBox(d display.Display, options *BoxOptions) []string {
// ...
var img = v.Image()
// Resize the image
if IMAGE_SCALE != 1 {
resized := image.NewRGBA(image.Rect(0, 0, img.Bounds().Dx()*IMAGE_SCALE, img.Bounds().Dy()*IMAGE_SCALE))
draw.NearestNeighbor.Scale(resized, resized.Bounds(), img, img.Bounds(), draw.Over, nil)
img = resized
}
content = append(content, renderer.RenderSixel(img))
// ...
return content
}Here’s how it looks:
Not bad, right? It started off very stuttery and flashy, but as soon as I optimized it and moved the rendering to a separate thread, it was passable.
Conclusion
I’m not sure if this is a feature that will be widely adopted, but it’s certainly a cool one. I can see it being useful for lots of different applications, performance allowing. I can already imagine streaming FFMPEG output to the terminal, or even YouTube videos.
There were a couple of limitations I ran into, but I’m not sure if those are Windows Terminal’s fault or the Sixel protocol’s. For example, I couldn’t get the image to align with text at all, and I couldn’t stack images side-by-side. I’m hoping this is consistent across implementations, for the sake of the feature. (I can’t test other terminals; I’ve been locked in the office using Windows for 3 weeks now, and they won’t let me out! Send help!)