Disclaimer

The following text was drafted by me and then rewritten by ChatGPT as a test for how “well” it currently works. There was some proofreading, but most of the language is from ChatGPT, with minor style adjustments.

I used the free version. It took only 2 minutes. I checked for any logical mistakes, and even though it uses ChatGPT’s language, the story and technical details are accurate.

Introduction

Projects often begin similarly: with a burst of inspiration or by revisiting a lingering idea. This particular project was sparked by Harry Potter’s Marauder’s Map and the magical, moving photos.

In this post, I’ll walk you through one of my early projects that turned out pretty well. We’ll delve into battery-powered IoT devices, ESP32, E-Ink displays, and the marvels of Stable Diffusion.

The Idea

As 2022 was drawing to a close, it struck me that we still didn’t have moving pictures on our walls. It’s surprising how technology often trails behind our imaginations.

The idea was simple: take an e-ink screen, combine it with an ESP32 for wireless connectivity, and attach a battery. Et voilà!

Selecting the Display

Since I already had ESP32s on hand, the first hardware choice was the e-paper display. When selecting e-paper screens, you need to consider several factors, including:

• Refresh Time
• Colors, Greyscale or Black and White
• Resolution
• Size

You also need to consider your budget, as all these features increase the price.

For example, you can get a cheap e-paper display from Waveshare with a high (~5s) refresh time or a more expensive one with a refresh time of less than a second. Note that the second one also supports 16 levels of grayscale.

There are many black-and-white, greyscale, tri-color, or multi-color displays available. More colors (not greyscale) mean longer refresh times and/or higher prices. On the other hand, more colors are better for displaying images. The same goes for resolution and size.

I decided to start with the cheap black-and-white one, as it was sufficient for a prototype. Having a refresh time of 5 seconds isn’t great for the battery or requires a good library. More on this later.

Building the Prototype

I chose a Waveshare display for several reasons:

1. The Display comes with the required controller board (as most displays)
2. Waveshare also sells ESP32 boards with the correct cable adapter already soldered on the PCB
3. Waveshare offers libraries and documentation (which other vendors sometimes don’t)

Simply plug the ESP and the display together, write some code, and it’s done.

I spun up a backend using Java and Spring Boot to serve an image via REST (you could use simpler technologies, but I needed the backend for later anyway) and wrote some code for the ESP32. The job of the esp is to wake up every now and then, download a new image if any and send it over to the display.

The first version didn’t work well, as the images were too big to send via a single transmission. I sent the images base64 encoded. When the body is too big the HTTPClient library cannot reserve enough memory to return the string. As my display was 800x480 pixels, taking 1 bit (black and white) per pixel it needs 48kb per image. Base64 adds some overhead but allows us to use strings. So we need to transfer 64kb per image. This is too much for the used esp and needed to be split up into chunks. I settled on a simple REST API that allowed the requester to provide a byte offset and max-length parameter and then receive the requested amount of bytes of the image. The ESP32 would cache the data, stitch it together, and then call the e-paper library to send the data to the controller.

Additionally the provided drivers by Waveshare were a bit tricky to use. I had some trouble with the official docu and needed some try and error until it worked out.

Great, now I had a display to which I could send images. But what did I want it to show?

Generating the Images

The initial method for pushing images to the screen involved auto-generating them using Stable Diffusion. The Java backend would generate random prompts based on templates.

Here are some example template strings:

{actor} (playing) {location}
{actor} in a {location}
{actor} playing (chess) with {actor}
{actor} surfing {location}
{actor} in love with {actor} {location}
{actor} fighting with {actor} {location}
{actor} riding a bike
{actor} riding a bike {location}
{actor} doing yoga
{actor} doing yoga {location}
{actor} driving a car through {location}
{actor} driving a car
{actor} riding {actor} {location}
{actor}, {actor} and {actor} playing poker
{actor}, {actor} and {actor} playing poker {location}

And an example of the kind of values used in these templates:

• Actors:

  A whale
  An orca
  A cat
  A dog
  A house
  A cup of coffee
  [...]
  

• Locations:

  in New York
  in Boston
  in Berlin
  in Paderborn
  in the USA
  in Germany
  [...]
  

Finally, each template would also be prepended with a style:

4K, 8K, drawing, black and white
4K, 8K, sepia coloring, hyper-detailed, dusk, octane render
[...]

Whenever an image was requested a new prompt was generated. This prompt was sent to a local Stable Diffusion server, and the result was printed to the screen:

Processing the Images

Since Stable Diffusion images often contain color, even if the style specifies ‘black and white,’ additional work was needed for the black-and-white display.

I used a simple dithering algorithm to dither my color images to black and white images. This greating increased the quality of the shown images.

The Result

I added a battery pack and optimized the Arduino code to use deep sleep, extending the battery life to about 2 to 4 weeks. I also added a cheap USB-C battery charging board.

The final backside looks like this:

The Battery Lifetime

Long refresh times and inefficient libraries reduce battery life. E-Ink displays use little power, but the ESP, especially when connected to WiFi, drains a lot. It’s crucial for the ESP to be in deep sleep as much as possible.

If the displays needs 5 seconds to refresh instead of 1 it increases the esp wake time by a factor of 5. Some libraries display combinations can counter this. Instead of waiting for the display to finish the update process they can send the image to the display driver board, trigger the update process and go to sleep.

Till time of writing I did not add this and the battery lifetime is still two to four weeks. I could also decrease the poll interval but this depends on the use case.

Version 2

After the initial success, I decided to add functionality to push images manually via Telegram, which provides an easy-to-integrate bot API.

An alternative would have been to set up a web service or website, but using an app I already had installed was a convenient choice.

This was a small extension to the backend. I also added some rights management so only registered users could push images.

The result was fantastic because now I could push my own images.

And people from my hackerspace started playing with it:

A Personal Touch

The best part of this addition is that you can also use it for your family. Shortly after finishing the Telegram extension, I went on a 2-week trip to NY and Boston with my little brother. We wanted to keep our grandma in the loop, so I deployed the picture frame at her home. We could then send her one or two images per day so she always had something nice and new to look at.

We even sent her a birthday message:

Final Thoughts

This project was both enjoyable and efficient. While I’m not entirely satisfied with the screen size and black-and-white limitation, it serves as a solid starting point. Upgrading would be costly, so I’ll stick with the current setup for now.

I’m still not sure where I want to go with this project, as the battery life is still too short. Upgrading from Wi-Fi could help, or investing more time in low-power programming might be the solution.

Hope you enjoyed the story! Feel free to to write me if you have any thoughts or questions!