Streaming Video on 1602 Character LCD Display With Arduino

What makes this project special is that we are going to render graphics on a text LCD display which is not supposed to be used this way. Since you can’t control any particular pixel, this idea appears to be more of a pipe dream than a realistic plan. And this is why you should try it and push the hardware limits.

Spoiler: it will not run DOOM for now, but it will run "Bad Apple"!

🔗 You might also like the my text-based LCD Invaders game and my salvaging LCD tutorial.

HARDWARE

1. A character LCD display. The most common is 1602. 0802 and 1604 would work too. In fact we are going to use 4x2 area. I do not recommend LCD I2C units. They are significantly slower when it comes to display update rate.
2. An Arduino compatible board. Virtually any board which is capable of driving an LCD display would do. The most critical parameter for this project is flash memory. The more the better. Video streams consume a huge amount of memory. Arduino UNO is a good choice to start. But Mega or even ESP32 would allow you to process longer videos.

SOFTWARE

1. Python with cv2 library for image processing.
2. My img2lcdino script.
3. Some editor to split video into frames (I suggest that you use ffmpeg).
4. Arduino IDE.

A typical 1602 display features a 16x2 character matrix, and each character fits 5x8 pixels rectangle. There is also a gap between characters about 1 “blind” pixel, that is to say these margins have no pixels. All the character glyphs are hard-coded in the display. But you may upload up to 8 custom character bitmaps. I used this trick in another instructable to make a Space Invaders game.

As for video stream, eight 5x8 characters give us a 20x16 graphical viewport. Taking into account blind gaps between characters gives us a 23x17 viewport with “dead pixels” lattice. We should consider these blind zones in our further calculations to avoid video deformation.

First you should resize your video to fit a 23x17 viewport. You may do it with any video editor you like. For example, ffmpeg, which is a useful command line tool for video processing. The command would look like this:

ffmpeg -i input.mp4 -vf scale=24x18 output.mp4

Note that ffmpeg does not allow odd video dimensions like 23x17. So 24x18 would be ok.

Now you should split your video into frames. Again, you may use any video processing tool you like. With ffmpeg the command would look like this:

ffmpeg -i output.mp4 -r 10 output_%04d.png

This command will produce a lot of png images named output_0001.png, output_0002.png etc. The -r 10 flag means that you want 10 images per second. Note that LCD display update rate is not very high, so 10fps is a reasonable guess to start with.

I’ve made a Python script to generate an Arduino sketch from a series of frames. It needs cv2 library to work properly.

It loads each image, splits it into eight 5x8 rectangles, omitting blind zones, and generates Arduino code. You may collect further details from the attached flowchart and comments inside the code, but if you just want to get things done run the following command:

./img2lcdino.py > sketch.ino

It will produce a huge Arduino sketch.

If your sketch exceeds your microcontroller's memory, consider changing some settings in the script or generate less frames per second (see previous step).

It depends on the actual hardware you selected. The most typical layout is shown on the attached figure. Don't forget to set correct pins in the generated sketch:

LiquidCrystal lcd(/*your actual pins*/);

Is it possible to push Bad Apple into an Arduino UNO with 32Kb Flash? Each of 8 characters is represented by a 5x8 bitmap, that is to say 8x5x8 = 320 bits = 40 bytes per frame. 32 Kb gives you space for about 800 frames (in fact, you don't need to redraw unchanged characters, so this number depends on video and frame rate). Obviously you should decrease frame rate and implement a better compression algorithm. When I finished this project I found out that Ian Ward has been working on a similar idea recently. His outstanding implementation is hardware-only, without any microcontroller. And yes, it really fits 32Kb! Of course, with quality losses. So technically it is possible with aggressive and lossy compression algorithms.

Is it possible to increase resolution? It depends on your particular stream and how much quality loss you are ready to tolerate. This implementation provides a larger viewport by rendering nearly black characters black and nearly white characters white. It also uses other smart and aggressive compression techniques. However, if you want a lossless stream, 8 custom characters is your limit.

Will it run DOOM? Well, maybe. Certainly not on Arduino UNO. If you grab some single-board computer which is able to run DOOM and make it convert video frames to 23x17 monochrome bitmaps… And adjust contrast so that gameplay would be more or less distinctive… But I doubt it will be playable or even recognizable. Maybe I’ll give it a try one day.

Hope you enjoy this project!