ESP32 Webcam With Autofocus: Using Adafruit Ov5640 Breakout With Esp32-s3-devkitC-1-N8R8 (Also, a Basic Guide to Using PlatformIO)

Project Goal:

• 1) Describe the Adafruit OV5640 camera breakout board and how to enable the autofocus feature and internal clock on this breakout board.
• 
  
• 2) Describe how to physically select and connect the pins between the OV5640 breakout board and an esp32-S3-DevkitC-1-n8r8
• 
  
• 3) Use Platformio to load and modify the "ESP32CameraWebserver" example to use an ESP32-S3-DevkitC-1-N8R8 and Adafruit OV5640 breakout board (with an autofocusing 72 degree camera) instead of the familiar AI-Thinker module that has an ESP32 with an OV2670 camera,and enabling the 8MB of PSram on our esp32-S3 board.
• 
  
• 4) Use Erik Nam's autofocus library for OV5640 cameras-with-autofocus to enable autofocus in our example
• 
  
• 5) Write up a tutorial on how to use Platformio

Introduction:

I have used the AIThinker esp32 module with built in camera for previous projects. I liked it, but it had too few pinouts for my needs, and no autofocus (which was really important for one of my projects, also, too little PSRAM). Recently the OV5640 camera modules have come out, and OV5640 cameras have come onto the market that feature autofocus (ex: there are OV5640 cameras that are autofocus at 72 degrees viewing angle and autofocus at 120 degree viewing angle.

Adafruit came out with a breakout board for the OV5640 camera modules. The Adafruit OV5640 breakout board includes optional jumper settings for enabling both

• a built-in internal clock (saves a pin on the ESP32 and provides stable 24MHz to the camera) and
• enabling continuous autofocus. (SCORE!!!)
• they even allow you to remove the camera and replace them with other OV5640 cameras with different features like wide angle lenses and autofocus cameras.

Erik Nam (DEBT OF GRATITUDE to you sir) came out with a library for interfacing with an OV5640, enabling autofocus, and allowing it to run continuously

• Here is his github for the OV5640 zoom library (BIG SCORE!!!) https://github.com/0015/ESP32-OV5640-AF/tree/mainhttps://github.com/0015/ESP32-OV5640-AF/tree/main
• See his youtube tutorial on his project here .

In this tutorial we demonstrated how to

a) Enable autofocus and internal clock on the Adafruit OV5640 board,

b) Physically wire pins on the Adafruit OV5640 breakout board to an Esp32-S3-DevkitC-1-N8R8 (ESP32S3 board with 8MB of code and 8MB of external PSRAM),

c) create a pinout map for connecting our OV5640 breakout board to the ESP32-S3-DevkitC-1-n8r8 and porting it to our code instead of using the pinout maps for pre-existing esp32 camera modules such as an AI-Thinker board.

d) modify code in the pre-existing arduino esp32 "CameraWebServer" example project to enable us

• connect to both the esp32-s3 and the adafruit esp32 breakout board (match pinouts, select board, enable PSRAM)
• add Erik Nam's autofocus library to the project
• enable continuous autofocus

We will modify the arduino example: "esp32->camera->camerawebserver" to connect to our esp32-s3 and enable our custom pinout connections to the OV5640 camera.

To make connecting the physical hardware easier, we will modify the breakout board to connect to some terminal blocks and a stand so it can be easily mounted on a breadboard for testing.

We need the following

BASIC HARDWARE (a-e):

a) esp32-s3-devkitC-N8R8 (8mb of program memory, 8mb of psram)

b) adafruit breakout board for OV5640 (I ordered one with a 72 degree autofocus lens, and another with 120 degree autofocus lens, but cameras can be switched out at will, and can be ordered separately)

• https://www.adafruit.com/product/5840 (72 degree autofocus)
• https://www.adafruit.com/product/5838 (120 degree autofocus)
• (random aliexpress store with OV5640 cameras that have autofocus)

c) Breadboard (preferrably with a minimum of two rows)

d) Assorted 22 AWG wires for breadboarding (Use different colours to make stuff easier to tell apart)

e) MicroUsb cable (for powering our boards and programming our ESP32-S3 board)

f) 3.3VDC power supply (use either a breadboard power supply or a 3.3VDC boost regulator)

• Breadboard power supply 5VDC/3.3VDC https://leeselectronic.com/en/product/106865-breadboard-power-supply-5v-33v.html
• Pololu 3.3VDC 1A step up/step down power supply https://www.pololu.com/product/2830

Optional hardware (g-o):

To make this project breadboard compatible, we made our own mounting boards with screw terminals to make it easier to reliably wire the breakout board to the esp32-s3-devkitC-1 board

g) 0.1" pitch screw terminal blocks (an assortment of 2 & 3 pin 0.1" pitch terminal blocks at a minimum)

• (enough for two (2)x rows of 9- pins each)
• (enough for a row of 9-pins)
• (enough for a row of 6-pins)
• (enough for a row of 2-pins)

https://www.aliexpress.com/item/4000867583795.html

h) two (2) Hexagonal 1/2" female-female 4-40 offsets

i) Four (4) 4-40 x 1/4" pan-head philips screws

j) Four (4) #4 washers

k) one piece of 3cm x 7cm double sided protoboard (0.1" pitch) we will cut to 3cm x 5cm

l) One piece of 1.3"x1.8" breadboard protoboard (we will cut it in half)

m) 0.1" pitch single row male header with minimum 4mm post height and minimum 4mm mating height (get post & mating heights between 4mm and 6 mm in length)

• enough for minimum of four (4-6) 4-pin rows

n) 0.1" pitch female header

• enough for enough for minimum of four (4-6) 4-pin rows

o) 0.1" pitch single row male header with minimum 6mm post height and 3mm mating height

• 1x 17-pins row
• 2x 3-pin row

O_a) Arduino stackable mounting headers

SOFTWARE TOOLS (o - p):

o) Arduino IDE (get the most recent one)

p) Visual Studio code (with the following extensions)

• o.1) PlatformIO
• o.2) Espressif IDE
• o.3) Arduino for Visual Studio Code

LIBRARIES & CODE EXAMPLES (q-r):

Github code to download: (download this and install in one of your coding directories, or make your own fork on github)

q) https://github.com/espressif/arduino-esp32

(includes the camera library and the camera webserver example we will modify to add autofocus and use the esp32-s3)

• We want to open the example that is stored in the directory arduino-esp32-master\libraries\ESP32\examples\Camera\CameraWebServer

r) https://github.com/0015/ESP32-OV5640-AF/tree/main

(This is Eric Nam's library for enabling and using autofocus on autofocus enabled OV5640 cameras)

Read through the Adafruit OV5640 breakout documentation to get a good understanding of what we want our OV5640 to do, and what specific features of this breakout board we want to take advantage of.

For the Adafruit OV5640 breakout board, Adafruit has made a very good tutorial on how to use their board. Read the tutorial sections for

• Overview
• Pinouts
• Assembly.

Looking at the section with the jumpers, we would like to enable the following features

a) USE INTERNAL CLOCK: Enabling the feature of the OV5640 breakout board will provide the camera with a stable 24MHz clock, and save a pin on the esp32. For the CLK pad, cut the trace from the "center pad" to the "external clock" pad, and put a solder blob to join the "center pad" to the "internal clock" pad

b) ENABLE AUTOFOCUS: Autofocus enabled cameras have a small Voice coil Motor (VM) to adjust the focus length of the camera. Enabling this feature provides power to the VM (Voice Motor) that is used in autofocusing cameras (enabling this feature has no effect on cameras that do not have autofocus).

The different pins on the camera breakout board do different things. We do not need to detail what they do (please read the "Pinouts" section of the adafruit OV5640 tutorial. We just need to outline what pins we will connect to. Based on the "camera_pins.h" file used in our example (see https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/Camera/CameraWebServer/camera_pins.h)

We will name our pins, and show the way that each pin is named in the "camera_pins.h" file

Ex: the pin "XC" on the OV5640 breakout is named as "XCLK_GPIO_NUM" in the "camera_pins.h" file. So we will write

"1) XC (external clock) // #define XCLK_GPIO_NUM"

(Please note that different ESP32 camera boards use different pins to connect to the OV5640 camera, so the ESP32 IO pins used to connect to specific OV5640 pins differ for different ESP32 Camera boards, and the value of these ESP32 IO pins get defined differently for different boards. If a pin on the OV5640 is UNUSED, it is defined as "= -1")

The OV5640 has 18 pins total (including GND and 3.3V DC power).

We will ignore the two (2) pins

1) XC (external clock) // (#define XCLK_GPIO_NUM =-1)

2) RT (Reset) //(#define RESET_GPIO_NUM = -1)

We will use the following sixteen (16) pins on the OV5640 board

3) PD (Power Down) //(#define PWDN_GPIO_NUM)

4) SDA(I2C Serial Data) //(#define SIOD_GPIO_NUM)

5) SCL (I2C serial clock) //(#define SIOC_GPIO_NUM)

6) D9 (D9) //(#define Y9_GPIO_NUM)

7) D8 (D8) //(#define Y8_GPIO_NUM)

8) D7 (D7) //(#define Y7_GPIO_NUM)

9) D6 (D6) //(#define Y6_GPIO_NUM)

10) D5 (D5) //(#define Y5_GPIO_NUM)

11) D4 (D4) //(#define Y4_GPIO_NUM)

12) D3 (D3) //(#define Y3_GPIO_NUM)

13) D2 (D2) //(#define Y2_GPIO_NUM)

14) VS (Vertical Synch) //(#define VSYNC_GPIO_NUM)

15) HS (Horizontal Synch) //(#define HREF_GPIO_NUM)

16) PC (Pixel Clock) //(#define PCLK_GPIO_NUM)

17) G (Ground)

18) 3V (3.3V DC power)

Please read the documentation for the ESP32-S3-Devkit1 here at https://docs.espressif.com/projects/esp-idf/en/stable/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html. If you download the page as a PDF (see the attached PDF file), then read through pages 1-20 (I cannot attch the PDF file to this instructable, because it exceeds the size limit for allowable attachments.).

The ESP32-S3-DevkitC-1 is a type of development board from Espressif for enabling users to work with the ESP32-S3-WROOM microcontrollers. It provides WIFI interfacing, bluetooth wireless interfacing, and serial and USB interfaces as well as exposing most of the pins on the microcontroller. These things are awesome.

(NOTE!!!! just because a pin on the ESP32 is EXPOSED, that doesn't mean that the pin is useable in a specific project or USEABLE AT ALL... please make note of STRAPPING PINS, and pins used to access PSRAM).

The ESP32-S3-DevkitC-1-N8R8 is a version of the ESP32-S3-DevkitC-1 with additional program memory (N8 = 8MB of program memory instead of 520K), and 8MB of external ram (PSRAM) that the ESP32-S3-WROOM has to connect

PINS TO NEVER USE:

Some pins on the ESP32-S3-DevkitC-1 are called strapping pins, and connecting these to hardware will likely interfere with the operation of the microcontroller. On our ESP32-S3-DevkitC-1 we avoid connecting hardware to these pins.

The following four(4) boot/strapping pins

• GPIO03 / GPIO46 / GPIO00 / GPIO45 : These are the boot/strapping pins

The following three (3) pins only apply to ESP32-S3-DevkitC-1 boards with octal PSRAM (like the ESP32-S3-DevkitC-1-N8R8). They are used to connect to the external PSRAM using the three (3) pin SPI protocol

• GPIO35 / GPIO36 / GPIO37 : These are used for SPI interfacing to PSRAM

PINS TO STRONGLY AVOID:

The ESP32-S3-DevkitC-1 has two separate ports (USB & UART) that can be used for programming the ESP32-S3 over USB.

a) The UART port: (GPIO43 / GPIO44): This UART serial port provides USB communication through an internal on-board CH340 serial-to-USB interface. This port can be used to

• communicate with a PC over USB (USB serial communications),
• program the ESP32-S3,
• operate as a USB slave device.
• You can also bypass the USB functions of this port and use the pins either as UART serial pins or as GPIO

b) The USB port: (GPIO19 / GPIO20): This is a USB port built into the ESP32-S3 microcontroller itself. It is used for

• communicating with a PC over USB (USB serial communications)
• program the ESP32-S3,
• operating as a USB slave device, AND operating as a USB HOST device port,
• debugging the microcontroller over JTAG (this is the primary JTAG debug interface for the ESP32-S3)
• You can also set the pins of this port to operate as GPIO.

While debugging and programming your ESP32-S3, make sure that at least one of these ports is available. By defult, you should be using the UART port. If you need JTAG debugging, try and use the USB port.

PINS TO WEAKLY AVOID:

These are pins that you can use, but there are some specialized purposes where you may want to reserve them for alternate purposes.

c) Secondary JTAG pins: (GPIO39 / GPIO40 / GPIO41 / GPIO42): The ESP32-S3 has two JTAG debugging ports. The USB port is the primary JTAG port, but if you set some internal fuses you can use these alternate pins as JTAG pins. Usually you can use them as GPIO

d) RGB LED pin: (GPIO38): This pin connects to an RGB colour changing LED mounted onto the ESP32-S3-DevkitC-1 board. You can use it as GPIO or you can run the RGB LED on the board

In this step we will take our OV5640 breakout board and do the following

a) enable the internal 24MHz clock (saves a pin on our ESP32 and provides a reliable CLK to our camera)

b) enable autofocus

c) Solder 0.1" pitch terminal blocks to our OV5640 to ensure we have a reliable connection to our wires

Tools:

1) Soldering iron/solder/flux & soldering tools

2) exacto knife (to cut trace)

Parts:

3) 0.1" terminal blocks (enough to make two (2) rows of 9-pin headers)

4) Adafruit OV5640 board

Steps:

1) On the OV5640 breakout, find the XCLK jumper, and identify the small trace between the XCLK:center-pad & XCLK:Ext pad (the one on the right)

2) Cut the trace with an exacto blade (scrape the thin connection away. You can trust your eyes or use a digital multimeter to verify the trace has been correctly cut)

3) Solder together the XCLK:Center-pad and the XCLK:Int-pad (the one on the left). The internal 24MHz clock is now enabled

4) Find the VM jumper, and solder the two pads together. You are now able to provide power to the Voice Coil Motor of an OV5640 with Autofocus (does not affect OV5640 cameras without autofocus).

5) We define "Right-side-up" on the OV5640 breakout as the side with the camera on top. Find the row of pins that include SDA / HS/ XC / D8 / D6 / D4 / D2 / PD. We will call this header #1. Solder the terminal blocks to this row right-side up.

6) We define "Upside-down" on the OV5640 breakout as being the other side of the board that does NOT have the camera. Find the row with SCL / VS / PC / D9 / D7 / D5 / D3 / RT. This is header #2. Insert and solder the terminal blocks into this row on the underside of the board. Now header #1 is on the top side of the board, and Header #2 is on the bottom side of the board.

Now all wires can be inserted into screw terminals, and attached securely, and both internal clock and autofocus are enabled.

With terminal blocks installed, any wired connections are very secure but a downside is that the board will not lie flat. It is convenient to have a mounting board to hold the OV5640 in place onto our breadboard. One that uses the mounting holes on the OV5640 breakout, and which can be plugged physically into a breadboard as a physical mount while experimenting and breadboarding.

A standard breadboard has a pitch of 0.1" between pins. Each rows on a breadboard has five (5) pins connected together internally, then a 0.3" gap, then another row of five (5) pins connected internally.

This is a set of instructions for making a camera mount to hold the OV5640 breakout board and physically mount it on a breadboard. The pin headers are spaced so that the board can be inserted in one of four angles in the breadboard

(0 degree / 90 degree / 180 degree / 270 degree )

Materials:

a) two (2) Hexagonal 1/2" female-female 4-40 offsets

b) Four (4) 4-40 x 1/4" pan-head philips screws

c) Four (4) #4 washers

d) one piece of 3cm x 7cm double sided protoboard (0.1" pitch) we will cut to 3cm x 5cm

e) 0.1" pitch female header

• enough for enough for minimum of four (4-6) 4-pin rows

f) 0.1" pitch male headers

Tools:

g) Soldering iron / flux/solder/ solder tools

h) Drill with 1/8" drill bit

i) Hacksaw (to cut the board)

j) Sharpie (to mark stuff)

Steps:

1) READ THE STEPS COMPLETELY IN THIS SECTION BEFORE TRYING TO BUILD THE BOARD, AND LOOK AT THE ATTACHED DIAGRAMS.

2) The OV5640 has two mounting holes spaced 0.7" apart. Mark on the protoboard the location where the holes should be drilled. Note that the pre-drilled holes in the protoboard are great centering holes for when we start drilling

3) Mark on the board where we will be inserting and soldering the four(4) female headers. Make sure that there is a three(3) pin gap between the blocks of headers. Please see the diagram of the marked protoboard before soldering. The pin headers are spaced so that the board can be mounted at 0 degrees, 90 degrees, 180 degrees, and 270 degrees on a standard breadboard.

4) Once layout is good, use a 1/8" drill bit to drill out the holes for the support mounts.

5) Solder in the four (4) 4-pin female pin-headers.

6) screw in the mounting posts to the protoboard, and screw the Adafruit OV5640 breakout to the mounting board. Even though it is only two screws, the board will be VERY rigidly attached.

7) insert some of the male pin headers into the breadboard for the female pin headers to mate to. Connect the mounting board to the breadboard, and orient and position as you wish.

8) For conenience, label the pins on the terminal block

This is another helper board to make wiring more secure (I've had some pretty nasty failures due to suddenly detached wires in a breadboard). It is a piece of breadboard-protoboard that has ben cut in half, and has had terminal blocks and arduino stackable pin headers attached. Now wires will be securely held, and pins can be tested at will.

Materials:

a) mini breadboard prototyping board 1.3" x 1.8" (cut in half to make two (2) 17-row boards)

b) Male header (Enough for a row of seventeen (17)). (trim down to 17-pins), or use a singlerow of nine (9) & a single row of six (6)

c) 0.1" pitch terminal blocks (enough for a row of nine(9), and a row of six (6))

Tools:

a) Hacksaw

b) Soldering iron / flux / solder / tools

c) flush cutter (to trim down rows of pin headers)

Steps:

1) Take the mini breadboard-prototyping-board (1.3" x 1.8", and cut it in half down the middle (so you have two (2) 0.65" x 1.8" boards.

2) Solder in the 17-pin header into the breadboard along one edge. This is the right side of the board.

3) Since the board will be mounted on the left side of the ESP32-S3-DevkitC-1-N8R8, we will make a row of six (6) terminal blocks on the bottom-left of the board (For GPIO09 (D3) / GPIO10(D5) / GPIO11(D2) / GPIO12(D6) / GPIO13 (PC) & GPIO14 (not connected to OV5640). Label each terminal block IO point with a fine point sharpie.

4) We will add on the top-left part of the board a row of 9-pin terminal blocks (For GPIO04(SD) / GPIO05(SC) / GPIO06(VS) / GPIO07(HS) / GPIO15(PD) / GPIO16(D9) / GPIO17(D8) / GPIO18(D7) / GPIO08(D4). Label each terminal block IO point with a fine tip sharpie.

5) Plug the terminal block array next to the correct pins on the ESP32-S3-DevkitC-1-N8R8

In our demonstration circuit, the pins on our ESP32-S3-DevkitC-1-N8R8 are wired to the Adafruit OV5640 breakout board. We use the helper boards to mount the camera and make it easier to wire the OV5640 breakout to the OV5640-S3-DevkitC-1-N8R8.

BOM:

a) Breadboard (minimum of two rows of breadboards)

b) assorted wires

c) ESP32-S3-DevkitC-1-N8R8

d) Adafruit OV5640 camera breakout (with terminal blocks and with internal XCLK and Autofocus enabled)

e) Mounting board for the OV5640 breakout board (optional but useful)

f) Terminal block Array (optional but useful)

g) 3.3VDC power supply (in this case a breadboard power supply set up to provide 3.3VDC, with a 12VDC wall-wart)

h) Label and sharpie to mark the board and identify the project being used on the breadboard. (a good habit)

PIN CONNECTIONS:

OV5640 pin ESP32-DevkitC-1-N8R8 pin

1) RT -NC-

2) D3 GPIO09

3) D5 GPIO10

4) D7 GPIO18

5) D9 GPIO16

6) PC GPIO13

7) VS GPIO06

8) SCL GPIO05

9) 3V 3.3VDC

10) PD GPIO15

11) D2 GPIO11

12) D4 GPIO08

13) D6 GPIO12

14) D8 GPIO17

15) XC -NC- 

16) HS GPIO07

17) SDA GPIO04

18) G GND

STATE OF THE PROJECT:

We have our adafruit OV5640 breakout properly jumpered, and both the OV5640 and the ESP32-S3-DevkitC-1-N8R8 are properly breadboarded, wired, and ready to be powered up.

REMAINING STEPS TO PROJECT COMPLETION:

Now begins the PlatformIO tutorial.

The hardware portion of the project is completed. The next set of steps focus exclusively on setting up and configuring software and then compiling the example and uploading it to the firmware.

These steps include

1. Download the example firmware,
2. Download the required libraries
3. Create a new platformio project for our Esp32-S3-DevkitC-1-n8r8
4. Configure the project to use the 8MB of flash memory, and 8 MB of PSRam
5. Include the required firmware and libraries in our project
6. Modify the firmware code to enable autofocus
7. Compile and upload code to our ESP32-S3-DevkitC-1-N8R8
8. Test our new example

Goal:

• Create our Platformio project "CameraWebServerAutofocus"
• Include our board and framework
• Modify the "platformio.ini" file to use PSRAM and match our ESP32-S3-DevkitC-1-N8R8

NOTES:

This project is based on taking one of the standard arduino example for the ESP32 camera (specifically "CameraWebServer" from espressif), and modifying it to use Erik Nam's autofocus library, and to make the example compatible with our esp32-S3 to OV5640 pinout map.

Aside from ensuring that our computer has the (free to download and install) Visual Studio Code compiler downloaded and installed with the following extensions added on

• Platformio (famous for being better than the Arduino IDE)
• Espressif
• Arduino IDE

(If you need a guide to installing Visual Studio Code and Platformio, here is a tutorial from Dronebot Workshop https://www.youtube.com/watch?v=JmvMvIphMnY&t=2113s

In this section we will do the following,

• we will create a new PlatformIO project,
• select our microcontroller board (We can only select ESP32-S3-DevkitC-1-N8 but we will enable PSRAM later)
• Select our framework (Arduino)
• Modify the Platformio.ini file to enable PSRAM, and enable the serial comms monitor at 115200 baud.

In the next section we will download libraries

Steps:

Step #1) Open "Visual Studio Code" -> Go to "Platformio" -> Click "Create New Project"

Step #2) In the dialog box that pops up, select the following options

• a) For Project Name: I wrote "CameraWebServerAutofocus"
• b) For Board: Select "Espressif Esp32-S3-DevkitC-1-N8" (we add PSRAM later)
• c) For Framework: Select Arduino
• d) For Save Location: I used the default Platformio Project directory that I have set up on my computer.

In my case, my whole project "CameraWebServerAutofocus" is now saved in the directory

• "C:\Users\user\Documents\PlatformIO\Projects\WebCameraServerAutofocus"

Step #3) The project has been created. The board our project uses is the Espressif Esp32-S3-DevkitC-1-N8, but we have also not yet intiated PSRAM. We have also not yet configured the Platformio monitor to use a baud rate of 115200 baud when monitoring the COM port. We will enable these features now.

a) Go to the Project Explorer section, find "platformio.ini" and open it.

b) Now add the following lines below after the settings already in the platformio.ini file to enable both flash memory and set the speed of the serial monitor

//////////////////////////////////////////////////////////////////////////
board_build.arduino.memory_type = qio_opi
board_build.flash_mode = qio    ; QIO is the type of flash memory used
board_build.prsam_type = opi    ; The PSRAM on the board uses Octal Psram
board_upload.flash_size = 8MB   ; Flash memory is defined as being 8 MB
board_upload.maximum_size = 8388608 
board_build.extra_flags = 
 -DBOARD_HAS_PSRAM     ; This tells the board that it has PSRAM (we don't need to define toal PSRAM size)
;
; add this to set the speed of the COM monitor port to 115200 baud
monitor_speed = 115200
////////////////////////////////////////////////////////////////////////////////////////////

Special thanks to user "Sivar2311" for his post on the Platformio Community Forum on enabling PSRAM and Flash memory from within the "Platformio.ini" file

;https://community.platformio.org/t/how-do-you-set-up-a-new-board-esp32-s3-n16r8/32306/25

I have attached my current (As of this current step) "platformio.ini" as a text file (because Instructables does not let us upload ".ini" files, just rename it if you use it). The "platformio.ini" file will change as we add our libraries and their dependencies.

Goal:

• Understand the layout and operation of the Platformio library manager
• Understand the difference between libraries stored in the Platformio online "registry", "Installed", "Built-in"
• Understand how to install and remove libraries from the project
• Understand how to find & browse through examples that are connected to specific Platformio libraries
• Understand how to use the "reveal" button to find where library files have been copied onto the local system

NOTES:

The PlatformIO Library manager is useful to find, download, and manage the libraries that are added to individual projects.

• You can search through and add libraries that are stored on the online Platformio library registry (which automatically updates the "platformio.ini" file to include the library dependencies).
• You can see what libraries are installed in your project
• You can use "reveal" on a library to actually open up the local directory where the examples are stored so you can find and copy files and folders into your project directories
• You can also go through libraries that are installed locally, find their examples

We will be using the library managment feature of Platformio to add the Esp32 Camera libraries from Espressif, and to add Eric Nam's Autofocus library.

For a GREAT tutorial on how to use libraries in Platformio, check out this tutorial from "Simply Explained" on youtube

https://youtu.be/buFKeqbafDI?si=ZT0sPT272eh5Anj7

Another more technical tutorial on how to add libraries to PlatformIO projects is from "Mission Critical" on youtube, who details three ways to include libraries in Platformio

https://youtu.be/T679yGD9Res?si=9fNjBqnEHeqLL1au

Library examples are great, because they can show you how to use libraries. W.r.t examples, the PlatformIO library manager lets you

• browse through the list of examples attached to a locally installed library,
• see the files used in each example,
• see what "lib_deps" need to be added to each "platformio.ini" file to use the libraries when compiling your firmware

Annoyingly, when browsing through the files associated with a library->example, the Platformio Library explorer DOES NOT LET YOU COPY AND PASTE LOCALLY STORED FILES WITHOUT USING "reveal". (BOO. I would like the Platformio Library Manager allow copying drag-and-drop for files in library examples)

Steps For Using Platformio to add a library from the Platform:

Click on the "Platformio Icon" (The alien face) -> "Library Manager"

In the window that comes up, you will see four (4) tabs

• "Registry" : (lets you search the Platformio online repository for code libraries)
• "Installed" : (This shows what libraries have been downloaded and installed into the project)
• "Built-In" : (Given your selected framework: lets you browse and select libraries and library examples that are provided by default for your framework)
• "Update" : (Lets you check for updates to a library and its files)

Looking at Each of the Tabs above in more detail

• "Registry:" : Allows us to connect to the PlatformIO online repository of libraries managed by PlatformIO, search through them, select specific libraries and then download and install the selected libraries to our project (including updating the "platformio.ini" file to properly include these library dependencies ("lib_deps").

• "Installed" : This shows what libraries have been downloaded and installed into our project. It will also show the current version of the library that is installed.

• "Built-In" : When a Platformio project is created we often will need to select a framework to operate within. These different frameworks and their code are installed locally and their libraries and code can be included into your projects. Example: in the case of our ESP32 microcontrollers, we could have selected either the ESP-IDF framework, or the Arduino framework. Given our board is an ESP32, and our selected project framework is the "arduino" framework, then we will have available to us the "built-in" framework "Espressif arduino-esp32", and its assortment of libraries (ex: ESP32, ESP32 BLE Arduino, ESPmDNS, etc...)

• "Update" : Libraries often get updated over time to fix bugs and add new features. Clicking on this tab lets us check if our libraries need to be updated, and (depending on the lib_deps settings in our "platformio.ini" file) can help update them automatically.

Now we will select a library ("ESP32 by Hristo Gochkov") and look at some of the code examples provided with the library and the platformio.ini settings that it will could add to our project. We will look at the "CameraWebServer" example file included with the "ESP32" library. Please not that the examples included in this library are the exact same example that are able to be browsed with the Arduino IDE (which is why they are listed as .ino files even though Platformio does not use arduino style ".ino" files and needs them renamed to ".cpp")

Goto "Libraries", select the tab: "built-in", and in the search-bar type "Esp32". Select the library "Esp32 by Hristo Gochkov"

The new windows that pops up says the name of the library selected and has four (4) tabs

a) Tab: "Examples"(*1): (allows users to

• select examples related to the selected library
• browse the files used in a selected example

b)Tab: "Installation": (Lists the necessary library dependencies in platformio.ini for the selected example)

c) Tab: "Headers": (Lists headers used in an example)

d) Tab: "Changelog" : (displays the changelog for the selected code example)

(Note *1): If an example is selected, and you want to include the files from the example in your code, it is not possible to do that through the Platformio browser. Instead, when selecting the library for use in the file, click on the "reveal" button to open a file-explorer window to the installation directory of the library, and then from threre to go into the directory storing your desired example and copy the files you want and paste them into your specified project directories in your project.

Goal:

• Find and copy the exmple code from the "CameraWebServer" example in the locally installed "Espressif - arduino Framework" code repository into our project. The "CameraWebServer.ino" file will be renamed to "CameraWebserver.cpp".
• Update the Platformio.ini file to include our necessary library dependencies.

NOTES:

Our project will use the "WebCameraServer" example as its base, and add a few lines of extra code to enable autofocus.

Steps:

Step #1) remove the current "main.cpp" (the one created at default when our project was first created) from our project source directory

Step #2) create in our project ("CameraWebServerAutofocus") library-folder a sub-folder called "CameraWebServer"

Step #3) Using the Platformio library manager, select the TAB "built-in", and search for the library "ESP32 by Hristo Gochkov". Use the "reveal" button to have file-manager open to the --BASE-- directory where the "ESP32" library is stored. (In our case our --BASE-- directory was C:\Users\user\.platformio\packages\framework-arduinoespressif32\libraries)

Step #4) Navigate from the --BASE-- directory to the example folder to the folder holding the files for "CameraWebServer". (In our case, this was the directory C:\Users\user\.platformio\packages\framework-arduinoespressif32\libraries\ESP32\examples\Camera\CameraWebServer)

e) Use copy-paste to copy the four files "app_httpd.cpp", "CameraWebServer.ino", "camera_pins.h", "camera_index.h" into our project library subdirectory "CameraWebServer".

f) Select the file "CameraWebServer.ino" and move it into the project src directory. Given that Platformio does not recognize ".ino" files, rename "CameraWebServer.ino" to "CameraWebServer.cpp"

g) If you used the Platformio Library manager to look at the ESP32 example code for "CameraWebServer", then in the tab "installation", you can see the lib_deps for "CameraWebServer".

g) As a final step, open up the "platformio.ini" file for our project and copy into it the line

lib_deps =  ESP32 @ ^2.0.0

Goal:

• Use the Platformio Library manager to search the Platformio online code repository for Eric Nam's "Autofocus" library.
• Use the Library manager to install the library "Autofocus by Eric Nam" into our project and automatically update our project's platformio.ini to include the necessary lib_deps

NOTES:

The ability of the PlatformIO Library manager to let us browse through a large online code repository, and then seamlessly install or uninstall libraries (including updating our platformio.ini file) is great. We will use it to add the "Autofocus by Eric Nam" library into our project.

Steps:

Step #1) Goto the Platformio Library manager, select the TAB: "Registry", and type in the search-bar "Autofocus"

Step #2) Select the "Library for OV5640 Autofocus by Eric"

Step #3) Click on the "Install Library" button

Step #4) Select our project "CameraWebServerAutofocus", and install the Autofocus library into our project.

You can now use the Vistual Studio Code Explorer to look at our project, and see that our Autofocus library was added to our project in the "Build"->"Lib_deps" directory, and the platformio.ini file has been automatically updated with the lib_deps for the Autofocus library.

Goal:

• Create a header file that specifies how our GPIO pins on the ESP32-S3-DevkitC-1-N8R8 map to the OV5640 breakout board.
• Modify the header file "camera_pins.h" to include a conditional #include of "Experimental_Camera_pins.h"

NOTES:

When a microcontroller board with a built-in camera (like an ESP32 AI-Thinker or an ESP32-S3-Freenove ) is produced, every single board of that type will consistently map the GPIO pins of the microcontroller to the pins of the OV5640 in the exact same way. The AI-Thinker board will have a different pin-map than the ESP32-S3-Freenove, but all AI-Thinkers use the same pin-map between the GPIO pins of the microcontroller and the pins of the camera. The header file 'camera_pins.h" has definitions of pin-maps for many different types of ESP32 camera boards, that is defined in a way that can be consistently used by the camera library. Simply define which board you are using, and if the board is already in "camera_pins.h" then up to sixteen (16) pins on the board will be defined in order to map to the camera.

This works out well when you have a board that is manufactured consistently, but when you are using a breakout board for a camera such as an Adafruit OV5640 breakout and prototyping with different types of pinouts then you have to write your own pin-map that is consistent with the camera library. Other camera module boards may be designed with specific design goals. In our case our board was wired in a way that was convenient to breadboard, and does not match the pin-map of any camera module on the market.

There are several ways to create the pin-map

a) you can define the variables in the same file as your main() function

b) you can alter "camera_pins.h" to include a pin-map for your breakout board (just be consistent with how you wired the parts together)

c) you can create an external header file that defines the pin-map (ex: "experimental_camera_pins.h") to use with experimental boards that only needs to be included when using camera breakout boards.

We will use option C, and define a new header file for mapping pins to breakout boards.

Steps:

Step#1) Open in your project "camera_pins.h" and find the last three (3)lines at the end of the file

#else
#error "Camera model not selected"
#endif

(this block of code means that no camera board was defined, so there is no pin-map defined)

Step #2) Add the following two lines to our own definition for Experimental_Camera_Pins.h into "camera_pins.h" before #else #error

//Adding our own definition for Experimental_Camera_Pins.h  
#elif defined(EXPERIMENTAL_CAMERA_PINS)
#include "experimental_camera_pins.h"
/////////////////////////////////////////////////////////
#else
#error "Camera model not selected"
#endif

Now, if your code has "EXPERIMENTAL_CAMERA_PINS" defined, then we will use the pin-map in "experimental_camera_pins.h"

Step #3) In our library directory "Lib->CameraWebServer", create a new header file "experimental_camera_pins.h"

Add the following lines into "experimental_camera_pins.h" (the code below is the complete header file for "experimental_camera_pins.h")

//Pins defined by users using OV5640 breakout boards or other boards
//Change pin-map to match your project.


#if defined(EXPERIMENTAL_CAMERA_PINS)  
       //Define the pin-map for the following sixteen (16) pins
#define RESET_GPIO_NUM   -1  //  -1 Only if not using reset
#define Y3_GPIO_NUM       9
#define Y5_GPIO_NUM      10
#define Y7_GPIO_NUM      18
#define Y9_GPIO_NUM      16
#define PCLK_GPIO_NUM    13
#define VSYNC_GPIO_NUM   6
#define SIOC_GPIO_NUM    5
#define PWDN_GPIO_NUM    15
#define Y2_GPIO_NUM      11
#define Y4_GPIO_NUM      8
#define Y6_GPIO_NUM      12
#define Y8_GPIO_NUM      17
#define XCLK_GPIO_NUM    -1  //-1 Only if using a breakout that provides an internal camera clock
#define HREF_GPIO_NUM    7
#define SIOD_GPIO_NUM    4


#endif

Step #4) Now go to the directory "src->WebCameraServer" and open the file "WebCameraServer.cpp". Reading through the file, we see that the current board that is defined in this file is the CAMERA_MODEL_ESP_EYE

// ===================
// Select camera model
// ===================
//#define CAMERA_MODEL_WROVER_KIT // Has PSRAM
#define CAMERA_MODEL_ESP_EYE // Has PSRAM        *****SEE ME*****
//#define CAMERA_MODEL_ESP32S3_EYE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_PSRAM // Has PSRAM
//#define CAMERA_MODEL_M5STACK_V2_PSRAM // M5Camera version B Has PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_ESP32CAM // No PSRAM
//#define CAMERA_MODEL_M5STACK_UNITCAM // No PSRAM
//#define CAMERA_MODEL_AI_THINKER // Has PSRAM
//#define CAMERA_MODEL_TTGO_T_JOURNAL // No PSRAM
//#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM
// ** Espressif Internal Boards **
//#define CAMERA_MODEL_ESP32_CAM_BOARD
//#define CAMERA_MODEL_ESP32S2_CAM_BOARD
//#define CAMERA_MODEL_ESP32S3_CAM_LCD
//#define CAMERA_MODEL_DFRobot_FireBeetle2_ESP32S3 // Has PSRAM
//#define CAMERA_MODEL_DFRobot_Romeo_ESP32S3 // Has PSRAM
#include "camera_pins.h"

comment out the line

#define CAMERA_MODEL_ESP_EYE // Has PSRAM

and add the line

#define EXPERIMENTAL_CAMERA_PINS

Now this block of code will define our pin-map using the definitions in the header file "experimental_camera_pins.h"

// ===================
// Select camera model
// ===================
//#define CAMERA_MODEL_WROVER_KIT // Has PSRAM
//#define CAMERA_MODEL_ESP_EYE // Has PSRAM  
//#define CAMERA_MODEL_ESP32S3_EYE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_PSRAM // Has PSRAM
//#define CAMERA_MODEL_M5STACK_V2_PSRAM // M5Camera version B Has PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_ESP32CAM // No PSRAM
//#define CAMERA_MODEL_M5STACK_UNITCAM // No PSRAM
//#define CAMERA_MODEL_AI_THINKER // Has PSRAM
//#define CAMERA_MODEL_TTGO_T_JOURNAL // No PSRAM
//#define CAMERA_MODEL_XIAO_ESP32S3 // Has PSRAM
// ** Espressif Internal Boards **
//#define CAMERA_MODEL_ESP32_CAM_BOARD
//#define CAMERA_MODEL_ESP32S2_CAM_BOARD
//#define CAMERA_MODEL_ESP32S3_CAM_LCD
//#define CAMERA_MODEL_DFRobot_FireBeetle2_ESP32S3 // Has PSRAM
//#define CAMERA_MODEL_DFRobot_Romeo_ESP32S3 // Has PSRAM
#define EXPERIMENTAL_CAMERA_PINS
#include "camera_pins.h"

Goal:

• Open "CameraWebServer.cpp" and set the ssid to "CameraWebServer", and wifi password to "CameraWebServerPassword"
• compile and load code onto our ESP32-S3-devkitC-1-N8R8
• set up a wifi hotspot with the wifi SSID "CameraWebServer", the wifi password "CameraWebServerPassword", and the hotspot will use the 2.4GHz Band, same as the ESP32-S3
• read the ip address (--IPADDR--) assigned to our esp32-s3-devkitC-1-n8r8 using the serial monitor or the wifi hotspot connection indicator
• type the address (http://--IPADDR--) into a web browser to see the camera images

NOTES:

We will now test our code to see if it works by

• setting up the microcontroller to use a local wireless network and password, (and setting up a local wifi hotspot with that wifi name and password)
• compiling our code and installing it onto our microcontroller
• Use the PlatformIO serial monitor to read the IP assigned to the microcontroller (--IPADDR---)
• Use a web browser to connect to http://--IPADDR-- to see the streaming images coming wirelessly from the camera.

We want to make sure the base example works before we add more complexity to the code. This is part of debugging and makes quality assurance much easier.

To start this section, we need to modify the code in the "CameraWebServer.cpp" file to include the local ssid (the network name) and the network password. Since we will be creating a temporary local wifi hotspot with a specific password only for this project, we will set the ssid and password to be the name and password for this specific local wifi hotspot. I like using a local wifi hotspot operating in the 2.4GHz band (same as the ESP32 WIFI bands), because

• sometimes we don't have the network passwords, and
• sometimes we are using an enterprise wifi network, and this example is unable to connect to enterprise networks, and
• sometime the available network only operates in the 4GHz or 5GHz bands which the wifi radio-networking-hardware in the ESP32-S3 cannot use..

Here is a tutorial for setting up a wifi hotspot on windows

https://youtu.be/sYC8EY6kJPY?si=XOkDJyRHOKKsOBss

Here is another example of setting up a wifi hotspot

https://www.howtogeek.com/214080/how-to-turn-your-windows-pc-into-a-wi-fi-hotspot/

and here is a youtube tutorial on what is needed to set up an ESP32 to connect wirelessly to an enterprise network (this tutorial is beyond the scope of my tutorial, but thank you Debsahu for your informative tutorial)

https://youtu.be/bABHeMea-P0?si=Kf9g-mpZ19VjbDF2

This microcontroller is set up to connect wirelessly to the network, and let the network's DHCP server automatically assign it a local IP address that you can connect to. The microcontroller is configured to output its assigned IP address (ex: --IPADR-- = 192.168.1.7) over the serial port, and this assigned IP address can be read using the PlatformIO built in serial monitor. Once you have the IP address open up a web browser such as "Internet Explorer, or "Chrome" and write in at the top of the web browser "http://--IPADD--" (ex: if --IPADDR-- = 192.168.1.7, then type http://192.168.1.7 into your web browser)

There are lots of ways to set up the ESP32 microcontrollers to interface with your network, and here is another link to examples shown here

https://www.reddit.com/r/esp32/comments/176tgmz/esp32cam_fixed_ip/

Note regarding loading our firmware to our ESP32-S3 board and monitoring the serial port. This code is set up to use the UART-based USB port which is labelled "UART" on the bottom-left of the board. Connecting to the port labelled "USB" on the bottom-right will not work unless you make the code changes & platformio.ini changes detailed in this link here

https://github.com/espressif/arduino-esp32/discussions/7546

Example: https://github.com/espressif/arduino-esp32/blob/master/libraries/USB/examples/USBSerial/USBSerial.ino

Docs: https://docs.espressif.com/projects/arduino-esp32/en/latest/api/usb_cdc.html

More: https://docs.espressif.com/projects/arduino-esp32/en/latest/tutorials/cdc_dfu_flash.html#usb-cdc

Steps:

Step #1) Get the ssid (network name), and the password for the local wireless network. For this example, instead of using my local ssid and password, I set up a temporary local wifi hotspot "WebCameraServer", with the password "WebCameraServerPassword" on my laptop.

I went to my network settings->mobile hotspot. I right-mouse-clicked on it to go to hotspot settings.

• changed hotspot network name to "CameraWebServer",
• set hotspot password to "CameraWebServerPassword", and
• set network hotspot Bandwidth to 2.4GHz.

Step #2) Open PlatformIO and use the editor to open "CameraWebServer.cpp". Find the section "// Enter your WiFi credentials" and change the ssid and password to ssid = "CameraWebServer", and password = "CameraWebServerPassword"

//#define CAMERA_MODEL_DFRobot_Romeo_ESP32S3 // Has PSRAM
#define EXPERIMENTAL_CAMERA_PINS
#include "camera_pins.h"

// ===========================
// Enter your WiFi credentials
// ===========================
const char* ssid = "CameraWebServer";               //Set the network SSID (network name)
const char* password = "CameraWebServerPassword";   //Set the network password

Step #3) Take your micro-USB cable and plug it into the USB-UART port (the one on the bottom-left labelled "UART" (you currently cannot build the code using platformIO and download to our ESP32-S3 on the USB-OTG port without using the changes detailed in this example here here https://github.com/espressif/arduino-esp32/discussions/7546).

• Note: Plugging in the USB cable powers both the ESP32-S3-DevkitC-1-N8R8, and the Adafruit OV5640 breakout board at the same time.

Step #4) Compile the firmware. Click PlatformIO -> Project Tasks -> esp32-s3-devkitc-1 -> General -> Build.

Step #5) Upload the code to the firmware.

• a) Ensure the MicroUSB cable is plugged into the UART port of the ESP32-S3-DevkitC-1, and your PC.
• b) Click on the PlatformIO TAB:"Serial Monitor", and verify that the serial monitor is NOT running on the COM port being used to program the ESP32-S3. Turn OFF the serial monitor prior to trying to upload the firmware..
• c) Press down on the "BOOT" button on your ESP32-S3-DevkitC-1-N8R8, and hold it down
• d) Click PlatformIO -> Project Tasks -> esp32-s3-devkitc-1 -> General -> Upload. (do not release the "BOOT" button until the PlatformIO starts uploading the firmware into the microcontroller

Step #6) Turn on the PlatformIO serial monitor to see the IP address (--IPADDR--) that the network DHCP server assigned to the microcontroller, or use the Wifi Hotspot dialog box to see the IP address and MAC number of the devices attached to the WIFI Hotspot. Record the currently assigned IP address (if the IP is not set as static, then it will likely change each time it connects to the network).

Step #7) Enter the IP address into a web browser and hit enter. Press the button marked "Start Streaming" to start getting images from the camera. Play with the screen resolution to see how the screen type affects both the resolution and the framerate of the camera. Play with other settings as well.

The attached video shows the camera (WITHOUT AUTOFOCUS) being tested from 1" away to 4" away from the camera at a screen resolution of UXGA(1200x1600)

https://youtu.be/xbaTaryeMDY

Goal:

• Add the autofocus code to the "CameraWebServer.cpp"
• build / upload / test the firmware

NOTES:

We will add the code to "CameraWebServer.cpp" to enable autofocus. The autofocus code is taken from Eric Nam's Autofocus example code.

Steps:

Step #1) Open "CameraWebServer.cpp" and add the line #include "ESP32_OV5640_AF.h"

#include "esp_camera.h"

//ADD AUTOFOCUS HEADER FILE
#include "ESP32_OV5640_AF.h"  //Autofocus library

#include <WiFi.h>

//
// WARNING!!! PSRAM IC required for UXGA resolution and high JPEG quality
//            Ensure ESP32 Wrover Module or other board with PSRAM is selected
//            Partial images will be transmitted if image exceeds buffer size
//

Step #2) Add the Autofocus object to the code by adding the line OV5640 ov5640 = OV5640()

void startCameraServer();
void setupLedFlash(int pin);

//ADD AUTOFOCUS OBJECT HERE
OV5640 ov5640 = OV5640();  //ADD the OV5640 autofocus object

void setup() {
  Serial.begin(115200);

Step #3) Find the line sensor_t *s = esp_camera_sensor_get(); and add the code below to pass the reference of the sensor_t object to the autofocus object, then call the three (3) OV5640 member functions

• "OV5640.start()" ,
• "OV5640.focusInit()",
• "OV5640.AutoFocusMode()".

//FIND THE CODE "sensor_t *s"
  sensor_t * s = esp_camera_sensor_get();

  //ADD THE NEXT nine (9) LINES OF AUTOFOCUS CODE
   ov5640.start(s);

  if (ov5640.focusInit() == 0) {
    Serial.println("OV5640_Focus_Init Successful!");
  }

  if (ov5640.autoFocusMode() == 0) {
    Serial.println("OV5640_Auto_Focus Successful!");
  }
  //DONE ADDING THE AUTOFOCUS CODE

Step #4) In the main loop add the following code to monitor the state of the autofocus camera

//ADD THE CODE FOR MONITORING THE AUTOFOCUS STATE
  uint8_t rc = ov5640.getFWStatus();
  Serial.printf("FW_STATUS = 0x%x\n", rc);

  if (rc == -1) {
    Serial.println("Check your OV5640");
  } else if (rc == FW_STATUS_S_FOCUSED) {
    Serial.println("Focused!");
  } else if (rc == FW_STATUS_S_FOCUSING) {
    Serial.println("Focusing!");
  } else {
  }
  //END ADDING THE AUTOFOCUS MONITORING CODE

Step #5) Now rebuild the code, and upload it into the microcontroller

• (follow the steps in the previous chapter,
• don't forget to shut down the serial monitor and
• hold the "BOOT" button prior to uploading the new firmware)
• see the results using the serial monitor

Step #6) verify the IP address and go to the test page to see the autofocus in action

The youtube link below shows how well autofocus works at different distances using an OV5640 autofocus camera hooked up to a 72 degree lens from 7" to 1" distances.

https://youtu.be/ABWifO8CXTw

As a final step, we replaced the autofocus 72 degree camera with an autofocus 120 degree camera. During testing it worked well and seems to work better at close distances. The streaming would also mysteriously crash at the UXGA setting of 1600x1200

Here is the video below

https://youtu.be/laGQNsrcKXE?si=QPaRDzEvx2wfs2sl

Here's a link to my code on Github

https://github.com/mutantcybertechnologies/WebCameraServerAutofocus/tree/main

If anybody can show me how to use my project to make an autofocusing microscope, I would be very grateful.

Here's an example of what I would like to see.

This is a post by John Lin

https://www.youtube.com/watch?v=SwH3V9Gh5Gw