WebStick - Instant ESP8266 Web Server / NAS in USB Stick Form Factor

Problem

As a full stack developer and a student, many of my friends ask me how to setup their own website for free. Although there are a lot of free web hosting services out there, they are quite picky about service provider and vendor lock in issues. As some of them are also interested in tech stuffs, I will sometime introduce them to self-hosting.

The issue with self hosting a web server is that many of them have no experience in Linux and just putting an old Windows PC with WAMP or Nginx is a waste of space and power. That is why I developed a fun little WebStick for these use case.

What is a WebStick?

A WebStick is a USB stick form factor web server that connects to your home WiFi and allow you to connect it via mDNS in local area network. You can also access it from the internet by changing a few port forward setting in your home router.

As I am a full stack developer who loves to work with NAS, with the limited processing power of this stick, I created a NAS interface for the WebStick as well. It have an ArozOS File Manager like interface with some handful utilities that allow you to store and stream music, video and photo into your webstick's SD card, while allowing you to host static web files like html, css and js into it and provide web programming interface so you can change your website without unplugging the SD card into your PC.

In this instructable, I will guide you through the steps to make this WebStick for under $5 USD each.

As this is a relatively simple build, you don't need much materials to build yourself a WebStick.

• A custom printed PCB
• USB A plug
• CH340C (USB to UART converter chip)
• A few resistors (10k) and caps (100nF & 10uF)
• AMS1117-3.3 (Voltage regulator)
• Micro SD card socket
• LEDs
• ESP12F (or ESP12E, both works)
• Micro SD card (with FAT file system)
• BC817 transistors

You will also need to have basic soldering tools and a computer for flashing firmware into your WebStick.

As the PCB is the main component of this build, you can order your PCB at any PCB manufacturer you prefered. This project is sponsored by JLCPCB. If you have never tried to order a PCB before, please feel free to try out JLC services with the link below.

https://jlcpcb.com/

In my case, I got mine for around $5 - 6 USD (5 pcs), which is perfect for small volume productions for making a few prototypes for fun. The PCB should take around 3 days to produce and another few days to ship to your location.

You can find the Gerber files for the PCB over here on my git server.

https://git.imuslab.com/TC/WebStick

How to Solder The Components?

This step is also quite straight forward. What you need to do is simply prepare all the components you need for the soldering. Then solder them onto your PCB one by one.

As each of the boards takes me around 30 minutes to solder, after a few successful prototype, I decided to go with JLC's PCBA service. If you are a STEM teacher and needs a bunch of these made like I do, feel free to try out the PCBA service as well. I will have another blog post documenting how to mass produce electronics later if anyone is interested about this.

Why the Components are So Small?

The smallest component, the NPN transistors BC817 are there to support automatic programming. In boards that do not contains build-in programmer like the ESP32CAM, you will often see a FLASH and RESET physical buttons. That is because it is required to held those pins to LOW in order to make the chip to enter flash mode. But adding these two components (although they are difficult to solder), it provide a much smoother automatic programming experience to myself, specially when I am developing the backend code which I need to flash a new firmware to it every few minutes just to test it.

It is important that you check if there are any shorts on the PCB before plugging it into your computer. As chips these small are usually found with soldering defects like bridging between pins or worst, short circuiting your VCC (5V from the USB) to ground, it is a good practice to check if your WebStick is well soldered by inserting it into a protected power supply like a proper lab bench power supply.

In my case, I have built myself a DIY USB hub that build in PTC fuse and protection circuits on each USB port. When I insert my prototype into it, the fuse didn't pop, which means I can safely insert it into my computer's USB port!

How Backend Works

As there is no OS on the ESP12F, I need to write my own web server backend which provide similar features to a standard apache or nginx installation. That is why I spent a month of free time to work on a C++ version of a web server that provides features like

• Static Web Server (serving HTML, CSS and JS and other web supported files)
• Directory Listing (like the Apache one)
• Cookie based Login System
• Private and Public storage space
• Upload and Download API (for byte streaming)
• Device Information APIs (like WiFi & Storage info )

You will need Arduino IDE to flash the firmware for the WebStick. You can install the ESP8266 from the board manager and select either the generic ESP8266 module or the Wemos D1 mini as your target board.

Recommended Upload Options

For best performance, I recommend choosing the following two options from the board's drop down menu in Arduino IDE

• CPU clockspeed: 160Mhz
• IwIP Varient: v2 Higher Bandwidth

But if you want to save even more power, you can use the power saving mode with lower CPU clock speed. It is up to your preference and expected number of visitors to your site.

After all the options are correctly selected, insert your WebStick into your computer and click Compile and Upload. After a few flashes on the WebStick's RX LED, the firmware should be downloaded to the board.

Arduino Code

You can find the source code of the backend server here:

https://git.imuslab.com/TC/WebStick/src/master/web-server

WiFi Setup Notes

The WiFi, admin and mDNS settings are loaded from SD card. So you don't need to change anything from the Arduino source code to setup your WebStick. See SD card section below.

For the SD card, you need a relatively old SD card to make this work, as the ESP8266 can only access the SD card in SPI mode, any card that without the SPI mode will not work.

In my case, I am using an industrial 512MB micro SD card with FAT file system.

Download and copy everything from the sd_card folder to your SD card and follow the instructions below to setup your WebStick with a few clicks.

1. Navigate into the cfg/ folder
2. Edit wifi.txt with first line as your home's WiFi SSID and 2nd line password
3. Edit admin.txt with first line as your admin username and 2nd line the admin password
4. Edit mdns.txt and modify it to something you want. For example, the default "webstick" will allow you to access your webstick using "http://webstick.local" later in your mDNS supported web browser.

After you are done setting it up, you can start replacing the html files in the SD card. For example, you might want to swap out the index.html to something you write for your home homepage, or delete unwanted demo folders. However, these folders are reserved and shell not be deleted or renamed.

Reserved Folders

• /www/ - The web root, do not remove this
• /www/admin/ - The admin web root, required for admin functions
• /www/store/ - The private storage folder, can only accessible via Web File Manager after login and will not be served as static web files

Lastly, insert your micro SD card into your WebStick and you are ready to get it running!

Notes

The mDNS name must be unique if you have multiple WebStick in the same network

To start your web server, simply plug your WebStick into a power supply. If you plug it into your computer's USB port, you will see it started to giving out some debug information on Serial @ 9600 baudrate.

At the bottom of the debug info, you will find the reported IP address of your WebStick. In my case here, it is 192.168.1.100. So you can connect directly to it using either the mDNS or IP address by opening the following link in your browser

http://192.168.1.100

or

http://webstick.local

And you will see your homepage shown up.

In the Web UI I developed, it got a build in File Manager where you can do all the file access and operations there. Simply click on the "admin panel" or navigate to /admin (i.e. http://{your_device_ip}/admin) and login with the username and password you set in the admin.txt file.

Afterward, you will see all file contents in your SD card's web/ folder. If you want to open or edit any of them, just double click them and the File manager will pick the correct web apps to open them. Some build in webapps includes

• Music player (recommend < 5MB mp3 files)
• Video player (recommend < 5MB webm files)
• Photo viewer (web supported formats only)
• Markdown Editor
• Code Editor

If you want to edit your webpage without unplugging the SD card, you can do so by directly select (or multi-select when holding Ctrl and click the html files you want to edit) and click Open. The build in Notepad editor will allow you to change your HTML files directly in your browser.

If you prefer editing locally and upload to the WebStick, you can also do it by drag & dropping the file from your local File Explorer (or Finder) into the File Manager to upload it. However, as the ESP8266 is a bit slow when handling the upload, you might need to wait for a bit while the WebStick is processing the upload & writing to the SD card.

In theory the ESP12F can reach a network speed up to 2 - 4 Mbps, what I usually see is more like 400 - 600kbps for concurrent read write. I guess this might be just the SD card issue but I am not sure about it. So what I would normally recommend is not to put large resources (e.g. pictures / css files) into it, and choose to use CDN (for scripts) or compressed image format (like jpeg).

How can I access my website from the internet?

If you want to access your webstick from the internet, you can enable port forward in your home router to forward the external port 80 to your WebStick IP address. As this is not a technical post (and there are lots of tutorial out there teaching you how to port forward to a given IP address), I am not gonna go into details here.

How can I have my own domain?

If you want to access your webstick with a domain name (e.g. http://yourdomain.com instead of http://{your_home_ip_here}), you will also need to purchase a domain from domain name service provider.

Now, you have yourself a tiny USB stick form factor web server ready to use!

This little web server use less than 2W. You will never notice a different in your electric bill even if you plug it into your wall adapter and running 24/7. So have fun hosting your own little personal homepage!