CamCoord - Dedicated Geotagging for Sony Cameras

I had a Sony A6000 a while ago, and geotagging wasn't commonplace then, so my workflow has always been adding geolocation in post-process. A few years ago I switched to a Sony A6600, which can be paired with a smartphone and get location information from the phone; this simplifies the geotagging process but the feature has to be turned on manually from the phone every time, which I often forget. I also don't want my phone to be constantly searching for the camera and sending location information as it drains the battery.

I wanted an external device to replace my phone for sending location information; but at the time, hobby development boards with BLE weren't commonplace and the support wasn't great. I don't have the technical competency to implement low-level BLE functionality, nor can I reverse engineer Sony's BLE protocol, so the idea went nowhere.

Fast-forward to now, there is abundant information regarding Sony's BLE protocol, and even sample Python scripts to send location information to a Sony camera. BLE development boards are easier than ever to get into, so I have no more excuse but to try and realize this project. Here's how I created the CamCoord, a dedicated solution for Sony camera geotagging.

Electronics

• Adafruit Feather nRF52840 Express
• Adafruit Ultimate GPS FeatherWing
• Lithium Ion Polymer Battery

Tools

• 3D printer (Optional) - we will 3D print the housing for this project
• A Soldering iron

Hardware

• Jumper wires
• Some M2.5 screws and nuts

There are three functionalities I need for this project. The development board needs to be able to communicate via BLE, able to hook up to a GPS module, and run on a Battery for a reasonable amount of time. A quick research led me to Raspberry Pi Pico W, we can get a battery SHIM, it has built-in BLE, and runs Micropython, which means I could leverage the Python script that I found, perfect.

Unfortunately, after I got my hands on the Pico W and started developing, there's one aspect I wasn't unaware of - pairing. We need our development board to act as a central device and pair with the camera to be able to communicate with it; however, while Micropython has a BLE library, it does not support pairing (as of the writing of this, development notes are indicating that this feature will be available in the next Micropython release), so I am back to finding another suitable development board.

I came across a powerful microcontroller that does what we need - nR52840. AdaFruit has a Feather board based on this microcontroller, the Adafruit Feather nRF52840 Express. The feature-packed development board supports CircuitPythin, and has built-in BLE, a built-in Battery charging circuit, and is compatible with the Adafruit Ultimate GPS FeatherWing; it ticks all the boxes. It also has a built-in RGB LED, which we can leverage as a status light. I also picked up a 3000mha Lithium Ion Polymer Battery (the link is for 2500mAh, I got my battery from a different source).

We are using CircutPython for this project so we could leverage some example Python scripts, and Arduino as a backup option if it has better support for my requirements. I followed the instructions here to set up my Feather for CircuitPython.

The goal for this step is to make sure that we can send location data to the camera via BLE, we will ignore the GPS module for now and send mock data to the camera. Completing this step would iron out a lot of uncertainties for this project, mainly the BLE connection and pairing process and data format Sony camera expects. To understand what we are doing at this step, it's helpful to have a little knowledge of how BLE works, here is a good primer. Our development board acts as a central device, and here are the steps to send information to our camera:

1. Connect to the camera directly using the camera address
2. Pair/Bond to the camera
3. Get location service information from the camera
4. Get the characteristic used for writing data from the location service
5. Write our mock data to the camera, here is the protocol and mock data we are using.

import time
import binascii

from _bleio import Address
from adafruit_ble import BLERadio
from adafruit_ble.uuid import VendorUUID, StandardUUID

LOCATION_SERVICE = VendorUUID("8000DD00-DD00-FFFF-FFFF-FFFFFFFFFFFF")
LOCATION_SERVICE_WRITE_CHARACTERISTIC = StandardUUID(0xDD11)
CAMERA_ADDRESS = Address(binascii.unhexlify("6e7faca44dd4")) # get address from camera menu under bluetooth, flip the value

# 1. connect to the camera directly
radio = BLERadio()
connection = radio.connect(CAMERA_ADDRESS)

# 2. pair with the camera
connection.pair()

# 3. get location service
loc_srv = connection._discover_remote(LOCATION_SERVICE)  # location service

# 4. get write characteristic under the location service
loc_w_char = [char for char in loc_srv.characteristics if char.uuid == LOCATION_SERVICE_WRITE_CHARACTERISTIC][0]

# 5. write mock data to camera
data = binascii.unhexlify("005d0802fc0300001010100bf79e5e41c385a707e40b0504022a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001e00000") # mock data lat: 20.077731, long: 110.3332775, date: 2020-11-05 04:02:42, UTC +8, no DST
while True:
  loc_w_char.value(data)
  time.sleep(1)

The first picture shows the camera in shooting mode (lens cap on), both the Bluetooth location indicators are lid up, meaning the development board is connected to the camera and the camera is receiving location information. The second picture shows the camera in gallery mode, a photo I just took displaying the mock location and date information I sent. Neat!

Now we are going to get real data from GPS; first, we need to wire it up to the Feather board. I am not stacking up the Feather and the FeatherWing because I want to keep the profile low for the final design. As a result, I am using jumper wires to make the necessary connections. Here are the pinouts to the ultimate GPS FeatherWing, at minimum, we need to connect the power pins and serial data pins.

There is no uncertainty for this part, the Adafruit instructions and example are very good, loading the following code and we started getting GPS data.

import time
import board
import busio
import adafruit_gps

# setup GPS
print("set up GPS")
uart = busio.UART(board.TX, board.RX, baudrate=9600, timeout=10)
gps = adafruit_gps.GPS(uart, debug=False)
gps.send_command(b"PMTK314,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0")
gps.send_command(b"PMTK220,1000")
last_print = time.monotonic()

while True:
    gps.update()
    current = time.monotonic()
    if current - last_print >= 1.0:
        last_print = current
        if not gps.has_fix:
            # Try again if we don't have a fix yet.
            print("Waiting for fix...")
            continue
        # fixed
        print("=" * 40)  # Print a separator line.
        print(gps.timestamp_utc)
        print(gps.latitude)
        print(gps.longitude)

Once we start sending the GPS data to the camera, we have a functioning device! However, there is one problem - timezone. The Sony BLE protocol allows us to send timezone and daylight saving time (DST) information along with the location information, the camera will then update its settings using the timezone and DST information. That way our photo will be tagged with proper local time, and we don't have to worry about changing the timezone setting on the camera. However, the timezone and DST information is not readily available from the GPS unit, we would have to use the latitude and longitude and look up the timezone and DST information. There are CPython library that does the conversion, however, it's not available on CircuitPython. Besides, the timezone information dataset is huge, it's around 80MB and would require around 40MB of RAM to perform the lookup. We don't have enough resources on our development board (2MB flash memory and 256KB RAM) to perform the lookup. I have a couple of ideas to work around this problem for future revision. For now, we will need to manually set the timezone and DST information on our camera, it's a little inconvenient but it works.

Next, it's time to polish up the user experience. First, we are going to get the onboard RGB LED to work. The onboard LED is a single NeoPixel, we can manipulate its colour and brightness. neopixel.fill() is used to change the LED colour, and we can use neopixel.brightness to set how bright the LED is, from 0 being off to 1 being maximum brightness. Once we set the colour and brightness, we need to make sure to call neopixel.show() to apply the changes to the LED.

import board
import neopixel
import time


RED = 0xFF0000
GREEN = 0x00FF00
BLUE = 0x0000FF
PURPLE = 0xFF00FF
YELLOW = 0xFFFF00


colors = [RED, GREEN, BLUE, PURPLE, YELLOW]


pixel = neopixel.NeoPixel(board.NEOPIXEL, 1 , brightness=0.008) # 0.008 is the dimmest from experiment 


count = 0
while True:
    # pixel.brightness = 1.0 # we can change the brightness like this, 0 is off
    pixel.fill(colors[count%5]) # set color
    pixel.show() # apply the change to LED
    count+=1
    time.sleep(1)

With the LED working, we can use LED to notify the user of the status of the device. The current implementation of LED control is linear with the rest of the code, meaning the traditional way of using a sleep delay to blink the LED will block the rest of the flow, hence we are only turning the LED on or off instead of blinking.

I have also implemented the retry logic so the unit would run continuously, even if the device lost connection to the camera. The whole flow is shown in the flow chart. Because the camera goes to sleep after inactivity, our device will continue to try and reconnect indefinitely. To save power, I implemented a retry delay, on the first 5 retries the delay is 1 second, and for any subsequent retries, the delay is 5 seconds. To use the device, we need to power it on when the camera is on, so the unit finds the camera. When the connection is established, we no longer need to interact with the device. The device will work in the background, connecting to the camera as it powers on or wakes up, and sends geotagging information.

We had a gorgeous day, I decided to go out for a hike and test the prototype out in a real-life scenario. I plugged in the battery and used a rubber band, a piece of cardstock, and a ziploc bag to hold the device together temporarily. Then I connected and paired the device with the camera, I saw my device flashing purple (sending location information) and verified that I was getting location information on my camera, I threw the device into my backpack and turned my camera off. During the hike, when I turned my camera back on, after a few seconds, the connection was re-established and location information started to stream in. Throughout the hike, the camera went in and out of the sleep state, but whenever the camera woke up, the connection would be re-established. During the whole trip, I didn't have to fiddle with the device, truly set and forget. I was out for about 6 hours, and the device has been running in the background for just as long, the battery seems to be holding up well. Overall it's a success, the device is functioning as I intended. However, here are a few issues I still need to work out:

1. Reconnection could take up to 30 seconds after the camera wakes up, this is likely due to the retry delay. I would like the experience to be more snappy, likely at a cost of higher power consumption.
2. I don't have a way to power the unit off. The unit stays on all the time. While the expectation is I would connect the unit with the camera at the beginning of the shoot and leave the unit on, I still like a way to shut it off at the end of the shoot.
3. I am not sure how much battery I have left, I would like to know when my battery is low and needs to be recharged.

The first problem is a quick fix, after modifying the code to remove the retry, the reconnection time is down to about 10 seconds max, and as fast as 5 seconds.

Our development board comes with a switch we can use for this project. Unlike Arduino, CircuitPython doesn't support hardware interrupts but leverages the concept of cooperative multitasking to achieve similar behaviour, more information can be found here. The challenge with this approach is that I now need to convert/wrap all my code from the previous steps to leverage asyncio. I gave it a quick try and converting the adafruit_ble's scanning function into async generator seems to be more complicated than I thought, more research is needed. I want to get this project to a working state first before I come back and address this, so for now, we will keep the code linear.

The downside is that if the code can be blocked in a certain state, such as scanning or trying to connect to the camera. When the code is in this state, the press of the switch cannot be detected. So to send the device into the deep sleep state, we would need to press and hold the switch until we see an indication that the unit is going into deep sleep. When the device is in a blocking state, I would have to wait up to 5 seconds; but when the device is sending location information, the response is almost instant. The inconsistent behaviour is the trade-off to keep the code simple for now.

Here's the idea - when the program is sending data to the camera, it's running in a loop, this loop executes fast; at the beginning of each loop, we check if the switch is pressed. If the switch is pressed, we will initiate deep sleep. To wake up from deep sleep, before we enter deep sleep we need to set an alarm on the switch, if the switch is pressed when the board is in a deep sleep state, the alarm will wake up the board and restart the flow from the beginning.

One thing we need to pay attention to here is we cannot use the switch as an input pin and alarm pin at the same time. As a result, during the initialization phase, we will set the switch as an input pin, so we can detect it while the board is running. Before we go into deep sleep, we will detach the switch as an input pin and bind it to the alarm. In the deep sleep state, pressing the switch will wake up the board and it will go through the initialization phase again, the switch will be initialized as an input pin again.

import board
import alarm


from digitalio import DigitalInOut, Direction, Pull


# set up the switch to send the baord into deep sleep
power_switch = DigitalInOut(board.SWITCH)
power_switch.direction = Direction.INPUT
power_switch.pull = Pull.UP


while True:
    # send location to camera
    if power_switch.value == False: # button pressed
        # blink the LED red to indicate we are going into deep sleep
        power_switch.deinit() # detact the switch
        wake_up = alarm.pin.PinAlarm(pin=board.SWITCH, value=False, pull=True) # bind the switch to alarm to wake up the board
        alarm.exit_and_deep_sleep_until_alarms(wake_up) # go into deep sleep

An updated flowchart with new and modified state highlighted is attached, it shows the complete flow of our code so far.

I picked up a 3000mAh battery for this project, thinking a bigger battery would allow the device to run for longer. What I did not take into consideration when selecting the battery is the charging speed. The out-of-box charging rate of the onboard charger is 200mA, meaning it will take about 15 hours to charge our battery fully. Fortunately, it's possible to increase the charging speed to 1000mA, or 3 hours to fully charge our battery, by replacing a resistor, it's a very delicate job so we will add this to the backlog for the next revision. Now, I'd like to have a way to see how much battery I have left, and Adafruit already has this problem solved here.

import board
import time

from analogio import AnalogIn

vbat_voltage = AnalogIn(board.VOLTAGE_MONITOR)

def get_voltage(pin):
    return (pin.value * 3.6) / 65536 * 2

while True:
    battery_voltage = get_voltage(vbat_voltage)
    print("VBat voltage: {:.2f}".format(battery_voltage))
    time.sleep(1)

The plan is to leverage the same power button, if we short press we show the battery level using LED, but if we long press we manage the deep sleep state. However, for this to work, we need to refactor our code to leverage asyncio. We will make that a future enhancement; for now, let's keep this code snippet in the back pocket until then.

At this point, we have a fully functioning prototype, it's time to design a case and get rid of the rubber band and Ziploc contraption. The sky is the limit here, or in my case, my artistic talent and CAD skills are the limiting factors here. I want the device to be compact and needs to expose the USB port, status LED, power button, and reset switch. As you can see, I went through multiple revisions, trying to get all the dimensions correct, ports and screw holes to line up, as well as dealing with failed print due to bad bed adhesion and terrible surface finish. While not perfect, I am pretty satisfied with the result (except for the colour, this is the only filament I have available).

From the image you can see I am using a momentary switch as my power switch instead of leveraging the built-in user switch, it's because the built-in user switch is used by the bootloader until the boot is completed, and wakes up from deep sleep goes through the boot sequence so we cannot use the built-in switch as wake up pin. This issue only surfaced when it was running on battery power because in development mode the board only pretended to be in deep sleep.

I contemplated different carrying options:

1. Cold shoe mount (sit on top of the camera)
2. Tripod mount (attached to the bottom of the camera)
3. Strap mount (camera strap, backpack trap)
4. Clip/Carabiner mount
5. Magnetic mount
6. Standalone (throw in the backpack)

While I am delighted with how this project turned out, I am already considering the next revision. Here are some of the improvements I would like to incorporate:

Hardware

• Size - the device is larger than it needs to be, here are a few things I am considering:
• Custom PCB - remove unused components, reduce the footprint
• Snap-fit the enclose to eliminate room for screw
• Use M1 screws instead of M2.5 to reduce space needed
• Smaller battery
• Better PCB-battery orientation, maybe side by side to reduce the thickness
• Weatherproof - a seal ring along the seam and a cover for the USB port and power button.
• Battery - For the prototype I picked a big battery, I would like to measure the actual power consumption, then pick a suitable-sized battery, and ensure the charging circuit can charge it up in a reasonable amount of time.
• Finish - maybe it's because my 3D printer is old, and the enclosure surface is hideous. I'd like the case to look more polished, maybe CNC the frame and mould the top and battery cover.
• Screen? - An e-ink screen instead of LED will be cool but not sure if I want the added cost and complexity.

Software

• Arduino - It seems Arduino operates at a lower level, so I have more control over the LED and using hardware interrupt, etc.
• Asyncio - If I stick with CircuitPython, I need to get the concurrent process to work so I have more flexibility with LED light patterns and reacting to button press
• Battery indicator - As mentioned in the battery section, getting asyncio to work would allow me to leverage the power button for more than just putting the device into deep sleep.
• Timezone - this is my most wanted feature, I want to simplify the timezone polygon data (reduce accuracy near the boarder) and try to fit it on the microcontroller. With timezone data working my device will tell my camera what the timezone offset is, not needing me to manually set it.
• Multi-camera support - while I only have the one camera, I can see this being useful for some people. For now it's low priority for me.

I want to acknowledge the good information that helped me out a lot with this project, I want to thank these authors for recording and sharing their knowledge and experience, that's what made this project possible:

• https://gethypoxic.com/blogs/technical/sony-camera-ble-control-protocol-di-remote-control
• https://blog.jonasbengtson.se/freemote
• https://github.com/anoulis/sony_camera_bluetooth_external_gps
• https://github.com/whc2001/ILCE7M3ExternalGps/blob/main/PROTOCOL_EN.md

Also a big thank you to the AdaFruit community on Discord, I got a lot of help from there as well.