Auto Drift RC Car

We (Josh Samulak and Arturo Paras) made an RC car that automatically detects pink arrows and the direction their pointing in, then drifts in that direction.

We made this for our controls final project, but also because we thought it would be really cool to make an RC car that drifts.

Left photo:

1. Disassembled Citroen C3 RC Car (any RC drift car works)
2. Disassembled Radio Transmitter (comes with the RC car, shown disassembled above)

Top right photo:

1. Arduino Uno R3
2. Breadboard
3. Disassembled Radio Transmitter (again)

Middle right photo:

1. 2 Digital Potentiometers (zoomed in on the breadboard)

Bottom left photo:

1. Raspberry Pi 5
2. Raspberry Pi Camera Module 2 (Module 3 works better because you don't need the cable below)
3. Raspberry Pi Camera Cable Standard to Mini | 200mm (adapter cable for camera Module 2 to Raspberry Pi 5)
4. Raspberry Pi Charging Cable (comes with Raspberry Pi 5)

Bottom right photo:

1. Pink Arrow for CV Detection

In order to make our RC car drift, we need a nice way to control it. A person would use the radio transmitter that comes with the car to control the throttle and steering, but we want AUTOMATION. This means we need some way to have a computer control the throttle and steering instead, but how?

Our initial idea was to control throttle by sending signals directly to the car's onboard Electronic Speed Controller (ESC), which sends voltages to the motor, making it move; and to control steering by sending signals directly to the steering servo. However, this is a very invasive approach, and would likely take more time than we had. Plus, we already have a method of communicating to these pieces of hardware through the radio transmitter...

We decided instead to rip apart the radio transmitter and hack its components. It consists of one physical potentiometer for throttle, and another physical potentiometer for steering, so all we have to do is swap these out for our own digital potentiometers which a computer can control and voila! We can automate.

To do so, we first need to figure out some inputs from the world that a computer can detect and make decisions from. For this, we decided to have a pink arrow that a camera can detect. If we don't see an arrow, then we go straight, otherwise we drift in the direction of this arrow.

Now we need a way to get a camera onboard the car. We decided to use a raspberry pi, because we can easily attach a small camera to it. Plus, it's a tiny computer itself, meaning it's capable of doing the arrow detection onboard the car!

All that's left is to communicate the arrow information back to the arduino, which is not on board the car. This means we need some form of wireless communication. We do this by connecting another computer to the arduino (laptop, PC, anything). The computer acts as a proxy, receiving signals wirelessly from the raspberry pi via wifi, and sending them to the arduino via serial communication.

Now that we understand the setup, we can get into the details.

We need to collect some data to understand how we can make our digital potentiometers control our car. It's much easier to perform this data collection with the car and radio transmitter intact, so don't rip it apart yet.

When opening the hand-held controller, we see that the user input actually just boils down to two potentiometers. The first is for the trigger, and controls movement forwards and backwards. The second controls steering.

Note that when you are not touching the controller at all, the potentiometers are in a position that divides the voltage roughly equally. Using a voltmeter, get the value of this "base state" for your controller. You will use this to calculate at what step your digital potentiometers should reset themselves to to start (since it won't trivially be zero). As an example, my potentiometer has a wiper to ground value of 1.23 in its resting state, and goes up to a 2.8 when directing the car to go forward, and 0 when directing the car to go backward. The steering potentiometer had 1.63 resting (straight), 2.8 right, and 0 left for it's respective values. Your highest values should be close to the full voltage across the potentiometer (high to ground), which may vary between RC cars you choose.

If you don't want to do all that, honestly, I'm pretty sure you don't need to be this precise with the values, so it should be fine. In retrospect, it seems the steering is just straight if the ratio is about a half, then gets gradually turned more right or left as the ratio goes up or down. The throttle is a little more complicated. Since the motor can't do anything below a certain threshold of voltage, the car uses the value below a certain threshold to tell the car to go backwards, instead of it being just a low value of forwards. For our car, if it was going forwards, then you push the trigger out to tell it to go backwards, it will just break. You need to re-center the trigger, then push it out to go backwards again, and then it will actually go backwards. Not any similar oddities in your own car's controller as you try to automate it.

Remember, as far as the controller is concerned, there are only the two potentiometers for input. So, all we have to do to insert our own inputs is to swap out the physical potentiometers with our own digital ones that we can control automatically. That is our first goal.

To get there, first test all your connections by breadboarding in a different pair of potentiometers. You should get results similar to that of the attached video. I personally made the connections by first fully removing the old potentiometers and soldering on pin headers in their place. Then, I simply used jumper wires from the pins onto a breadboard, where I placed some other 10K potentiometers I had lying around (what resistence value the potentiometers are doesn't really matter, the controller only cares about the resulting voltage ratio that is produced by them).

Now that you have easy access to the controllers input system, we can easily swap out the potentiometers you used for testing with digital potentiometers! Above is an image of how this looked for us.

For the ones we used (listed in the materials section), here's the corresponding data sheet.

Detailed in the data sheet is how we need to interface with the chip to get it to do what we want, that being control the car. Note the number of steps whatever digital potentiometer you choose is capable of. You may want to test that it behaves as expected. For our project, we had to use two different digital potentiometers, which is why our variables in code may be a little weird for you looking through them. The potentiometer we used for the steering only had 34 steps (STEERING_MAX), while the motor had a finer number of 100 steps (MOTOR_MAX). Whatever you change those variables to, you should also change their corresponding resting variables: STATIC_MOTOR_STEP should be about 55-60% of the max number of steps. STATIC_STEERING_STEP should be as close to half of the STEERING_MAX as possible.

On your end, I don't think any other modifications to the code will be necessary other than those variables (if you use a digital potentiometer at least within the same family as the ones we did). See the last steps for all the Arduino code.

Now that we hacked into the RC car, it's time to automate! But before we can look at the world around us for our pink drift arrows, we need to set up our communication network to tell our arduino how to control the car. This is where the raspberry pi comes in, and we'll also need to use a laptop, PC, or any other programmable computer. As described in Step 1, we'll treat this extra computer as a proxy between the raspberry pi and the arduino. Our results are shown in the video attached below.

To have the raspberry pi send signals to the computer, we'll have them communicate over wifi. We'll start a server on the computer, and the raspberry pi will be the client. In this setup, the client sends the arrow information to the server, and the server's job is to forward this information to the arduino.

Connect the computer and the raspi to the same wifi network. Start the server on the computer, and choose a port (we chose 5000). Figure out the computer's IP address on this wifi (on Linux, open up a terminal and enter the command "hostname -I"). Connect the raspi to that IP and port, and you will be connected to the server. Now the raspi can send signals to the computer via the server!

Next, we need to have the computer forward these signals to the arduino. We can connect them via USB, and initiate serial communication between them. Now, anytime the computer receives a signal from the arduino on the server, it can immediately forward it to the arduino over serial communication!

The code on the computer end is also shown in the last steps. Unfortunately, we no longer have access to the code on the raspberry pi, because I (Arturo) left the raspi in New York, and neither of us are in New York anymore. However, setting up a client in Python is very standard and instructions can be easily found on by using the internet.

Now that the entire framework for automatic control is set up, it's time to take inputs from the outside world! Again, we're just looking for pink arrows pointing us in the direction to drift. To do this, we'll attach our raspi camera module 2 to our raspi, and continually take images and attempt to detect pink arrows in those images. If we find one in an image, we'll send the direction it's pointing over the server, otherwise we'll send a command to keep going straight. The results are shown in the attached video below.

Detecting the arrows is actually quite simple. For each image we capture, we'll put it through the following pipeline:

1. Convert the image from RGB to HSV. This makes thresholding (step 2) easier, because combinations of hue, saturation, and value has a much more intuitive translation to a color than combinations of red, green, and blue.
2. Threshold the image such that pixels that match the color of the pink arrow are white, and all other pixels are black. To do this, you'll have to choose a range for which hue, saturation, and value are valid. For example, we chose a hue range of about 165 to 180 to be valid, in order to detect pink. We tune these ranges through trial and error. Our approach was to start with all ranges full. We slowly brought up the minimum value of hue until we began cutting out peices of the arrow. Then we brought down the maximum value of hue until we saw the same happening. We do the same for saturation and value. Then we take more images at varying lighting levels and distances, again adjusting our ranges to ensure none of the arrow is cut out.
3. Put bounding boxes around all blobs of connected white pixels. If your thresholding is good, this should give you few boxes around tiny blobs that don't correspond to the arrow, and it should give you a large box around a blob that looks exactly like your arrow if the image contained an arrow.
4. Take the bounding box with the largest area, and threshold it against a minimum area. If your area threshold is good, and again, if your HSV thresholding is good, then: if the image contains an arrow, you should get the box around that arrow; if the image doesn't contain an arrow, you should get no box.
5. If you have no bounding box at this point, and all your tuning is good, then the image should have no arrow. Thus, you send a signal on the server to move forwards, and move on to the next image. Otherwise, you should have a box around your arrow, and it's time to detect the direction it's pointing in.
6. Crop the image to the size of the bounding box.
7. Scan each row of the cropped image. As soon as you encounter a white pixel, check if it's on the left side or right side of the cropped image. Whichever side it's on is the direction the arrow is pointing, so send a message on the server with that direction.
8. Now that you detected a direction, you want to drift. But if you turn even for a tiny bit, the arrow might go out of view of the camera, causing you to stop steering before you can slide. To solve this, continue sending the direction you detected over the server for about 2 seconds before moving on to the next image.

Again, unfortunately the code for this is on the raspi, which we do not have access to. However, I modeled this approach off of code that I wrote in the past, which is attached in the last steps, and is very similar to the code on the raspi.

A run of our car auto drifting is attached below. As you can see, our car does drift, but it crashes into the wall instead of fully making it around the corner. To improve our performance, we could have figured out the exact distance it takes for our car to full start driving in the drift direction, and somehow set that to be the distance it detects the arrows from. Then we could time how long it takes to drift that distance, and have it drift for that amount of time. This way, the car could drift around corners, and potentially run a track!

Additionally, we could likely remove the computer proxy between the raspi and the arduino. We would need to attach a wifi module to the arduino, and then we could set up the server directly on it. Still, this approach suffers from requiring an area with wifi to work. Instead, we could attach a radio transmitter to the car, and a receiver to the arduino, and then we'd have radio in both directions between the car and arduino!

Here's all the code we wrote for this project in zip files (except for the code on the raspi, since I (Arturo) left it in New York, and neither of us are there, whoops).

1. Arduino code: drift_car_control.ino
2. Computer server: drift_server.py
3. Computer serial communications with arduino: arduino_serial_comms.py
4. Computer vision code that I've written previously, that I modeled this car's computer vision off of: cv_model.py