Animatrack - Object Tracking Camera

Our team selected a technology created by Disney to have animatronics that can make eye contact. The technology was detailed in this article about a patent filed by Disney. The technology detailed in the patent is extremely complex so the purpose of this project was to make a simple demonstration of the technology that can be implemented by a high school student and within a $100 budget. To make this happen simplifications were needed. First, an entire animatronic setup was out of the question so our team opted for a simple gimbled camera setup. After much research the team determined facial recognition would be difficult to implement within the project constraints so instead of tracking a face the project would track an object that the user could select. This sufficiently demonstrated the technology and could a provide a user with some fun skills in the project field.

• Pixy2 Camera: https://pixycam.com/pixy2/
• Pan tilt assembly: https://pixycam.com/pixy2-pan-tilt-kit/
• Computer with USB port and runs Windows vista or newer, MacOS, or Linux
• Small Phillips head screwdriver (Example)
• needle nose pliers (Example)
• Side Cuts (Example)
• Heavy-ish object for base

Note: there are two versions of the Pixy2. One with a white cover over the camera and one with a black cover. They are functionally the same so either will work for this project.

There are two assembly instructions below. One is the Pixycam pan tilt assembly instructions and the other is a video the team produced demonstrating the written instructions. Use whichever option you prefer to assemble the mechanism.

Here is a link to a video walkthrough: https://youtu.be/nSN8B20ZbqQ

Here is a link to text instructions by the makers of the Pixycam:

https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:assembling_pantilt_mechanism

The next several steps will relate to the setup and tuning of the software. The video included shows some of the basics setup but the next few steps will go into more detail. The software installation page is here and the Pixycam support page is helpful in debugging any issues with software setup. The next steps detail the relevant software parameters.

It is important to note that for this project we are running the pan tilt demo program that is included in the PixyMon software.

If at any point you are having too much trouble getting the pan tilt software to work you can reference this link to try and debug any issues.

Finally, it is important to choose a good object to track. The object should be a consistent color that has good contrast with the background color. It is also better if the object is not an irregular shape.

Once you have the PixyMon software installed you can use the gear icon to open the configure window and look at the next few steps.

Since the computer vision algorithms are very specific to environment your settings will likely have to be different than our settings to get the best performance. We are still going to include the file for our settings but do not expect it to work perfectly without a little tuning.

The attached file is the PixyMon parameters file. Once downloaded then file extension needs to be changed to .prm

Here are methods of changing file extensions for Windows and Mac (video).

Finally, there is a glossary in Step 11 that contains a little more detail about some of the terms used.

The main parameters to adjust here are the Pan and Tilt gains if you servos are acting erratic and lose the object or if it cannot keep up with a fast object. The Pixy Pan Tilt uses a pd control loop so the parameters to control are the p and d values. The easiest method of tuning is just playing around with the values of the gains and seeing what happens.

If you are interested in learning more about pd controls check this out.

If you would like to study up on the setting to maximize tracking ability these are expert settings. We did not find it necessary to adjust these settings but they do impact the tracking ability so feel free to investigate them if you would like.

This setting allows you to name the objects you are detecting. This is more helpful when identifying multiple objects so it is not needed for our project. But if you would like you can add a label.

We kept the camera parameters the same but if there is an issue with lighting noise the frames per second can be slowed down.

These parameters are used for serial data transfer.  We did not use this in the project, but it would be cool to interface with a pick and place robot since you can get analog X and Y tracking data. If you are itching for more then try exploring this option.

Sets the travel limit of the servos and the servo frequency. These are fine if left the way they are. They can be used to limit the travel distance if you do not want to the camera to turn as much.

Time to get creative and make a face for your camera! You can draw or print whatever you want to make a face or whatever else you want to do. Hopefully your artistic ability is better than that of an engineering student.

Proportional Gain: will control the speed of the servos. If your servos aren't moving fast enough, try increasing the proportional gain.

Derivative Gain: will help ensure that you don't overshoot or oscillate. If your servos are oscillating, try increasing the derivative gain. If increasing the derivative gain doesn't reduce oscillations, try reducing the proportional gain and reset the derivative gain to 0, then try increasing the derivative gain again.

Signature # Range: This setting determines how inclusive colors are to each other such as different shades of the same color.  Narrowing the range may decrease false positives in the background..

Block Filtering: if a given block is likely to be valid or just a false positive detection. An increase will reduce false positives but increases latency.

Signature label #: You can name your taught signatures and the name will appear in the programming window over the top of the detected object.

PixyCam references:

"Products – PixyCam", Pixycam.com, 2021. [Online]. Available: https://pixycam.com/products/. [Accessed: 15- Dec- 2021]

"Pixy2 Pan-tilt Kit – PixyCam", Pixycam.com, 2021. [Online]. Available: https://pixycam.com/pixy2-pan-tilt-kit/. [Accessed: 15- Dec- 2021]

"wiki:v2:overview [Documentation]", Docs.pixycam.com, 2021. [Online]. Available: https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:overview. [Accessed: 13- Sep- 2021]

Project Idea References:

M. Baker, "Disney Patent Application Shows How "Project Kiwi" Free-Roaming Groot Animatronic Makes & Maintains Eye Contact - WDW News Today", WDW News Today, 2021. [Online]. Available: https://wdwnt.com/2021/05/disney-imagineers-developing-animatronics-capable-of-making-maintaining-eye-contact-while-moving/. [Accessed: 10- Sep- 2021]

Disney Enterprises, Inc., "Eye Contact Sensing and Control for Robotic Characters," US 11,072,075 B2, Jul. 27, 2021.

References from development:

These link to some ideas that were explored but we ended up going a different route but they make for an interesting read, especially if you want a more complex version of the project.

Rovai, M., n.d. Automatic Vision Object Tracking. [online] Instructables. Available at: <https://www.instructables.com/Automatic-Vision-Object-Tracking/> [Accessed 9 September 2021].

"About - OpenCV", OpenCV, 2021. [Online]. Available: https://opencv.org/about/. [Accessed: 15- Oct- 2021].