Do It Yourself Tracking Software

Introduction

In a preview I showed a radio controlled boat where I hacked the remote control. With an Arduino Uno you can send a signal to the remote which then sends it to the boat. A small video of my daughter trying it out to show it works.

My ultimate goal is to make the boat autonomous. I want to use a webcam to determine the position of the boat and then be able to steer in the right direction. So that could be making sure it doesn't crash into the wall or that it follows a certain route, I haven't made up my mind about that yet.

Choosing the system

When I started with this project I was thinking about all the obstacles and steps I would find along the way. But I soon realized that I was making the rules here. I could choose what system I was using and how difficult it was going to be. That sounds a bit vague so I guess I'll give an example. When I saw the remote control I immediately was thinking about the Arduino. Nowadays there are quite a few ways for computer-machine interaction but since I have some experience with the Arduino I knew I would have less problems implementing it.

One of the bigger challenges was using a webcam and it's images to send the right commands to the Arduino and so the boat. I followed part of a course last year on Coursera called: Creative Programming for Digital Media & Mobile Apps. All the apps they worked with were built in Processing which has a very low entry level and let's you make Android apps and JavaScript apps. It's really meant for fun user interfaces.
Bonus: It's really similar to the Arduino language and even the programs look similar. So if you know one, the other will be a cinch.
And yes you can access the webcam with processing and use the images to do whatever you want.

Object tracking

That was a long intro, bear with me.
Processing doesn't have object recognition functions built in and I figured any program you could find on the internet would not be suitable for what I wanted. So I decided to make it myself which is always more fun.
Accessing the webcam wasn't that hard (there are examples in Processing) but I needed some boundaries.
What did I want to recognize? The boat, but I didn't feel like building a Machine Learning algorithm that knows what the boat looks like and finds. Possible I guess, but pretty hard work. I wanted something simple.
I made the choice of placing a colored sticker (or maybe two) on the boat, let's say red. I intend to place the webcam above the water and I can choose what color bottom of the water will have. I could make sure that the only red thing in the webcam image is the red sticker. This is what I talked about earlier, I can make the choices of how difficult the problem is going to be. I have reduced the problem to finding a red dot in an image.

Finding the red dot

To give you a break from all this reading, here is a demo of my Processing program using the webcam on my laptop to follow a small red object.

In my hand I'm holding a small piece of red plastic. The program takes a webcam image and analyses it in two steps.
First it performs a threshold. Using the fact that red objects are red :) it assigns a '1' to pixels that have lot's of red and little green and blue (i.e. white pixels have lot's of red, green and blue) and it assigns a '0' to all the other pixels.
Next it calculates how many pixels in the neighborhood are found red and finds the pixel with the most neighboring red pixels. That neighborhood can be quite big, for instance this was done with 10 by 10 pixels.
The way these calculations are done can and will be topic of a separate blog entry.

To show the result it draws a white circle at the found position.
You can vaguely see image it works with around the white circle. That represents the number of pixels around every pixel. If you would have a stepfunction in your image the result of counting the neighboring pixels would look like something below.

Sketch of how the counting of neighboring pixels would look like

It reminds me of a Gaussian blur. Most important is that you see that the middle of the original platform results in a high peak in the second image. Important is choosing the right neighborhood size as you can imagine.

In the near future

Keeping the main goal in sight, tracking and controlling the boat there is still some work to be done.
I intend to place two stickers one the boat, a blue one and a green one and track them both. With the position of these two stickers you can determine the position and orientation of the boat. If you use previous positions and orientations you can calculate the speed and direction of the boat.

Then I have to think of some rules the boat has to follow like: don't touch the walls, or go around in circles.

So far the program is perfectly capable of doing all the calculations in real time, and I hope that it still does so when all these features are added.

To be continued...

The boat