Computer Controlled RC Car


This page describe's Nasa Tjan's final-year project. He wrote some computer software to steer a radio-controlled car towards a goal. The software sees the car through an image captured from a webcam. Nasa Tjan completed this under my supervision during the final year of his computer science degree at the Malaysia Campus of the University of Nottingham. The custom hardware was built by Ellias Ismail.


Here's some video footage of the car in motion, with a voiceover that explains briefly how it worked.


There are five basic 'hardware' components to the system, 'connected' in a loop, described below.

  • The most important is the software, written by Nasa Tjan. Written in java, there are classes and methods to send the car commands such as "move forwards", "turn left", and so on.
  • These commands are actually sent to the interface board, from "Go forwards" actually means "switch digital output 0 on for 250 milliseconds". provides code that allows their boards to be easily programmed from many languages, including java. They claim it makes it easy (at last) to connect a computer to hardware. We found this claim to be mstly true.
  • The digital outputs of the 8/8/8 PhidgetInterfaceKit are connected via transistors and relays to the car's remote control. Thanks, Ellias, for doing the electronics for us! Without you, this project would have been doomed!
  • The remote control drives the car in the normal way, the relays taking place of human thumbs. The car is a cheap made-in-china job, off the shelf of a department store. It cost about 30 or 40 ringgit (equivalent to about 10 US dollars). It's been equipped, as you can see, with bright orange stickers.
  • The 640x480 Logitech Webcam provides the next link in the chain. Pointed at the car, it allows the software to see.
  • Yes, closing the loop is the software. The captured image from the webcam is processed, and based on the results of the processing, the software decides how the car will be instructed to move.

That's the 'hardware' loop. The 'software' loop is described below.

  • Capture Image: The software captures the image from the webcam using classes and methods available in the java media framework.
  • Filter Colour: Then, the captured image is put through a simple colour filter - the orange stickers remain orange, everything else is changed to black.
  • Dilate and Shrink: In order to coalesce the edges of the image of each sticker, the orange part of the image is dilated (black pixels next to orange ones are changed to orange), then shrunk (orange pixels near black ones are changed to black). This ensures that a stray pixel near the edge of an image of a sticker is not counted as a very small sticker.
  • Detect Regions: Using a simple region-filling algorithm, the orange pixels are collected into groups, each group (hopefully) representing one sticker.
  • Apply Geometry: The centre and size of each group is measured. The large one is assumed to be the goal, the other five to be the four corners and the front of the car. From this, the software finds the distance and angle from car to goal, and the orientation of the car. The software doesn't bother with real-world coordinates, it measures everything in pixels.
  • Decide Strategy: Knowing the relative positions of the car and goal, the software applies a set of rules to decide what to do. An example might be if distance is LARGE and angle is between -5 to 5, then move straight forward.
  • Drive! Once the decision is made, the appropriate commands are sent to the interface board. The software waits for a reasonable amount of time, captures another webcam image, and the software loop begins again.

Strengths and Weaknesses

The system works.

  • It doesn't depend on carefully calibrating the camera. In fact, in the video clip above, the webcam was hand-held, about a metre above the car.
  • It doesn't depend on careful mechanical engineering. We used a cheap toy. Even on a slippery floor, where the effect of each steering command is unpredictable, the car is able to find the goal.
  • It doesn't depend on careful path planning. No long-term plan to reach the goal is formed. Instead, the car formulates only the next step in the plan in each cycle. This allows it to adapt well to slippery tyres or researchers who keep moving the goal.

There are limitations however.

  • If the system detects too many regions of orange colour, the loop halts. For example, if there are 6 or 7 'car stickers' detected, the software doesn't know what to do. This could be overcome via a 'best fit' approach that tried each set of 5 regions out of the 6 or 7 in turn, and seeing which set was closest to the configuration of stickers on the car.
  • Partly because of this, the system is quite sensitive to lighting conditions. Because the thresholds used in the colour filtering step were decided under bright overhead fluorescent light, the system performs poorly under other lighting conditions. This could likely be overcome using histogram equalisation, more thorough experimentation with lighting, or more forgiving thresholds.
  • Also partly because of the first issue, the system is somewhat sensitive to the height at which the webcam is held. Too high, and the stickers are not properly resolved. (Too low, and the car drives out of view!)

Personally, I feel these limitations could be overcome with a one or two more months of work. However, like any undergraduate project student, Nasa had to meet his deadline...


You may leave comments at the blog entry for this project.