Teaching the PixyCAM Camera an Object

PDF24    Send article as PDF   
Teaching PixyCAM an object is super easy. Before we start, which objects will work well with PPixyCAM, and which wont? PixyCAM uses a hue-based colour filtering algorithm to detect objects. Since PixyCAM uses hue (colour), the object needs to have a distinct hue. Here are some objects that are good because they have good, distinct hues.

Fluroescent Pigs

Below is are some bad objects because either there is no hue (black, white, brown or gray) or the hue is not distinct.

Brown Pig
Black Pig

Keeping these guidelines in mind, choose an object to teach PixyCAM. Begin by applying power to PixyCAM via battery or USB cable if you haven’t already. When you power up PixyCAM, it will go through a series of LED flashes. Wait for the LED to turn off before teaching PixyCAM an object.

Now begin by holding down the “white” button on top of PixyCAM, and after about 1 second, the LED will turn on – first white, then red, then other colours – but when it turns red, release the button.

pixycamupdatefirmware-00

When you release the button, PixyCAM will enter what’s called “Light Pipe” mode, where the LED colour is the colour of the object that PixyCAM has “locked” onto. PixyCAM will lock onto objects in the center of its video frame, so hold the object directly in front of PixyCAM, between 150mm and 450mm from the lens.

Teach PixyCAM Object - 01

The PixyCAM uses a region growing algorithm to try to determine which pixels are part of your object, and which pixels are part of the background. Using these pixels, PixyCAM will try to create a statistical model of the object, so it can detect it reliably under different lighting conditions. Use the LED colour as feedback to determine if PixyCAM has a good lock on the object, and use the following guidelines to judge:

  1. When PixyCAM has locked onto your object, the LED colour should match the colour of your object.
  2. The brighter the LED, the better the lock. Move the object a little and see if you can maximize the LED brightness.
  3. Look at the region-growing grid in the PixyMon window and see how the size of the grid corresponds to the LED colour and brightness. A grid that covers more of your object is better than a grid that covers only some of your object and results in a brighter LED. The picture below shows a grid that covers most of the object (good). 

Teach PixyCAM Object - 02

Getting to grips to when the PixyCAM has a decent colour lock by looking at the LED might take some practice, but once you get the feel of it, you can teach PixyCAM new objects reliably without needing to use PixyMon!

When you are satisfied, press and release the button on PixyCAM, like you’d click your mouse. The LED will flash a couple times indicating that PixyCAM has now “learned” your object. It will now start tracking your object.

The video below is a good short “how-to” regarding teaching PixyCAM objects.

Multiple Signatures

PixyCAM can learn up to seven colour signatures. We can teach PixyCAM the 1st colour signature by releasing the button when the LED turns red. If we continue to hold down the button, the LED will turn orange, yellow, etc., indicating the remaining colour signatures. Here are the signatures in order:

  1. Red
  2. Orange
  3. Yellow
  4. Green
  5. Cyan (light blue)
  6. Blue
  7. Violet

The colour signature number is determined by when you release the button. Release the button when the LED is Yellow and you’re teaching signature 3. Release the button when the LED is Blue and you’re teaching signature 6. These colours are not related to the hue of the object. The colours are used only to indicate the signature number. So, for example, signature 1 can be a yellow object, even though signature 1 is indicated by a Red LED, and signature 2 could be a pink object even though signature 2 is indicated by an Orange LED.

After you teach PixyCAM a signature, it saves the signature in flash, so when you power the PixyCAM off, it will remember the signatures you taught it and continue to track objects that match these signatures.

If you accidentally find yourself teaching signature 2 when you meant to teach signature 1 (i.e. you released the button when it was orange instead of red), for example, just hold down the button until the LED turns off. This is how you tell PixyCAM to cancel teach mode. You can then start over by holding the button down again.

White Balance

Some types of lighting (such as incandescent) have a reddish hue and others (such as fluorescent) have a bluish hue. The lighting can affect your colour signatures. For example, if you teach an object under incandescent lighting and move into a room with fluorescent lighting, the colour signatures will likely no longer work as well. You can either re-teach all signatures or you can adjust the white balance.

When you first apply power to PixyCAM, it will spend the first 5 or so seconds determining the correct white balance to use. It will then disable automatic white balance. If you wish to readjust the white balance, hold down the button until the LED turns white and release. It happens quickly, so be prepared! PixyCAM is now in automatic white balance mode. You can hold a white sheet of paper in front of PixyCAM so PixyCAM can adjust the white balance. It only takes 2 or 3 seconds to adjust the white balance, after which you can press and release the button (like a mouse click). The LED will flash, indicating success, and now PixyCAM is “rebalanced“.

Teaching through PixyMon

You can also teach PixyCAM an object through PixyMon. This may be useful if the object you want to teach is small, or if you want more control over which pixels are used for teaching. Begin by plugging in the USB cable between PixyCAM and your computer and running PixyMon.

Now hold the object you want to teach in front of PixyCAM and select Action➜Set signature 1 from the pull-down menu.

Teach PixyCAM Object - 04

Using the mouse, click and drag to select the region you want PixyCAM to use to learn the object.

Teach PixyCAM Object - 05

That’s it! After you select the region, PixyCAM will “learn” the object and automatically go into “Cooked” video mode so you can verify how well your colour signature is working.

Teach PixyCAM Object - 06

Signature tuning

Signature range tuning is probably the most effective method to improve detection accuracy. Bring up the Configure dialog by clicking on the gear icon or select File➜Configure. Now select the Signature Tuning tab under PixyCAM Parameters (it should be the first pan you see when you bring up the Configuration Dialog).

Teach PixyCAM Object - 07

Use the slider for Signature 1 range to adjust the inclusiveness of signature 1 (assuming it’s signature 1 you’re wanting to adjust). Slide it to the left if you want to be less inclusive (i.e. you’re seeing false positives, like the picture below):

Teach PixyCAM Object - 08

or slide it to the right if you want to be more inclusive (i.e. you’re seeing false negatives), or the detection is intermittent or sparse (like in the picture below):

Teach PixyCAM Object - 09

Choose a slider value that provides good strong detection like the picture below:

Teach PixyCAM Object - 10

You can adjust all seven colour signatures this way to maximize detection accuracy. Be sure to press Apply or OK to save the slider ranges! The adjusted values won’t be saved if you press Cancel or dismiss the dialog.

Enabling overexposure highlighting and adjusting camera brightness/exposure

Like a camera, PixyCAM needs to get the correct exposure setting or its images won’t contain enough dynamic range, and detection accuracy will suffer. PixyMon has a feature that highlights regions of the image that are overexposed. You can enable it by clicking on the box next to Highlight overexposure in the PixyMon Parameters tab (see below).

Teach PixyCAM Object - 11

When this is enabled, overexposed parts of the image will be highlighted as black, as shown below:

Teach PixyCAM Object - 12

We highly recommend enabling overexposure highlighting, because if the object you want to detect is overexposed, detection accuracy will suffer. So, after enabling overexposure highlighting, go to the Signature Tuning tab in PixyCAM Parameters and choose a good setting for Camera Brightness by adjusting the slider (see below).

Teach PixyCAM Object - 13

A good exposure setting leaves almost all of the pixels in your object correctly exposed, but not too dark (underexposed). It’s OK for small parts of your object to be overexposed, especially if your object is shiny. See the picture below — note that a small part of the object (yellow ball) has a black region indicating overexposure. This is fine.

Teach PixyCAM Object - 14

Adjust minimum brightness

The Min brightness slider in the Signature Tuning tab sets a minimum brightness setting for all signatures. That is, if a pixel is below the minimum brightness setting it won’t be considered as part of any colour signature. So if you are seeing false positives that are sufficiently dark, you can adjust the Min brightness setting higher to reduce the false positives. If you are not able to detect objects that are sufficiently dark, you can adjust the Min brightness setting lower to reduce the false negatives.

Use the mouse to teach PixyCAM your object

There are two ways to teach PixyCAM an object: the button-press method and the mouse-select method. They are both described here. The mouse-select method makes sure that no unintended pixels make it into the teaching set when making a colour signature.

Adjust the signature teach threshold

One of PixyCAM’s goals is to be able to easily learn different objects without needing to hook it up to a computer. The button-press teaching method allows you to do this. But for certain objects and/or lighting conditions, the button-press method might need some adjusting. The Signature teach threshold setting in the Expert tab allows you to adjust how inclusive PixyCAM is when determining which pixels are part of the object during teaching. You can adjust the Signature teach threshold while PixyCAM is in teach mode, so you can get “live” feedback while choosing a good threshold. The video below shows the Signature teach threshold in action.

Teach PixyCAM Object - 15

The picture below is an example of a threshold that is too inclusive. Note the regions that are not part of the purple dinosaur (the intended object). In this case, you should reduce the threshold.

Teach PixyCAM Object - 16

The picture below is an example of a threshold that is not inclusive enough. Note that the regions do not include much of the purple dinosuar. In this case, you should increase the threshold.

Teach PixyCAM Object - 17

I have found the PixyCAM to be far more intuitive and easy to use when compared with other LEGO Mindstorms Vision Sensors to use, and to teach to recognise objects. I highly recommend the PixyCAM for those reason, but that said, it isn’t an easy sensor to program for when compared with the standard array of LEGO Mindstorms Sensors that are available. 

 

Previous Post

Installing PixyMon for the CMUcam5 Pixy Camera on a Windows PC

I have finally gotten around to purchasing a CMUcam5 PixyCam for myself. The (CMUcam5) PixyCam is a Smart Vision Sensor, with advanced Object Tracking. One advantage of this Camera is ... Read more

Next Post

'Metal Minds Robotics' FLL Team are Winners & Grinners !

Thank you to the 'Metal Minds Robotics' team for a great FIRST LEGO League (FLL) Competition at the Tasmanian State Championships on Saturday 19 October 2016, held in Hobart. I ... Read more

Short URL: http://tinyurl.com/huseds5

Leave a Reply

Your email address will not be published. Required fields are marked *



Do NOT follow this link or you will be banned from the site!
error: Content is protected !!