Cameras :: QUARC

navigation bar

Table of Contents > QUARC > User's Guide > Using Devices

Cameras

Using cameras and vision systems for the purposes of control typically involves several steps. First, a source image is acquired from a camera, vision system, or possibly a simulation. Next, the image data is processed using image processing algorithms designed for various purposes, eg., detecting a specific image or shape within the source image. The image may be modified or altered, and then displayed to the user. QUARC provides a set of camera and image processing blocks that can be used to accomplish these tasks.

These blocks may be divided into two categories: generic image processing blocks and Point Grey Research blocks.

Generic Blocks

The generic image processing blocks use one of two image formats: MATLAB RGB format for color images or MATLAB Greyscale format for grayscale images. Both formats are compatible with The Mathworks' Computer Vision Toolbox.

MATLAB RGB: The MATLAB RGB format organizes color image data as a 3-dimensional matrix, which each pixel represented by red, green and blue components. The pixels are organized in a planar, column-major arrangement of dimension [H x W x 3], where H is the height of the image and W is the width. In the last dimension, red is the first element, green the second and blue the third. Hence, in memory, a plane of red values in column-major order appears first, followed by a green plane and finally a blue plane. For example, for a 2x2 MATLAB RGB image, the pixels will appear in memory in the order [R11 R21 R12 R22 G11 G21 G12 G22 B11 B21 B12 B22]. An image 640 pixels wide and 480 pixels high would be stored in MATLAB RGB format as a [480x640x3] matrix. Notice that the height is the first component. In MATLAB, given a variable im containing the image, the expression im(:,:,1) would extract the red plane, im(:,:,2) would extract the green plane and im(:,:,3) would extract the blue plane.
MATLAB Greyscale: The MATLAB Greyscale format organizes grayscale image data as a 2-dimensional matrix, which each pixel represented by a single luminance component. The pixels are organized in column-major order with dimension [H x W ], where H is the height of the image and W is the width. For example, for a 2x2 MATLAB Greyscale image, the pixels will appear in memory in the order [Y11 Y21 Y12 Y22]. An image 640 pixels wide and 480 pixels high would be stored in MATLAB Greyscale format as a [480x640] matrix. Notice that the height is the first component.

Both image formats support the following data types: uint8, uint16, uint32, single and double. The uint8 format is generally the most efficient.

To read from a camera, the Video Capture block from the QUARC Multimedia library may be used. Any webcam, for instance, supported by Windows should be supported on the Windows target. Support is more limited on embedded systems. Refer to the Video Capture block help page for more details.

The image formats output by the Video Capture block are compatible with the Computer Vision Toolbox and many of the standard Simulink blocks. QUARC also supplies some of its own image processing blocks that are optimized for the QUARC targets. See the QUARC Image Processing Blocks for details.

Video output may be displayed using the Video Display block or Display Image block. The Video Display block is recommended as it has much higher performance. The Display Image block uses a MATLAB figure window for the image display and may be used to drive the axes in a MATLAB GUI. An example of using the Video Capture and Video Display blocks is illustrated below.

Video Capture

Point Grey Research Blocks

The QUARC Point Grey Research (PGR) blocks are used with Point Grey Research cameras. Not all cameras offered by Point Grey Research are supported.

There are a number of image formats that can be used with the QUARC PGR blocks, and the following list describes these formats.

RGB8: This format describes colour image data organized as a 3-dimensional matrix, with each pixel represented as a 3-element set of [red, green, blue]. The pixels are organized in row major order so that an image of width 640 and height 480 is stored in a 3-dimensional matrix with dimensions 3x640x480.
BGR8: This format describes colour image data organized as a 3-dimensional matrix, with each pixel represented as a 3-element set of [blue, green, red]. The pixels are organized in row major order so that an image of width 640 and height 480 is stored in a 3-dimensional matrix with dimensions 3x640x480.
Greyscale (Y8): This format describes grayscale image data organized as a 2-dimensional matrix, with each pixel represented as a single luminosity value ranging from 0 to 255, where 0 represents black and 255 represents white. The pixels are organized in row-major order so that an image of width 640 and height 480 is stored in a 2-dimensional matrix with dimensions 640x480.

All three image formats only support the uint8 data type. To convert from these formats to image formats compatible with the Computer Vision Toolbox and the the QUARC Image Processing blocks, use the PGR Image Convert block.

See the Generic section above and the QUARC Image Processing Blocks help pages for details on the generic image processing functions.

To accomplish image acquisition, processing, and display with PGR cameras, QUARC provides several blocks. These blocks, along with an example of their usage, is given below. Note that since image processing is typically a computationally-intensive process, some blocks impose sample time constraints. Refer to the documentation for each block for details on sample time constraints.

The PGR Grab Image block is used to acquire images from Point Grey Research cameras. Currently, the supported cameras that have been tested are the Dragonfly2 HI-COL, FireflyMV and FireflyMV Mono. Other PGR cameras may also work, but have not been tested and are not officially supported. The PGR Dragonfly2 HI-COL is depicted below.

QUARC also provides image processing blocks that can be used to find objects of a given colour within a source image and convert images from one format to another. The PGR Find Object block can be used to search through a source image (supplied from a PGR camera or some other source image that fits the appropriate image format) and find connected points of a specified colour. The block then outputs the (x,y) coordinates (in pixels) of up to five objects. See the help page for the PGR Find Object block for specific usage details. The PGR Image Convert block is used to convert images from one format (RGB8, BGR8, or Y8) to a format that can be displayed in MATLAB using the Video Display block or Display Image block.

QUARC's Video Display block is a high performance video display block that shows the images in a window on the host. It is the recommended block for displaying video.

The Display Image block provided with QUARC is used to display colour or grayscale images in a figure window or on axes within a Matlab GUI. As with the PGR Find Object block and the PGR Image Convert block, the Display Image block can be given input image data from any source as long as the data is formatted as expected.

Below is an example diagram illustrating how the aforementioned blocks can be put together to acquire, process, and display images. Note that this is just an example diagram and that certain blocks may need to run at different sample times depending on their usage. Refer to each block's documentation for details on constraints. Make sure the blocks are all configured with the same Sample Time in this example, such as 0.1 seconds.

navigation bar