Image Processing and Computer Vision

A strong capability in various fields of image processing allows the CSIR to support several applications in surveillance. Some of these include:

• Maritime object detection and tracking
• 360 degree image stitching
• Atmospheric turbulence mitigation in images
• Detection of moving targets from aerial platforms
• Distributed surveillance

Image processing and computer vision is a broad research area that involves specialists from various fields of expertise. These include:

Photogrammetry

Camera calibration is achieved using several methods in numerical optimisation. The relationship between the image and the world around it, can be determined using photogrammetry. Simply put, photogrammetry is the science behind taking measurements from photographs. Many of the maps we use today, were created with photogrammetry and photographs. Using this capability, a normal optical camera can be turned into a precision aiming device, which can be used to create images for real-time stitching, helmet tracker based navigation and targeting, see-through armour and image fusion.

Image enhancement

Video frames can be distorted by several factors, such as atmospheric turbulence, for example mist or rain, and platform motion, for example when an optronic system is mounted on a ship in stormy weather. Several algorithms have been developed by the CSIR’s optronic experts for tone mapping, low light image enhancement, image de-warping, image stabilisation, motion de-blur and super resolution. The real-time implementation is achieved using multiple central processing units (CPU) and/or graphics processing units (GPU). This capability is particularly useful when looking at real-time video during surveillance missions.

Pattern classification and machine learning

The CSIR’s OSS group has conducted research into microdot processing, optical character recognition (OCR) and target classification. These problems rely on state of the art methods, such as deep learning and convolutional neural networks (computing systems made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs). These systems can detect the difference between pedestrians and vehicles in cluttered environments.

Object tracking

The object trackers developed by the CSIR, use particle filters to estimate a target’s state. Coarse to fine image matching strategies are also used to align video frames. For particular applications, computer vision models have been developed for robust tracking. The object trackers also consist of target detection and segmentation modules for automatic operation in real-time. Image segmentation refers to the extraction of pixels of objects of interest from video.

Augmented reality

Augmented reality is the ‘real-world’ views of an environment with computer-generated sensory input, such as graphics, combined with see-through or transparent head mounted displays, such as the overlaying tactical information and training simulations.

Computational photography

The CSIR looks at image synthesis by changing the aperture size, aperture location, and the depths of the lens and sensor planes computationally using, for example, coded apertures or plenoptic cameras (light fields). This allows for digital refocus, perspective shifts on images and foliage penetration.

3D vision

3D object reconstruction from a stream of images and 3D object recognition using point clouds captured using the Kinect are some of the capabilities of OSS.

Applications supported by image processing include target detection and tracking the wide area surveillance system, and real-time long-range video enhancement.