The new concept of Advanced Driver Assistance System (ADAS) has rapidly emerged in the market to ensure the safety of people driving their vehicles. The system not only tries to ensure the safety of people behind the wheels but also prevents accidents and provides assistance in better driving. One of the main components of this system is “Top View” or “Surround View”. Automotive surround view, also called “around view” or “surround vision monitoring system”, provides the driver a 360-degree view of the area surrounding the vehicle.
In order to achieve the goals of ADAS, the image sensors should continuously input images that will be processed and monitored for precise and perfect maneuvering of vehicles. To start from the basic, first, a setup needs to be installed on the vehicles that would provide images to the image-processing unit that would later be displayed on the LCD or Central Information Display.
- Four cameras are installed on the vehicle: one on the front bumper, second on the rear bumper and the other two on each of the side view mirror.
- The cameras have fish-eye lenses because the images need to cover a large area of the surrounding without loss of much data. The fish eye lenses have as large a field of view as around 180⁰.
- These cameras are also tilted at some angle from the horizontal so that the road areas next to the vehicle are also covered which increases accuracy of the system.
The cameras have to work on real time and output a video at 30 fps. Similarly, the surround view system should provide an output at the same pace such that as and when the vehicle moves, the system is updated. The cameras input the images and the image-processing unit (surround view system) takes these four images one at a time and stitches them together to provide a top-down view of the vehicle and the area around the vehicle.
SURROUND VIEW SOLUTION
The surround view system has three main components that are as follows:
Geometric Alignment itself has two parts: one, that includes the correction of distortion produced in each image due to the fish-eye lenses and second, the transformation of image from their individual perspective to top-down view perspective. The photometric alignment removes the varying intensity between the images captured by the cameras and lastly, composite view synthesis stitches the four images together to provide the driver a bird’s eye view as if the driver is present at a certain vertical height from the car’s roof.
Due to the advantages of the fish-eye lenses over the other lenses, they are considered for this system, but these lenses achieve extremely wide angles of view by causing the straight lines to get distorted. The objects that are near to the lenses appear bigger and the ones that are away appear too small. The tangential distortion is neglected, as it is quite negligible as compared to the radial distortion. The radial distortion cause by the fish-eye lenses is the barrel distortion, wherein, image magnification decreases with distance from the optical axis. The apparent effect is that of an image which has been mapped around a barrel. The straight lines appear as curves and the points are moved in radial direction from their original position. Due to this distortion, the fish-eye lenses are able to map an infinitely wide object plane into a finite image area. The distortion is removed by lens-distortion correction algorithm.
- Lens Distortion Correction: In this, inverse mapping can be done, where, for every output pixel, an input pixel is identified and accordingly the output pixels will pick up their intensity values. The mapping from the input pixel to the output pixel is one-to many. The output image obtained is then vertically and horizontally corrected and then restored to its initial input size.
Final Corrected Image after Lens Distortion Correction
The image obtained as output, is then taken for the next level of processing where the images are transformed to top-down views.
- Perspective Mapping: Perspective transformation is to project the image onto a new viewing plane, it is also called projective mapping. To get the bird’s eye view from distortion corrected images, four points are selected on the distortion corrected images and corresponding to them, four points are taken up on their top- down view. Then, homography matrix is calculated for those two planes. Using this matrix, an inverse mapping is done and again for every output pixel, corresponding input pixels are picked up from where they take up their intensity values.
After the above two steps, the objects present in the overlapping field of views of every camera, are checked for their positions so that stitching can take place without any misalignment in the overlapping regions.
Due to illumination differences, the brightness or intensity of the same objects captured by different cameras can be quite different. For getting a seamless stitched top-down view, it is necessary that the photo metric difference between two adjacent views be removed. Using this, the composite view will appear as if it were taken by a single camera placed above the vehicle. This further minimizes any discrepancies in the overlapping regions of adjacent views.
COMPOSITE VIEW SYNTHESIS
Synthesis function receives input video streams from four fish-eye cameras and creates a composite surround view. Synthesis creates the stitched output image using the mapping that is pre-decided. There are overlapping regions in the output frame, where image data from two adjacent input frames are required. In these regions, each output pixel maps to pixel locations in two input images, which is managed accordingly.
BENEFITS OF SURROUND VIEW
The benefits of the surround view can be summed up in the following points:
- Eliminates blind spots during critical maneuvers in crowded and narrow spaces
- Provides help during parking of vehicles
- Prevents any kind of accidents that include collision into another vehicle, which is turning into or crossing a road, collision between vehicle and pedestrian, collision with any obstacle such as any animal crossing the road etc
- Assistance in changing or leaving a lane
- Increases the efficiency in traffic and transport