Obstacle detection is one of the most important tasks for a visual navigation system. Aiming to identify the depth of objects in the scene, Stereo Vision techniques (based on the Disparity Map information) and Optical Flow analysis are used to identify movement of objects in consecutive frames.
We are working on a new method combining Stereo Vision and Object Recognition with Convolutional Neural Networks to Identify Threats/Points of Interest in autonomous vehicular navigation.
More details and information in: Original Paper
Source code used: GitHub
Identification&Object Segmentation with Neural Network + Stereo Disparity Maps + Optical Flow showing the set of displacement vectors in objects:
Identification&Object Segmentation with Neural Network + Stereo Disparity Maps + Optical Flow showing the resulting displacement vectors in objects:
Identification&Object Segmentation with Neural Network + Stereo Disparity Maps + Optical Flow showing values analysis results as textual information in objects:
Our Approach
The figure below shows the successive results, the steps in the videos above: from the stereoscopic video are generated both the identification and segmentation of the objects of interest as well as the disparity/depth map of the scene. From there, both the optical flow and the data combinations that appear in the video are calculated.
@misc{rateke:2020.2,
author = {Rateke, Thiago and Wangenheim, Aldo von},
title = {Road obstacles positional and dynamic features extraction combining object detection, stereo disparity maps and optical flow data},
journal={Machine Vision and Applications},
year = {2020},
month={Sep},
day={25},
volume={31},
number={7},
pages={73},
issn={1432-1769},
doi={10.1007/s00138-020-01126-w},
url={https://doi.org/10.1007/s00138-020-01126-w}
}
For obstacle detection it is important to know the distance from the obstacle to the vehicle. In this way, the Disparity Map (MD) calculation is applied, which is the depth representation of the scene in intensity tones, where the lighter means closer and the darker means it is more distant.
Besides the distance, another important feature to be analyzed in obstacles is its movement. In order to obtain this movement information we apply Optical Flow analysis.
* We used, in the experiments, images from the case base provided by KITTI.
Thiago Rateke is a Computer Vision Researcher with experience mainly focusing on visual perception for autonomous navigation. Finished his PhD degree at Federal University of Santa Catarina (UFSC) in 2020 with focuses on visual perception for Autonomous Navigation. Using approaches like: Stereo Vision, Optical Flow and Convolutional Neural Networks.