Design and Synthesis of Multiple View Volumetric 2D To 3D Reconstruction Technique and Its Application

Abstract

Today 3D is a buzz word in the entire multimedia and television industry. 3D reconstruction is an integral part of various applications in almost all disciplines. Few important applications of this field are 3D terrain rendering, virtual reality, robot navigation, augmented reality tasks, games, animations and motion pictures, industrial measurements, surface analysis, volumetric analysis, archeological and forensics applications. In medical field it is used in minimal invasive surgical techniques, 3D rendering of patients anatomy, computer guided surgeries, preoperative planning etc. The large variety of these interdisciplinary applications emphasizes the importance of 2D to 3D reconstruction. There are several attempts made by the researchers worldwide to reconstruct 3D shape of the object from multiple images. There is always a trade-off between computation speed, computation complexity, accuracy and feasibility in implementation of these methods/techniques. Consequently to bring more accuracy and quality in 3D reconstruction is still a challenging and open research problem. Stereo based reconstruction methods face challenges in reconstruction of the objects with very few features. Homography based approaches give accurate estimates yet do not provide system dynamics. Hence, these approaches are used in applications such as tennis referral systems where system dynamics is not important. On the other hand, voxel based volumetric reconstruction methods provide system dynamics which is essential for system modeling. These methods provide true essence of the object’s volume. However, these approaches face challenges while modeling concavities. The dissertation presents a novel vision based mathematical model for combining homography estimation and voxel mapping approach for improving the accuracy of 3D reconstruction. The complete reconstruction pipeline of the proposed volumetric 2D to 3D reconstruction method/technique is presented with the experimental results. Experimental results show that the method efficiently reconstructs objects of known and unknown geometry. Besides, being simple to implement, the multiple cameras placed at different elevations provide more field of view than single camera for each view position and help to bring out more details of the objects. Single camera can be placed at different elevations too. However, for each new object, to acquire different views at different elevations, a camera needs to be recalibrated for every new elevation. The use of multiple cameras overcomes this problem and makes image acquisition fast and accurate. As compared to Structure from Motion (SfM) approach, the proposed method efficiently reconstructs objects with fewer features too. In addition to that the proposed method successfully reconstructs fragile objects and complex scene with more number of objects. Performance of the proposed 3D reconstruction method is compared with the performance of existing approaches. It has been observed that performance of the proposed method is at par with the other approaches. Further, the performance of the proposed method is evaluated on the basis of reconstruction completeness and computation time for variations in number of views and voxel resolution. Increase in number of views and voxel resolution improves the quality of reconstruction. However it diminishes the computation performance. The volumetric 3D reconstruction method based on multiple images using probabilistic approach is developed for arbitrarily placed cameras. Probabilistic approach for computation of voxel occupancies during geometric intersection stage makes the method suitable for reconstruction of the objects whose images are acquired using arbitrary placed cameras. Performance of the method is tested on dataset available in the literature and own dataset created using image acquisition set up. Results show that the quality of reconstruction for own dataset objects is better than dataset objects imaged with arbitrary cameras. The reason behind this is a lesser amount of variation in consecutive images of the object in own dataset. In order to prove the usefulness of the proposed volumetric 2D to 3D reconstruction method, a nondestructive and accurate fruit grading system using volume and color based maturity feature is designed and implemented using Fuzzy Rule Based Classification (FRBC) technique. The system estimates volume of the fruit using volumetric 3D reconstruction method employing multiple cameras with high accuracy. The system computes the percentage of matured region from the entire surface of the fruit using multiple images as against the single image used in most of the 2D techniques used for grading the fruits. This overcomes the problem of determining percentage of matured region of self occluding surfaces to improve accuracy in maturity estimation. The experimental results and comparison of performance with the existing approaches show that the performance of the method is at par with the other approaches based on 2D techniques as well as 3D techniques. The proposed reconstruction method may face challenges if the object has deep concavities in the surface. However, this may not cause any limitation in the application of fruit grading since most of the fruits in the nature do not have deep concavities in the surface. Additional depth cues with volumetric approach can improve the accuracy of reconstruction further. However, this will increase the cost of the system along with the complexity in implementation and computation. For large scale scenes voxel representation of the objects and their processing is a challenge. Next generation high end graphics processors may solve this issue in future.