3D-DNN

3D-DNN

1. 3D Deep Learning Tasks

   • 3D Geometry Analysis
     • 3D Image/Dataset is Fed as Input to the Model
     • Classification
     • Parsing(Object/Scene)
     • Correspondance
   • 3D Assisted Image Analysis
     • 3D Image/Dataset is found in the Intermediate Layer
     • Cross View Image Retrival
     • Intrinsic Decomposition
     • Topic branch
   • 3D Synthasis
     • 3D Image/Dataset is at the end of the Model
     • Monocular 3D Reconstruction
     • Shape Completion
     • Shape Modelling

2. 3D Model Datasets to be Downloaded

   • Yobi3D
   • Clara.io
   • Google SketchUP
   • SketchFAB
   • TinkerCAD

3. Applications of 3D Data

   • * Most of the Historical Models only focus on  one Model  * Lacking 3D Datasets is a major bottleneck 
   • Robotics
   • Augmented Reality
   • Autonomous Driving
   • Medical Image Processing

4. 3D DNN-Training Algorithms

   • 3D Representation issue
     • Images: Unique representation with regular data structure for a 2D image
     • 3D Representation Types
       • Multi-view RGB(D) images
       • Volumetric
       • Polygonal Mesh
       • Point Cloud
       • Primitive-Based CAD Models
       • Rasterized form (regular grids)
       • Geometric form (irregular)
         • VoxelNET
           • Inputs-PointCloud
           • Output-Recognized objects in point cloud
           • Step 1. Calculate the features (4D sparse tensor) by feature learning network with raw point cloud data
             • Step 2. Use 3D convolution to calculate high-level features.
               • Step 3. Use Region Proposal Network to do classification and bounding box regression.
           • 3-Parts of a VoxelNET
             • Feature Learning Network
             • Convolutional Middle Layers
             • Region Proposal Network
           • Background of VoxelNET
         • Comparing Point Cloud Data
           • Octree Methodology
             • Quadtree Methodology
               • Quadtree methodoloy for a circle ( Hard to approximate )
               • Advantages and Disadvantages of Quadtree Methodology
               • Quadtree on a real 2D image
             • The Whole Volume space is divided into 8 parts and on and every node will have 8 children
               • Each node have either '1' or '0' or have 8 children nodes
           • Using the Binary obtained by appllying the octree method on original voxel space and the the deviated voxel space and them comparing the two binary codes will yeild the difference between two binary codes
   • Deep learning on different 3D representations
     • Deep learning on regular structures
       • Multi-view representation
         • 1. Convert irregular (3D) to regular (images)
         • 2. Circumvent any geometric representation artifacts (non-manifold geometry, polygon soups, no interior)
         • 3. Leverage pre-trained image-based CNNs
         • 4. Similarly to humans, analyze what can be seen: combine surface information from multiple views
         • Segmentation
         • Reconstruction
         • Classification
           • Traditional approach: Multiview feature+linear classifier-1.Get Many Different Views after rendering a "Geoemtric formed Image 2.Feed the multiview representations to obtain different feature map sets 3.Send through a view Pooling Layer (Element Wise Max Pooling Across All views) 3.Feed them into a second ConvNet to produce Shape descriptors by going through a softmax layer
             • Softmax Layer
           • Transfer Learning
             • Pre-training
               • Find a source of massive data D' with similar statistics
               • Learn Network Parameters from D'
             • Fine-tuning
               • Starting from the learned parameters on D' , minimize the network loss on D
           • Types of Neural Network Models used for Classification
       • Volumetric Representation
     • Deep learning on meshes
     • Deep learning on point cloud and parametric models-Geometric Form
       • Voxelization
         • Data Pre-Processing Stage
           • Removing Outliers
         • Voxel Fitting Accuracies as per example
           • 1 cm voxel fitting
           • 3 cm voxel fitting
           • 5 cm voxel fitting
       • Problems of Point Clouds
   • Fundamental challenges of 3D deep learning
     • Can we directly apply CNN on 3D data? Applying 3D Data into a model depends on the kind of dataset for example if the dataset is a Rasterised Dataset then it can be directly fed into an CNN and if not the Dataset should be converted into Rasteriesed Data from the Geometric form

Imagens e arquivos do mapa