3D AUGMENTED REALITY

Second cycle degree in ICT FOR INTERNET AND MULTIMEDIA

Campus: PADOVA

Language: English

Teaching period: First Semester

Lecturer: SIMONE MILANI

Number of ECTS credits allocated: 6


Syllabus
Prerequisites: Computer Vision class is recommended although not strictly necessary
Examination methods: Written exam + report
Course unit contents: a) From 3D scene to images via real imaging systems

1) Image formation and camera model

Perspective projection
Pin-hole camera
Thin lenses
Fish-eye lenses
Simplified and general camera model
Digital images

2) Computation of salient points and features

Harris e Stephens method
Scale Invariant Feature Transform (SIFT)
Salient points correspondences

3) Camera calibration

Basic notions of camera calibration

4) Homographies

Computation of the homography (DLT)
Homographics and object recognition
Applications of 2D augmented reality


b) From images to 3D scene model (or 3D reconstruction in Computer Vision)

1) Stereopsys: geometry

3D Triangulation
Epipolar geometry
Epipolar Rectification
Essential matrix and factorizzation
Motion and structure from calibrated homography


2) Stereopsys: Corrispondence

Local correspondence methods
Window correspondence methods
Accuracy-reliability trade-off
Occlusions
Other local methods
Global correspondence methods


3) 3D reconstruction from other sensors

IR Structured-light depth sensors (MS Kinect v.1)
Time-of-Flight depth sensors (MS Kinect v.2)
Active stereo sensors
Laser scanners

4) Non-calibrated reconstruction

Fundamental matrix and its computation
Projective reconstruction from 2 and N views
Method of Mendonca e Cipolla
Tomasi-Kanade factorization
Incremental reconstruction
Structure-from-Motion
Bundle adjustment

5) Optical flow

Motion field: computation of motion and structure
Optical flow: Lucas-Kanade method

6) Orientation methods

Quaternions
Orientation 2D-2D:
Orientatio 3D-3D: DLT and ICP methods
Orientation 3D-2D
Surface integration
Mesh semplification

7) Object detection and scene understanding

Image and 3D models features
Image/object classification strategies
Machine learning algorithms for object classification
Support Vector Machine for image/object classification
Deep Neural Networks for image/object classification
Human-computer interfaces (HCI)

c) From 3D models to images and beyond

1) 3D displays, VR visors, and augmented reality devices

2) Rendering

Rendering: projective geometry and convention
Ray tracing and ray casting
The radiance (or rendering) equation and its solution

3) Illumination

The radiance solution by local methods: Phong and Cook-Torrance models
Light types
The radiance solution by global methods
Ray tracing: Whitted method
Radiosity: radiosity equation in continuous and discrete form
Shading

4) Rasterization

The OpenGL pipeline