MACHINE LEARNING (Numerosita' canale 1)

Second cycle degree in AUTOMATION ENGINEERING

Campus: PADOVA

Language: English

Teaching period: First Semester


Number of ECTS credits allocated: 6

Prerequisites: Basic Knowledge of Probability Theory, Statistics and Linear Algebra
Examination methods: Written test and take-home computer simulations
Course unit contents: Motivation; components of the learning problem and applications of Machine Learning.
Supervised and unsupervised learning.

PART I: Supervised Learning

Introduction: Data – Classes of models - Losses
Probabilistic models and assumptions on the data
Models, Losses and the regression function. Regression and Classification
When is a model good? Model complexity, bias variance tradeoff/generalization (VC dimension – generalization error)

Least Squares, Maximum Likelihood and Posteriors.

Models for Regression
Linear Regression (scalar and multivariate) – Stein’s paradox and Regularization – Subset Selection
Linear-in-the-parameters models, Regularization.
Local and global models (Smoothing Kernels and NNR)
Dimensionality Reduction: Principal Component Regression, Partial Least Squares.
Classes of non linear models: Sigmoids, Neural Networks
Kernel Methods: SVM

Models for Classification

Linear Discriminant Analysis, Logistic Regression, NN, Perceptron, Naïve Bayes Classifier, SVM

Validation and Model Selection
Generalization Error, Bias-Variance Tradeoff, Cross Validation, SURE.
Model complexity determination

PART II: Unsupervised learning
Cluster analysis: K-means Clustering, Mixtures of Gaussians and the EM estimation
Dimensionality reduction: Factor analysis, Principal Component Analysis (PCA)