Educational offer - University of Padova

Syllabus

Prerequisites:	Basic knowledge of Mathematics, Probability Theory, Statistics, Linear Algebra, Algorithms and basic Programming skills.
Target skills and knowledge:	The aim of this course is to provide the fundamentals and basic principles of the learning problem as well as to introduce the most common algorithms for regression and classification. The course will be complemented by hands-on experience through computer simulations. At the end of the course the student will have the following skills and knowledge: 1. The student will know the basic principles and the main methodologies of machine learning, 2. He will be able to deal with both supervised and unsupervised learning problems. 3. He will be able to apply these methodologies to different scenarios and problems. 4. He will be able to select the best technique for the solution of a specific learning problem on the basis of the characteristics of the problem and of the available data. 5. He will have the skills allowing him to use and to adapt software applications to solve the considered problems. 6. The skills relative to more advanced and modern topics including deep learning will be provided.
Examination methods:	The evaluation of the acquired skills and knowledge will be performed using two contributions: 1. A written exam without the book, where the student must solve few problems, with the aim of verifying the acquisition of the main ingredients of a learning problem and of the main machine learning tools, the analytical ability to use these tools and the ability to interpret the typical results of a practical machine learning problem. 2. Computer simulations (optional) with the aim of acquiring the practical competences for using machine learning tools. These simulations, to be performed at home, allow to verify the ability of practically exploiting the acquired theoretical concepts. The student will have to provide a brief document explaining the employed methodologies used to solve the assigned problem together with the obtained results. The final grade will be based on the written test with a bonus up to 3 point for the students who will hand in also the lab assignments.
Assessment criteria:	The evaluation of the acquired skills and knowledge will consider the following aspects: 1. The completeness of the acquired knowledge for what concerns the basic tools for prediction (regression and classification). 2. The analytical and practical ability in the use of these tools for the solution of basic problems. 3. The capability of using a proper technical terminology, both oral and written 4. The originality and independence in identifying the most suited methodologies for the solution of a specific machine learning problem 5. The ability to interpret the results in a practical machine learning problem. 6. The skills in the use of the machine learning software tools 7. The practical and analytic skills in the use of these tools for the solution of simple problems.
Course unit contents:	Motivation; components of the learning problem and applications of Machine Learning. Supervised and unsupervised learning. PART I: Supervised Learning 1. Introduction: Data, Classes of models, Losses. 2. Probabilistic models and assumptions on the data. The regression function. Regression and Classification. 3. When is a model good? Model complexity, bias variance tradeoff/generalization (VC dimension, generalization error). 4. Models for Regression: Linear Regression (scalar and multivariate), subset selection, linear-in-the-parameters models, regularization. 5. Classes of nonlinear models: Sigmoids, Neural Networks. 6. Kernel Methods: SVM. 7. Models for Classification: Logistic Regression, Neural Networks, Perceptron, Naïve Bayes Classifier, SVM, Deep Learning. 8. Validation and Model Selection: Generalization Error, Bias-Variance Tradeoff, Cross Validation. Model complexity determination. PART II: Unsupervised learning 1. Cluster analysis: K-means Clustering, Mixtures of Gaussians and the EM estimation. 2. Dimensionality reduction: Principal Component Analysis (PCA).
Planned learning activities and teaching methods:	Theoretical classes using both slides and blackboard. Problem solving sessions, involving students in the solution. Computer simulations (in the lab), also employing case studies.
Additional notes about suggested reading:	The course will be based on the four textbooks: “Understanding Machine Learning: from Theory to Algorithms”, "Machine Learning a probabilistic perspective", "Pattern Recognition and Machine Learning", and "The Elements of Statistical Learning" (see Section "Testi di Riferimento"). All the material presented during the lectures, additional material and detailed information regarding the exam will be made available on the course website, accessible from http://elearning.dei.unipd.it )
Textbooks (and optional supplementary readings)	Shalev-Shwartz, Shai; Ben-David, Shai, Understanding machine learning: From theory to algorithms. Cambridge: Cambridge University Press, 2014. C. M. Bishop, Pattern Recognition and Machine Learning. --: Springer, 2006. Murphy, Kevin P., Machine Learninga probabilistic perspective. Cambridge: Mit press, 2012. Hastie, Trevor J.; Tibshirani, Robert, The elements of statistical learning, data mining, inference, and prediction. New York: Springer, 2009.