The objective of the course is to introduce the students to the use of the numerical optimization for machine learning. The main subjects are the following: elements of nonlinear programming, supervised classification, unsupervised classification elements, semisupervised classification, multiple instance learning elements, optimization software for machine learning.
1. Main concepts at the basis of the Machine Learning.
Introduction to Machine Learning and classification problems. Some Machine Learning examples.
Evaluation of a classifier.
2. Elements of nonlinear optimization.
Convex functions, local and global minima. Optimality conditions for unconstrained optimization problems. Some solution techniques for unconstrained optimization:
the gradient method and the Newton method. The line search. Optimality conditions for constrained optimization problems.
3. Supervised classification.
Linear separability. Support Vector Machine (SVM) and kernel. Model selection and cross validation. Polyhedral separability. Spherical separability.
4. Elements of unsupervised classification.
Clustering problems and their formulation.
5. Semisupervised classification.
Tranductive Support Vector Machine. Semisupervised polyhedral separability. Semisupervised spherical separability.
6. Multiple Instance Learning.
Introduction to Multiple Instance Learning. Instance-space, bag-space and embedding-space methods.
SVM and PSVM (Proximal Support Vector Machine) type models for Multiple Instance Learning.
Some solution algorithms.
7. Laboratory.
Octave for Machine Learning.
J. Nocedal, S.J. Wright – Numerical Optimization – Springer, 2006.
M. Swamynathan - Mastering Machine Learning with Python in Six Steps - Apress, 2017.
I.H. Witten, E. Frank - Data Mining: Practical Machine Learning Tools and Techniques - Elsevier, 2005.
T.M. Mitchell - Machine Learning - McGraw-Hill, 1997.
F. Herrera, S. Ventura et al. - Multiple Instance Learning: Foundations and Algorithms - Springer, 2016.
Lecture notes and scientific articles.
The exam consists of two parts: an oral exam and a discussion on a specific project assigned by the teacher. The University (or identity) card is requested.