Kernel methods and semi-supervised learning
(short course)
Slides
[pdf]
Contents:
- Machine learning. Supervised learning
- Kernel methods
- Hilbert spaces
- Kernels
- Centroid method
- kNN
- k-Means
- Semi-supevised learning
- Assumptions in SSL
- Simple semi-supervised methods
- Self-training
- Committee-based learning
- Graph-based distances
- Manifold learning
- Cluster kernels
Bibliography:
- B. Schölkopf, A. J. Smola. Learning with Kernels. MIT Press, Cambridge, Mass., 2002. [link]
- O. Chapelle, B. Schölkopf, A. Zien. Semi-Supervised Learning. MIT Press, 2006. [link]
- F. Jäkel , B. Schölkopf , F. A. Wichmann. A Tutorial on Kernel Methods for Categorization.
Journal of Mathematical Psychology 51(6), pp. 343-358, 2007. [pdf]
- X. Zhu. Semi-Supervised Literature Survey. TR 1530, University of Wisconsin - Madison, 2008. [pdf]
Datasets
Programming exercises
- Implement the (kernel) kNN algorithm (for binary classification).
- Implement the (kernel) centroid algorithm (for binary classification).
- Create two functions: one for transforming a kernel (Gram) matrix to a (squared) distance matrix and another one
for the inverse process (distance -> kernel).
- Using the linear, polynomial and Gaussian kernels try your classification algorithms
on the above datasets.
- Implement the Floyd-Warshall algorithm.
- Construct kNN and eps-NN graphs using the Floyd-Warshall algorithm and then build
kernels from these graphs. Test the classifiers.
- Implement the (kernel) k-means clustering algorithm.
- Implement the neighborhood kernel.
- Implement the bagged cluster kernel.
- Test the classifiers using these kernels.
Matlab/Octave solutions