Kernel methods and semi-supervised learning

(short course)

Slides

[pdf]
Contents:
Bibliography:
  1. B. Schölkopf, A. J. Smola. Learning with Kernels. MIT Press, Cambridge, Mass., 2002. [link]
  2. O. Chapelle, B. Schölkopf, A. Zien. Semi-Supervised Learning. MIT Press, 2006. [link]
  3. F. Jäkel , B. Schölkopf , F. A. Wichmann. A Tutorial on Kernel Methods for Categorization. Journal of Mathematical Psychology 51(6), pp. 343-358, 2007. [pdf]
  4. X. Zhu. Semi-Supervised Literature Survey. TR 1530, University of Wisconsin - Madison, 2008. [pdf]

Datasets

Programming exercises

  1. Implement the (kernel) kNN algorithm (for binary classification).
  2. Implement the (kernel) centroid algorithm (for binary classification).
  3. Create two functions: one for transforming a kernel (Gram) matrix to a (squared) distance matrix and another one for the inverse process (distance -> kernel).
  4. Using the linear, polynomial and Gaussian kernels try your classification algorithms on the above datasets.
  5. Implement the Floyd-Warshall algorithm.
  6. Construct kNN and eps-NN graphs using the Floyd-Warshall algorithm and then build kernels from these graphs. Test the classifiers.
  7. Implement the (kernel) k-means clustering algorithm.
  8. Implement the neighborhood kernel.
  9. Implement the bagged cluster kernel.
  10. Test the classifiers using these kernels.

Matlab/Octave solutions