Universitatea Babeş-Bolyai Cluj-Napoca
Facultatea de Matematică şi Informatică
Ciclul de studii: Masterat
Domeniul: Informatica
Programul de studii: Optimizarea modelelor informatice - în limba maghiară
Limba de predare: Engleză

SYLLABUS

I. General data
Code	Subject	
MIG1001	Stochastic Modeling of Data	

Semester	Hours: C+S+L	Category	Status	
3	2+1+1	speciality	compulsory	

II. Full status faculty members
Name and	Scientific	Didactic	Chair	Type of activity	
surname	title	title		C	S	L	
CSATO Lehel	Ph.D.	Assoc.Prof.	Programming  Languages and Methods	*			

Associated faculty members
Name and	Scientific	Institution	Type of	Type of activity	
surname	title		position	C	S	L	

III. Course objectives

To familiarize students with the use of probability theory to model real-world problem and to build models. It is important that they will be able to use the models and their parameters for further analysis steps.	

IV. Course contents
Stochastic data modelling is the application to various types of data of algorithms that exploit the rules of probability and allow direct representation of the stochastic nature of the real world. The emphasis in stochastic data modelling is – rather than considering just the algorithm itself – on a careful examination of the type of the data we want to model: whether is of continuous or discrete. It is commonplace that the more data you have, more information you have about the model. However, to extract the information, usually one has to include some prior knowledge into the algorithm. The inclusion of the prior knowledge is possible via the rules of probability.
Prior knowledge is needed to select the model we want to use. The employed algorithm depends on the size and type of available observations – called data. If there are huge data-sets available, then one must use statistical methods, for a few data a lot more emphasis is to be put on the model. The algorithms depend also on the type of the problem we want to solve. In this respect we prefer an algorithm that is developed for classification when classification is to be solved and we would use specific algorithms for discrete data and other set for continuous observations.
The course is aimed as a practical introduction to various models from machine learning and to familiarise the students with the use of randomness and stochastic models.	

V. Bibliography
[1]. Russell S, Norvig P. (2003) Artificial Intelligence: A Modern Approach (Second Edition), Prentice Hall. 
[2]. Bishop C.M (2006) Pattern Recognition and Machine Learning, Springer Verlag.
[2]. Mitchell T (1997) Machine Learning, McGraw Hill.
[3]. Bernardo J.M, Smith A.F.M (2000) Bayesian Theory, John Wiley & Sons.	

VI. Thematic of didactic activities per weeks
Thematic overview of the lectures:

•	Component analysis (weeks 1-4) ref. [2,4,8,9]:
•	Basics of statistical modelling – matrices and eigenvalues [9],	(2 h.)
•	Principal components used in signal de-noising [2,4].	(2 h.)
•	independent components used in blind source separation of signals [8].	(2+2 h.)

•	Bayesian modelling (weeks 5-7) ref. [1,2,3,4,5]:
•	Introduction to various estimation methods [1, 2, 3],	(2 h.)
•	Hierarchical model specification and parameter estimation [1,2,4],	(2 h.)
•	Bayesian modelling: obtaining the posterior and the predictive distributions [4, 5].	(2 h.)

•	Hidden Markov models (HMMs) weeks (8-11) ref. [7,8]:
•	HMM definitions [7], 	(2 h.)
•	Estimating the latent states [7],	(2 h.)
•	Applications of HMMs (1) speech recognition [7,8],	(2 h.)
•	Applications of HMMs (2) gene segmentation [7,8],	(2 h.)

•	Gaussian Process models (weeks 12-14) ref. [5,6]:
•	Joint Gaussians, kernel functions, Functional Bayesian models [4,5,6],	(2 h.)
•	Approximations: using the KL-projection, sparse approximations	(2 h.)
•	Applications of Gaussian process inference.	(2 h.)

VII. Didactic methods used
I will encourage individual work through additional exercises that are given during a lecture and discussed in seminars. This allows students the individual work. The laboratory exercises are chosen so that the students get a grip on the method they choose to apply to a data set.	

VIII. Assessment
The exam will consist of  (40%) a presentation of a topic chosen in the first 6 weeks of the semester, (20%) the solution of the laboratory examples, (40%) oral examination based on the topics of the lectures and the seminars presented by the students.	

IX. Additional references
[1]. 	Russell S, Norvig P. (2003) Artificial Intelligence: A Modern Approach (Second Edition), Prentice Hall.
[2]. 	MacKay D.J.C (2003) Information Theory, Inference and Learning Algorithms, Cambridge University Press, HTTP: http://wol.ra.phy.cam.ac.uk/mackay/itila/book.html.
[3]. 	Rasmussen C.E, Williams C.K.I (2006) Gaussian Processes for Machine Learning, The MIT Press.
[4]. 	Rabiner L.R, Juang, B.H (1986) An introduction to Hidden Markov models, IEEE ASSP Magazine, pp: 4-15.
[5]. 	Durbin R, Eddy S.R, Krogh A, Mitchison G (1999) Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press.
[6]. 	Hyvärinen A, Karhunen J, Oja E (2001) Independent Component Analysis, Wiley-Interscience.
[7]. 	Barto A. (2002): Statistical Pattern Recognition, John Wiley & Sons.

	Dean,							Chair head,

	Date,							Course responsible,
								Assoc.Prof. CSATO Lehel, Ph.D.