Time: Monday, Wednesday
2:003:20pm,
Location: Sennott Square, Room 5313
Instructor: Milos
Hauskrecht
Computer Science Department
5329 Sennott Square
phone: x48845
email: milos@cs.pitt.edu
office hours: by appointment
TA: Ali Alanjawi
Computer Science Department
5404 Sennot Square
phone: x41185
email: alanjawi@cs.pitt.edu
office hours: Tuesday, Thursday: 1pm  4pm
Quiz:
Final projects:
!!! CS 3750 Advanced Topics in Machine Learning (ISSP 3535) !!!
Additional readings for the course including topics still to be covered
Course description
Lectures
Homeworks
Term projects
Matlab
The goal of the field of machine learning is to build computer systems that learn from experience and that are capable to adapt to their environments. Learning techniques and methods developed by researchers in this field have been successfully applied to a variety of learning tasks in a broad range of areas, including, for example, text classification, gene discovery, financial forecasting, credit card fraud detection, collaborative filtering, design of adaptive web agents and others.
This introductory machine learning course will give an overview of many models and algorithms used in modern machine learning, including linear models, multilayer neural networks, support vector machines, density estimation methods, Bayesian belief networks, mixture models, clustering, ensamble methods, and reinforcement learning. The course will give the student the basic ideas and intuition behind these methods, as well as, a more formal understanding of how and why they work. Students will have an opportunity to experiment with machine learning techniques and apply them a selected problem in the context of a term project.
Prerequisites
Knowledge of matrices and linear algebra (CS 0280), probability (CS 1151), statistics (CS 1000), programming (CS 1501) or equivalent, or the permission of the instructor.
Lectures  Topic(s)  Assignments  

January 6  Course administration  .  
January 8 
Introduction.
Readings:

.  
January 13 
Designing a learning system.
Readings:

.  
January 15  Matlab tutorial.  .  
January 22  Evaluation of predictors
Readings:

Homework 1 (Data files for HW1) 

January 27  Density estimation
Readings:


January 29  Density estimation II
Readings:

Homework 2 (Data files for HW2) 

February 3  Linear regression
Readings:


February 5  Linear regression
(cont). Classification with linear models. Readings:

Homework 3 (Data files for HW3) 

February 10  Classification with
linear models.
Readings:

.  
February 12  Multiway
classification. Bayesian decision theory. Readings:

Homework 4 (Data files for HW4) 

February 17  Class cancelled due to snow storm  .  
February 19  Multilayer neural
networks
Readings:

Homework 5 (Data files for HW5) 

February 24  Support vector machines
Readings:

.  
February 26  Bayesian belief networks
Readings:

Homework 6 (Data files for HW6) 

March 10  Bayesian belief
networks. Inference
Readings: 
.  
March 12  Learning Bayesian belief
networks.
Readings:

.  
March 17  Midterm  .  
March 19  Density
estimation with hidden variables. EM.
Readings:

Homework 7 (Data files for HW7) 

March 24  Project proposals  .  
March 26  Expectation
maximization. A naive Bayes model with hidden class and missing values. Mixture of Gaussians. Readings:

Homework 8 (Data files for HW8) 

March 31 
Clustering. Nonparametric density estimation.
Readings:


April 2 
Dimensionality reduction.
Readings:


April 7 
Decision trees.
Readings:


April 9 
Ensamble methods. Mixture of experts. Bagging.
Readings: 

April 14 
Ensamble methods. Boosting.
Readings:


April 16 
Reinforcement learning.
Readings: 

April 21  Quiz. Reinforcement Learning (cont.) 

April 23  Term projects: reports due at 2pm. No class.  
April 25  Term projects: presentations. 
The homework assignments will have mostly a character of projects and will require you to implement some of the learning algorithms covered during lectures. Programming assignmets will be implemented in Matlab. See rules for the submission of programs.
The assignments (both written and programming parts) are due at the beginning of the class on the day specified on the assignment. In general, no extensions will be granted.
Collaborations:
You may discuss material with your fellow students, but the report and
programs should be written individually.
The term project is due at the end of the semester and accounts for a significant portion of your grade. You can choose your own problem topic. You will be asked to write a short proposal for the purpose of approval and feedback. The project must have a distinctive and nontrivial learning or adaptive component. In general, a project may consist of a replication of previously published results, design of new learning methods and their testing, or application of machine learning to a domain or a problem of your interest.
Matlab is a mathematical tool for numerical computation and manipulation, with excellent graphing capabilities. It provides a great deal of support and capabilities for things you will need to run Machine Learning experiments. Upitt has a number of Matlab licences running on both unix and windows platforms. Click here to find out how to access Matlab at Upitt.
Matlab tutorial files from 01/15/03.
Other Matlab resources on the web:
Online
MATLAB documentation
Online
Mathworks documentation including MATLAB toolboxes
Course webpage from Spring 2002