CS 3750 Advanced Topics in Machine Learning
(ISSP 3535)
Time: Tuesday, Thursday
4:00pm5:15pm
Location: Sennott Square, Room 5313
Instructor: Milos
Hauskrecht
Computer Science Department
5329 Sennott Square
phone: x48845
email: milos_at_cs_pitt_edu
office hours: by appointment
Announcements
!!!!!
 The class presentations for your final projects will be held on Thursday, December 11 from 4:005:30pm. The presentations should be 6 minutes long, and highlight the problem, the data, methods used to address the problem, results and conclusions. The presentations in the pdf format should be submitted by email by 3:00pm on December 11, 2014.
 Link to all presentation slides from December 12, 2014.
 The project reports for the class are due at noon on Friday, December 12, 2014. Please submit your report electronically by emailing it to milos@pitt.edu. The typical report should include introduction, background, methodology, experiments, discussion of results, and conclusions/future work sections. References to existing work should be included. The final reports should be selfexplanatory, in that the main ideas and methods should be clearly communicated and written in the report.
 If you are interested you may want to peruse slides and readings material for CS 3750 Machine Learning course offered in Fall 2011
 Topics to be covered next and tentative schedule (Readings can be found here)
 09/23. PCA and SVD (Eric Strobl)
 09/25. Applications of SVD (Daniel Steinberg)
 09/30. Probabilistic Latent semantic analysis (pLSA)(Lingjia Deng)
 10/02. Latent Dirichlet Allocation (LDA) (Mahdi Pakdaman)
 10/07. Probabilistic PCA, and extensions (milos)
 10/09. Probabilistic models of timeseries and sequences (Zitao Liu)
 10/16. Conditional Random Fields (CRF) (Patrick Luo)
 10/21. Latent component analysis and variational methods (milos)
 10/23. Laplacian Eigenmaps for dimensionality reduction (Daniel Steinberg)
 10/28. Spectral clustering (Salim Malakouti)
 10/30. Label propagation on graphs. Semisupervised learning(ChangSheng Liu)
 11/04. Metric, kernel learning (Eric Heim)
 11/06. Active learning (Nils Murrugara Llerena)
 11/11. Multilabel learning (Charmgil Hong)
 11/13. Transfer learning (Jaromir Savelka)
 11/18. Learning from multiple annotators (Gaurav Trivedi)
 11/20. Oneshot, zeroshot learning (Jeya Balaji Balasubramanian)
 11/25. Deep learning (Yoonjung Choi)
 12/02. Anomaly detection (Yanbing Xue)
 12/04. Compressed sensing (Ka Wai Yung)
Links
Course description
Lectures
TBA
Paper presentations
Projects
Abstract
The goal of the field of machine learning is to build computer systems that
learn from experience and that are capable to adapt to their
environments. Learning techniques and methods developed by
researchers in this field have been successfully applied to a variety of
learning tasks in a broad range of areas, including, for example,
text classification, gene discovery, financial forecasting, credit
card fraud detection, collaborative filtering, design of adaptive web
agents and others.
The objective of the Advances Machine Learning course is to expand on the
material covered in the introductory Machine Learning course (CS2750),
and focus on special topics in ML such as, latent variable and dimensionality reduction models, active, transfer, multidimensional learning, learning with multiple annotators, outlier detection. The
course will consist of a mix of lectures, presentations and discussions.
Students will be evaluated based on their participation in discussions, presentations and projects.
Prerequisites
CS 2750 Machine
Learning , or the permission of the instructor.
Readings:
We will use readings from:
 Chris Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
In addition we will use conference and journal paper readings that will be distributed electronically
or in a hardcopy form.
Other (useful) books
 R.O. Duda, P.E. Hart, D.G. Stork. Pattern
Classification. Second edition. John Wiley and Sons, 2000.
 J. Han, M. Kamber. Data mining. Concepts and Techniques. Morgan
Kauffman, 2001.
 T. Mitchell. Machine Learning. Mc Graw Hill, 1997.
 B. Schokopf and A. Smola. Learning with kernels. MIT Press, 2002.
Lectures
Lectures 
Topic(s) 
August 26 
Course Administration
Readings:

August 28 
Review of CS 2750 material
Readings:

September 2 
Markov Random Fields (MRFs)
Readings:
 Bishop. Pattern Recognition and Machine Learning. Chapter 8.

September 4 
Markov Random Fields (MRFs) II: inference,
variable elimination, belief propagation
Readings:
 Bishop. Pattern Recognition and Machine Learning. Chapter 8.

September 9 
Markov Random Fields (MRFs) III: inference, learning
Readings:

September 11 
Markov Random Fields (MRFs) IV: learning
Readings:

September 16 
Monte Carlo methods
Readings:

September 18 
Monte Carlo methods: MCMC
Readings:

September 23 
PCA and SVD (Eric Strobl)
Readings:
Other resouces: A book chapter on SVD

September 25 
Applications of SVD: Latent semantic analysis, Link analysis (Daniel Steinberg)
Readings:
Applications of PCA:
 Information Retrieval.
 Link analysis.

September 30 
Latent Variable Models for
text analysis, information retrieval and link analysis: PLSA, (Lingjia Deng)
Readings:
Probabilistic latent semantic analysis (pLSA)
pLSA for link analysis

October 2 
Latent Dirichlet Allocation (LDA) (Mahdi Pakdaman)
Readings:
Latent Dirichlet Allocation

October 7 
Probabilistic PCA, extensions (Milos)
Readings:
EM algorithms for probabilistic PCA
Extensions of probabilistic PCA
Wray Buntine and Sami Perttu.
Is multinomial PCA multifaceted
clustering or dimensionality reduction AI in statistics, 2003.
Michael Collins, Sanjoy Dasgupta, Robert E. Schapire.
A Generalization of Principal Component Analysis to the Exponential Family

October 9 
Probabilistic models of timeseries and sequences (Zitao)
Readings:

October 16 
Conditional Random Fields (Patrick Luo)
Readings:

October 21 
Latent component analysis and variational methods. (Milos)
Readings:
 Variational methods: Basics
 Variational ML learning for component analysis

October 23 
Laplacian Eigenmaps for dimensionality reduction (Daniel Steinberg)
Readings:

October 28 
Spectral clustering (Salim Malakouti)
Readings:

October 30 
Label propagation on graphs. Semisupervised learning
(ChangSheng Liu )
Readings:

November 4 
Metric, kernel learning
(Eric Heim)
Readings:

November 6 
Active Learning
(Nils Murrugara Llerena)
Readings:

November 11 
Multilabel learning
(Charmgil Hong)
Readings:

November 13 
Transfer learning (Jaromir Savelka)
Readings:

November 18 
Learning from multiple annotators (Gaurav Trivedi))
Readings:

November 20 
Zero and one shot learning (Jeya Balaji Balasubramanian))
Readings:

November 25 
Deep learning, representation learning (Yoonjung Choi))
Readings:
Videolectures:
Other readings:

December 2 
Outlier detection (Yanbing Xu)
Readings:

December 4 
Compressed sensing (Ka Wai Yung, Huichao Xu)
Readings:
Videolectures:

Course
webpage for CS2750, the introductory Machine Learning course from Spring
2014. It is the prerequisite of CS3750.
Readings
Readings will be assigned before the class at which the
discussion on the topic takes place. Most of the
readings will be electronic, however, some readings will be in the
paper form or from the books. See a summary list of Readings for different topics
Paper discussions
Every student is expected to be in charge of at least one topic, present it,
and lead the discussion on the topic during the class.
The readings for each topic will be distributed electronically. The assignment of the papers will be discussed during the first two week of the course.
Projects
There are no homeworks in this course. However, students will be
asked to prepare, submit and present two projects. The first project
will be assigned and due in the middle of the semester. The final
project (due at the end of the semester) and is more flexible: a
student can choose from a set of topics/problems or propose his/her own topic to investigate. If you plan to propose your own project/topic you will need to
submit a short (one page) proposal for the purpose of approval and
feedback. In general, the final project must have a
distinctive and nontrivial learning or adaptive component.
Last updated by milos
on 08/26/2014