CS1674: Homework 10 - Programming

Setup:


1. You will be connecting to the server via SSH. If you are using a Windows machine and haven't used SSH before, you will need to first download a SSH client such as PuTTY. You can download PuTTY from here. If you are using a Mac or Linux, you already have ssh installed.
2. If you are off campus and don't have the VPN installed, you will need to first connect to Pitt's server and then ssh over to the class4 server. If you are on a Mac or Linux, open a terminal and type: ssh USERNAME@unixs.cis.pitt.edu (where USERNAME is your Pitt username) and press enter to connect to the server. If you are on Windows, open PuTTY and for the host name, enter unixs.cis.pitt.edu and click Open to connect to the server. You will need to enter your Pitt username and password when prompted by the server. Once you are connected to that server, you can then type: ssh class4.cs.pitt.edu and press enter to connect to the class4 server. If you are on campus, or have the VPN installed, you should be able to connect to the class4 server directly.
3. Once you are logged in, you will be taken to your AFS home directory and will probably see a "public" and "private" directory (if you have not changed these yourself). Make sure to put any assignment files you are working on in the private directory (or another directory which no one except you can access).
4. You can either write your Matlab assignment file on your own computer and transfer it to the server using scp (on Mac or Linux) or WinSCP (you'll need to download this on Windows) to run it on the server or directly write the Matlab assignment file on the server using a text editor such as nano. On Mac or Linux an scp command to copy a file you've written to the server might look like this (where my username is clt29): scp your_matlab_assignment_file.m clt29@class4.cs.pitt.edu:/afs/pitt.edu/home/c/l/clt29/private/  This command will copy the Matlab file from your computer to your AFS storage space. If you are on Windows and install WinSCP, you will be presented with a GUI interface where you can drag and drop files from your computer to your AFS space.
5. Caffe requires libraries to be visible to Matlab for it to work. We need to tell Matlab where these libraries are located. Before starting matlab, type the following directly into the bash shell on the server: export LD_LIBRARY_PATH=/tmp/caffe/build/lib:/usr/local/lib
6. To launch Matlab on the server, type matlab -nodisplay. We include the -nodisplay so that Matlab does not attempt to open the GUI interface. You can then begin typing commands as you normally would in Matlab or run a script that you write.
7. Matlab needs to know where to find its interface to the Caffe library. The Matlab addpath command allows you to specify the location of files your scripts need to run. Add the following line to the top of your script: addpath('/tmp/caffe/matlab/')

Loading the model and data, extracting features:


8. We will now load in a pretrained CNN model. The model we are loading has been trained on 1.4M images to classify images into 1000 classes (which aren't necessarily the animals we will be classfying). Add the following line to your script: net = caffe.Net('/tmp/caffe/models/deploy.prototxt', '/tmp/caffe/models/alexnet.caffemodel', 'test')
The caffe.Net function loads a network model for use in Matlab. The first argument specifies a file containing the network structure which tells Caffe how the various network layers connect. The second argument specifies the learned model to load containing the weights learned during training and copies those weights into the network structure created by the first argument.  The final argument tells Caffe to load the network in test mode, rather than train mode. You will see a lot of output appear once you execute this command which you can ignore.
9. We need to subtract the mean of the train data from each image before the CNN classifies it. Load the mean image of the train data by using the following command: image_mean = caffe.io.read_mean('/tmp/caffe/models/imagenet_mean.binaryproto');
10. The data for this assignment is located at /tmp/data/. You will find 20 folders with animal names. Each folder contains approximately 100 images of the animal specified. You can use the matlab imageSet function with the 'recursive' parameter to obtain a list of folders and all images in them for easy processing. The name of the folder is the image's ground truth label. For each image, you will need to extract three features from the CNN and store them in a variable. Later, you will train a linear SVM on each of the features.
11. In order to extract the features for an image, first load the image in Matlab using the caffe.io.load_image('/path/to/image/to/load') function. Do not use Matlab's imread function. Caffe expects images in BGR format (instead of RBG), needs to have the width and height dimensions flipped, and needs to be in single precision. The load_image function will do all of these things for you automatically. After loading the image, use Matlab's imresize function to resize the image to height and width 227 (which is what this model expects. After resizing the image, subtract the image_mean we loaded previously from the image. You can then run the image through the neural network by using the command net.forward({image});
12. Once the image has been run through the neural network, we are ready to extract features from the network. You will be extracting features from the 3 fully connected layers of the network, 'fc8', 'fc7', and 'fc6'. To extract an image feature from the network, use the command net.blobs(feature_name).get_data() Store the features you extract somewhere for training the SVM.

Training and testing the SVM:


13. Randomly split data into a 80% train set and a 20% test set.
14. Train 3 linear SVMs using Matlab's fitcecoc function. To specify that Matlab should train linear SVMs, pass the following templateSVM to the fitceoc function: templateSVM('Standardize',1,'KernelFunction','linear'); Matlab will also automatically standardize your train and test data for you.
15. Test each of your SVMs on the test set and report the accuracy. Also include a confusion matrix for one of the features using the confusionmat function and include it in your submission. What do you observe about the types of errors the network makes? Enter your answer in a new text file observations.txt.

• [30 pts] Extracting the features from the CNN.
• [10 pts] Training the SVM.
• [10 pts] Accuracy, confusion matrix, and report.