CS1674: Homework 10 - Written

1. Why do we say neural networks are not linear classifiers?
2. Briefly describe one way in which artificial neurons resemble brain neurons.
3. Say a vision system predicts the following scores for a cat image: 10 for the category "cat," 5 for the category "dog," and 3 for the category "cow." Another system predicts scores 8, 6 and 1 for the same categories. Is the SVM loss the same or different for these two systems? What about the softmax loss? Why?
4. Briefly, how does gradient descent work?
5. What is mini-batch?
6. How can we prevent overfitting in a neural network?