CS1674: Homework 2 - Programming

Due: 2/1/2018, 11:59pm

This assignment is worth 60 points. The provided files can be found on CourseWeb.

Part I: Content-Aware Image Resizing (35 points)

You will implement one part of the content-aware image resizing technique described in Shai Avidan and Ariel Shamir's SIGGRAPH 2007 paper, "Seam Carving for Content-Aware Image Resizing", available here. First read through the paper, with emphasis on Section 3. Most functions will be provided for you, and some you will have to write. The goal is to get more practice with Matlab, examine the performance of the method on different kinds of input images, and produce visually interesting results.

Here are some example results.

Put each function you write in a separate file called [function-name].m and submit all of them, excluding ones that were provided with the assignment. All the functions you are asked to write are brief, but some require more thought than others.

Here is the code you need to develop:
  1. [5 points] Function [energyImage, Ix, Iy] = energy_image(im) - to compute the energy at each pixel using the magnitude of the x and y gradients.

    Inputs: Outputs:
  2. [10 points] Function displaySeam(im, seam, seamDirection) - to display the selected type of seam on top of the input image.

    Inputs:
    Tips:
  3. [5 pts] Function [reducedColorImage] = reduceWidth(im, display_flag) - to reduce the width of your image in a content-aware away. The function should call the energy_image, cumulative_minimum_energy_map, and find_optimal_vertical_seam functions.

    Inputs: Outputs:
    Tips:
  4. [5 points] Function [] = generate_results(filename, reduceAmt, reduceWhat) - to run the functions you wrote, and the provided functions, on an image, and show the results. In your function, after reading in the image, you should call reduceWidth or reduceHeight as many times as needed to reduce the width/height by the requested amount, but should only set the display flag to true in the first iteration. You should also display, in a 1x3 subplot: (a) the original image, (b) the final content-aware-reduced image, and (c) the content-agnostic reduction using a simple imresize. Label the figure with a title indicating what was reduced, and by how many pixels. Label the subfigures with titles "original", "content-aware", and "standard" (put one title after each subplot).

    Inputs:
Required visual results:
  1. [5 pts] Include the visual results of running generate_results on the provided prague.jpg and shrinking its height by 100 pixels. Also include the results of shrinking the width of the provided mall.jpg by 100 pixels. For both images, include:

  2. [5 pts] Provide one interesting result on a different image of your choice. The goal is to form some perceptually pleasing outputs where the resizing better preserves content than a blind resizing would, or an example where the output looks unrealistic or has artifacts ("failure cases"). Be creative in the images you choose, and in the amount of combined vertical and horizontal carvings you apply. If you have a large image, try applying imresize first to resize the image to a small size, in order to make the computations faster and allow you to try more ways of transforming the image. Create a 1x3 subplot and display the following three: (a) the original image, (b) resizing using imresize, and (c) content-aware resizing in your submission. Use a title in your figure to say by how much you reduced the size, and in which direction. Name the file interesting.png and include it in your submission.
Provided functions: Part II: Image Responses with Filters (15 points)

In this problem, you will measure the responses of images to different filters. For each image, you will compute the response of each image pixel to the filters from a "filter bank" as discussed in class. In a later assignment, you will use these responses to filters to compute image representations.
  1. Download these images: cardinal1, cardinal2, leopard1, leopard2, panda1, panda2.
  2. Download the script makeLMfilters from the Leung-Malik filter bank (also provided on CourseWeb), and run it (F = makeLMfilters;) to get a set of filters. Each filter F(:, :, k) is of size 49x49, and there are 48 filters.
  3. [5 pts] In a script filter_responses.m, use a cell array to store the list of filenames so you can loop over them (e.g. filenames{i}). Read in all images, convert them to grayscale, and resize them to the same square size (e.g. 100x100), so that the visual map of responses (filtering outputs) can be more comparable across images. Compute the cross-correlation of each image with each of the 48 filters using imfilter.
  4. [5 pts] For each filter, generate a 2x4 subplot showing the following subplot rows: (1) the filter and a blank subplot, (2) the responses to the cardinal images, (3) the responses to the leopard images, and (4) the responses to the panda images.
  5. [5 pts] Choose and include in your submission (1) one filter where the responses for the same animal category are similar, while the responses for different animal categories are quite distinct; and (2) one filter where responses of different animals look fairly similar. Name the files same_animal_similar.png and different_animals_similar.png. Explain what you see in each case. See an example for one filter below.

    Tips:


Part III: Hybrid Images (10 points)

In this problem, you will create a hybrid image (which looks like one thing zoomed in and another zoomed out) like the one shown in class.
  1. Download one pair of images:
  2. [3 pts] In a script hybrid_image.m, read in the first image in the pair as im1 and the second as im2. Convert both images to grayscale, and resize them to the same square size (e.g. 512x512).
  3. [2 pts] Apply a Gaussian filter to both, using e.g. imgaussfilt(im1, 10, 'FilterSize', 31). Save the results as im1_blur, im2_blur.
  4. [2 pts] Obtain the detail image, by subtracting im2_blur from im2, and save the result as im2_detail.
  5. [3 pts] Now add im1_blur and im2_detail, show the image, save it as 'hybrid.png', and include it with your submission. Play with scaling it up and down (by dragging the corner of the Matlab figure) to see the "hybrid" effect.

Submission: Please submit the following files:
Acknowledgement: Part I is partially based on an exercise by Kristen Grauman.