CS1699: Homework 1
Due: 9/22/2015, 11:59pm
Instructions: Please provide your code and image results (for parts I, III and IV) and your written answers (for parts I, II and IV). Your written answers should be in the form of a PDF or Word document (.doc or .docx). Your code should be written in Matlab. Zip or tar your written answers, image results and .m files and upload the .zip or .tar file on CourseWeb -> CS1699 -> Assignments -> Homework 1. Name the file YourFirstName_YourLastName.zip or YourFirstName_YourLastName.tar.
Note: It's ok to look up what Matlab functions accomplish a given task, but you are only allowed to use built-in Matlab functions. For part III, you have to write your own code and, other than basic and image display functions, can only use Matlab's imfilter and fspecial functions and the linked tight_subplot function. If you have questions about what you can use, ask the instructor.
Part I: Matlab Basics (15 points)
- Generate a 1000000x1 (one million by one) vector of random numbers from a Gaussian distribution with mean approximately 0 and standard deviation approximately 5. Use Matlab's randn function.
- Add 1 to every value in this list, by using a loop. To get the number of loops you need to run, use Matlab's size function. Time this operation, print the number, and write the total time taken to add 1 to every number in your answer sheet.
- Now add 1 to every value in the list, without using a loop. Time this operation, print the time and write it down. Use a different function to print the number than you did above.
- Read in this image into Matlab as a matrix, and write down its dimensions.
- Convert the image into grayscale.
- Find the darkest pixel in the image, and write its value in your answer sheet. Hint: Convert to a vector first, and use Matlab's ind2sub function.
- Consider a 31x31 square (a square with size 31 pixels) that is centered on the darkest pixel. Replace all pixels in that square with white pixels.
- Make a new figure, display the modified image and save it as a file. Use saveas(gcf, '[filename].png') to save your image.
Part II: Short Answer Problems (15 points)
- Give an example of how one can exploit the associative property of convolution to more efficiently filter an image.
- This is the input image: [0 0 1 1 0 0 1 1]. What is the result of dilation with a structuring element [1 1 1]?
- Name two specific ways in which one could reduce the amount of fine, detailed edges that are detected with the Canny edge detector.
Part III: Image Pyramids (15 points)
- Choose an image that has an interesting variety of texture (from Flickr or your own images). The image should be at least 640x480 pixels and converted to grayscale. Use the Matlab function rgb2gray.
- Write code for a Gaussian and Laplacian pyramid of level N (use for loops). In each level, the resolution should be reduced by a factor of 2. Use the Matlab function imfilter.
- Show a Gaussian and Laplacian pyramid of level 5 for your chosen image using your code.
You can (but don't have to) use the tight_subplot function to format your plot. Your displayed images for the Gaussian and Laplacian pyramids should look something like the image below. Note that the image at the bottom-right can be skipped.
- Include your code (one function to generate the pyramids, and another to display them), the original image you chose, and an image that shows the image pyramids, in your submission.
Part IV: Seam Carving (55 points)
For this exercise, you will implement one part of the content-aware image resizing technique
described in Shai Avidan and Ariel Shamir's SIGGRAPH 2007 paper, "Seam Carving for Content-Aware Image Resizing", available here. The goal is to implement
the method, and then examine and explain its performance on different kinds of input images.
First read through the paper, with emphasis on sections 3, 4.1, and 4.3. Note: choosing the next
pixel to add one at a time in a greedy manner will give sub-optimal seams; the dynamic
programming solution ensures the best seam (constrained by 8-connectedness) is computed.
Use the dynamic programming solution as given in the paper and explained in class.
Write Matlab functions as below. Save each of the below functions in a separate file called [function-name].m and
submit all of them.
- [5 points] energyImage = energy_image(im) - to compute the energy at each pixel using the magnitude of the
x and y gradients (equation 1 in the paper; sqrt(dx^2+dy^2)). There are at least two ways to compute the gradient in each direction (including with a filter), but don't use Matlab's imgradient function.
The input im should be a nrows-x-ncols-x-3 matrix of datatype uint8, e.g. the output of imread on a color image. However, you need to convert it to grayscale before computing the gradients, using rgb2gray.
The output should be a 2D matrix of datatype double.
- [5 points] M = cumulative_minimum_energy_map(energyImage,seamDirection) - to compute minimum cumulative energy. The input energyImage should be a 2D matrix of datatype double.
(It should be the output of the energy_image function defined above.). The input seamDirection should be
the strings 'HORIZONTAL' or 'VERTICAL'. The output must be a 2D matrix of datatype double.
- [5 points] verticalSeam = find_optimal_vertical_seam(M) and horizontalSeam = find_optimal_horizontal_seam(M) - to compute the optimal
vertical and horizontal seams. The input should be a 2D matrix of datatype double. (It can be taken from the
output of the cumulative_minimum_energy_map function defined above). The output must be a
vector containing the column indices (row indices, respectively) of the pixels which form the seam for each row (each column, respectively).
- [5 points] displaySeam(im, seam, seamDirection) - to display the selected type of seam on top of an image. The input
im should be the result of an imread. seamDirection should be the strings 'HORIZONTAL' or 'VERTICAL'. seam should be
the output of find_optimal_vertical_seam or find_optimal_horizontal_seam. The output should
display the input image and plot the seam on top of it. To plot points on top of a displayed image, use imshow(im); followed by hold on;
followed by plot(...). The origin of the plot will be the top
left corner of the image. Note that for the plot, rows are the y axis and columns are the x axis.
- [5 points] Functions with the following interface:
[reducedColorImage,reducedEnergyImage] = reduceWidth(im, energyImage)
[reducedColorImage,reducedEnergyImage] = reduceHeight(im, energyImage)
These functions should take as inputs a) a 2D matrix energyImage of datatype double and b) a nrows-x-ncols-x-3
matrix im of datatype uint8. The input energyImage should be the output of the energy_image function.
The output must return 2 variables: a) a 3D matrix reducedColorImage same as the input image, of datatype uint8, but with its width or
height reduced by one pixel; b) a 2D matrix reducedEnergyImage, of datatype double, the same as energyImage, but
with its width or height reduced by one pixel.
Matlab hints:
- Useful functions: imfilter, im2double, fspecial, imread, imresize, rgb2gray,
imagesc, imshow, subplot;
- Use saveas(gcf, '[filename].png') to save your images.
- Be careful with double and uint8 conversions as you go between computations with
the images and displaying them -- filtering should be done with doubles.
Answer each of the following, and include image displays where appropriate. Save your code for each of the parts below as a separate .m script. Name the scripts part4_a.m, part4_b.m, part4_c.m, and part4_d.m. Submit the scripts along with the functions above. Also submit the image results, as indicated below. Your grader should be able to run your scripts and get the image results you submitted.
- [5 points] Run your reduceHeight function on the provided prague.jpg and shrink the height by 100 pixels. Then run your
reduceWidth function on the provided mall.jpg and shrink the width by 100 pixels. Also show what standard image resizing would do (use B = imresize(A, [numrows numcols])). Display the outputs, save them, and submit them.
- [5 points] Display, save and submit (i) the energy function output (total gradient magnitudes e1(I)) for the
provided images prague.jpg and mall.jpg, and (ii) the two corresponding cumulative minimum
energy maps (M) for the seams in each direction (use the imagesc function). Explain
why these outputs look the way they do given the original image's content.
- [5 points] For the same two images, display, save and submit the original image overlaid with (a)
the first selected horizontal seam and (b) the first selected vertical seam. Explain why
these are the optimal seams for these two image.
- [15 points] Use your system with different kinds of images and
seam combinations, and see what kind of interesting results it can produce. The goal is
to form some perceptually pleasing outputs where the resizing better preserves content
than a blind resizing would, as well as some examples where the output looks unrealistic
or has artifacts ("failure cases"). Include results for at least three images of your own
choosing. Include an example or two of a "bad" outcome. Be creative in the images you
choose, and in the amount of combined vertical and horizontal carvings you apply. Try to
predict types of images where you might see something interesting happen. It's ok to
fiddle with the parameters (seam sequence, number of seams, etc) to look for interesting
and explainable outcomes.
For each result, include the following things, clearly labeled (see title function):
- the original input image,
- your system's resized image,
- the result one would get if instead a simple resampling were used (via
Matlab's imresize),
- the input and output image dimensions,
- the sequence of enlargements and removals that were used, and
- a qualitative explanation of what we're seeing in the output.
Acknowledgement: The author of Part II and Part IV is Kristen Grauman, and of Part III is Derek Hoiem. Devi Parikh and Yong Jae Lee added further specifications to Part IV, which are also used.