CS 1538: Introduction to Simulation

Course Project

Outline

  1. Introduction
  2. Details
  3. Report
  4. Presentation
  5. Deadlines and Suggested Time Table
  6. Final Submission
  7. Peer Evaluation
  8. Groups
  9. Grading

Introduction

For this group project, you will analyze a system following the procedures discussed in class. You will follow all of the major steps in running a simulation study. The final deliverables for the project are a group presentation and a final paper.

Details

Identify a system that you would like to simulate. Consider things you might want to improve about the system. Identify what you might need to observe about the system to both simulate it and to perform experiments on possible improvements. Observe the system for a sufficient period of time, collecting relevant information. Meanwhile, start implementing a simulation of the system, using your observations to inform the implementation of the simulation. Perform input modeling on your observations. Use your input models to generate random variates for your simulation. Simulate the system as it currently is, recording any information that might be useful for experiments. Implement your possible improvements and rerun the simulation, recording the same information. Analyze the results to determine whether your improvements actually improve the system.

Homework 4 gives a good example of most of these steps. However, the input modeling has been done for you and the optimization parameters have been decided already. You must do these for the project.

Report

You should submit a clearly organized, well-written report, including properly formatted citations and a bibliography section if applicable. (Note: do not embed bibliographical information as footnotes.) Even though your project report is not a typical research paper, you might find this website helpful: http://www.columbia.edu/cu/biology/ug/research/paper.html. It offers a general skeleton for scientific reports as well as the appropriate citation format.

Your report will be about 8 pages long. If you find that you have less than 8 pages to write, do not try to pad the report with a longer introductory and/or background section. It is almost always better to just turn in a shorter report. Similarly, if you find that 8 pages are not enough to explain your work, it's fine to turn in a longer report. Below is a suggested outline for your report. You'll notice that the general organization is similar to your presentation.

Suggested Outline

  1. Introduction (1-1.5 pages) – It should clearly lay out the general problem domain as well as the specific problem(s) you are trying to address. It should also briefly summarize your approach, experiment, and findings. Basically, a reader should be able to get an overall sense of what the report is about from reading the introduction alone.
  2. Background (1 page) – In this section, you elaborate on the problem domain. You can also use this section to describe relevant previous work.
  3. Approach/Method (2-3 pages) – This is where you describe your full simulation system. Talk about how you collected the data. Show how you performed input modeling; what distribution did you decide on and how well did your data fit that distribution. Describe your model. Provide graphs and flowcharts as necessary. As in the presentation, discuss what assumptions and simplifications you've made and give justifications for why they are reasonable.
  4. Experimental Setup (1 page) – For each experiment you performed, talk about the goal of the experiment (in terms of the problem you are trying to address), what you expected to find, what the results actually showed, and what the implications of these results are. Include relevant tables and graphs – be sure to explain how to read these tables and graphs if you include them.
  5. Conclusions and Future Work (at most 1 page) – Briefly summarize what your results show. Were there any shortcomings in your experiment (i.e. to what extent was your approach successful at addressing the problem you wanted to tackle?). Talk about what you might do differently or what you might do next if you were to continue.

You may wish to refer to homework 4's report description for more details of what to include.

Presentation

Logistics

Each team will have approximately 15 minutes for the presentation, followed by up to 5 minutes for questions (exact time lengths will be determined later). You should rehearse at least once to make sure that your presentation comes under the time limit. To minimize the amount of time switching between teams, we will upload all the presentation slides onto one computer. On the day of your presentation, you should plan to arrive at least 5 minutes earlier to class to check the setup. We will begin promptly at the start of class. The presentation schedule will be posted later in the semester.

Presentations will be graded according to this rubric: Presentation Rubric

One purpose of these presentations is to get feedback on your projects before you turn in the final report. To accomplish this, the rest of the class will fill out a comment sheet for each presentation and will be collected at the end of class. These comments will then be emailed to your group. The comment sheets look like this: Comments Sheet. Note that you will be graded on your comments.

Slide Organization Suggestions

Do not have too much information on one slide, but do not have too many slides either. A good rule of thumb is that you should spend about a minute per slide. So, plan on having between 10-15 slides. Below is a suggested outline:

  1. Title slide (1 slide) – Title of project and names of team members
  2. Motivation (1 slide) – What's interesting/challenging about the problem? Why did you choose this problem?
  3. Specific problem statement (1-2 slides) – What precisely is your project topic? For example, if your overall problem is to do a simulation of elevators, your specific problem statement might be to compare two particular elevator pickup and dropoff policies.
  4. Approach (1-3 slides) – What is your approach? You should be precise enough for someone else to rerun your experiment. If appropriate, you may want to illustrate the process with a flow chart or other relevant diagrams.
  5. Experimental Setup (1-2 slides) – Explain the goal of the experiment and how you are setting it up.
  6. Results (1-2 slides) – Present the results from your experiments. What do the numbers tell you? How might you modify the system you were studying given your results?
  7. Discussion (1-2 slides) – What are the challenges you ran into while conducting this project? Were there simplifying decisions that you had to make? What would you do differently if you had more time?
  8. Conclusion (1 slide) – Summary of your problem and your findings, and some future experiments you'd recommend running to continue examining this problem

Deadlines and Suggested Time Table

Below is a suggested time table for completing the project. Bold dates are hard deadlines, when a graded element is due, and are not suggestions.

It would be wise to work on some of this in parallel.

Final Submission

Zip your source code files, your report, and a README on how to use your simulation program into one zip file and upload it to CourseWeb, in the Project location.

Peer Evaluation

Complete and upload the peer evaluation form by Tuesday, April 26 at 11:59 pm. Each group member must complete and upload their own evaluation form. The form will be used to determine how much each member contributed to the group project and will be used to help assign project grades. The form can be downloaded here. It should be submitted on CourseWeb, in the Project section, under "Peer Evaluation".

Groups

Below are the groups and their topics reported on 3/19.

Grading

Below are the graded items for the project and what percentage of the overall project grade each item is worth:

Based on feedback from peer evaluations, some group members may receive a different project grade than other group members.