CS 1538: Introduction to Simulation

Lecture Questions

Answers will not be posted for the questions below. However, if you wish to check your answers, you may email the instructor or the TA or meet with them during office hours.

Lecture 1: Class Introduction & Simulation Overview
1. True/False: You can submit an assignment late and still receive full credit if you encountered problems connecting/uploading to CourseWeb.
2. True/False: "System" and "model" are synonyms.
3. Which of the following is the definition of "entity": (select the best answer)
  1. an object of interest in the system
  2. a property of an object
  3. some action performed by an object
  4. a variable in the model
4. Think of a system to simulate (perhaps one you would like to simulate for the course project). Identify the question(s) you would want to ask about that system. Then, list the entities, attributes, activities, events, and state variables that are relevant for the system and the question(s) you want to ask.
5. How are attributes and state variables related? What are the differences between them?
6. What are the techniques for advancing time in a simulation? What are the advantages and disadvantages to each?

Lecture 2: Simulation Overview & Probability Review

Which of the following are not reasons to use simulations:
1. When direct experimentation is not possible
2. To diagnose the real system
3. When you have a small sample size from the real system and need more data for statistical results
4. To preview the real system before implementing it
5. When the system is very complex
6. To test new designs or policies on a system
7. To gain insights into how entities in a system interact
Which of the following is the definition of "probability distribution":
1. A variable that can take on ay value of a given experimental result
2. The set of possible outcomes of a given experiment
3. The set of possible outcomes of an experiment and the likelihood of each outcome
Which of the following is the definition of "sample space":
1. A variable that can take on ay value of a given experimental result
2. The set of possible outcomes of a given experiment
3. The set of possible outcomes of an experiment and the likelihood of each outcome
True/False: The joint probability is the probability that two or more random variables have particular values (i.e. Pr(A=a and B=b), where A & B are random variables and a & b are values).
True/False: The conditional probability is the probability that one random variable has a particular value, given another random variable has a particular value (i.e. Pr(A=a | B=b), where A & B are random variables and a & b are values).
What is the sample space and probability distribution of an unfair six-sided die where faces 1, 2, and 3 are twice as likely to face up as faces 4, 5, and 6?
What is the probability that a roll of two fair six-sided dice will result in a sum that is greater than 7?

Programming Practice: A student asked whether there was anything he could do to prepare for the programming assignments later in the semester. Although you will not need to learn a new programming language for this course (assignments can be written in Java, or any other language the TA is familiar with), you may find it helpful to work on writing a simple simulation.

Try simulating the single-server restaurant from the lecture slides:

First customer arrives at time 0.
Each subsequent customer arrives 6 - 10 minutes after the previous (uniformly distributed, discretized on the minute).

When a customer reaches the front of the line, they place their order and wait for the food to be served. No other customer can place an order during this time. This service time follows this distribution:

Time	3 min.	4 min.	5 min.	7 min.	8 min.	9 min.
Probability	0.1	0.2	0.3	0.25	0.1	0.05

Record sufficient information to answer these questions:

How often is the server/employee/cook (all the same person) idle?
What's the chance that a customer has to wait in the line?
On average, how long is a customer's wait in the line?
On average, how long is the customer's wait for their food after they've placed their order?

After completing the simulation above, try making changes to the system to improve it.

Lecture 3: Probability Distribution Properties, Discrete Probability Distributions, & Continuous Probability Distributions
1. Which of the following represents the average deviation for a probability distribution?
  1. Correlation
  2. Expectation Value
  3. Joint Probability
  4. Probability
  5. Variance
2. Which of the following represents the average value of a probability distribution?
  1. Correlation
  2. Expectation Value
  3. Joint Probability
  4. Probability
  5. Variance
3. Which distribution represents repeating Bernoulli Trials until a success has occurred?
  1. Bernoulli Distribution
  2. Binomial Distribution
  3. Geometric Distribution
  4. Negative Binomial Distribution
  5. Poisson Distribution
4. Which distribution represents a fixed number of repetitions of Bernoulli Trials?
  1. Bernoulli Distribution
  2. Binomial Distribution
  3. Geometric Distribution
  4. Negative Binomial Distribution
  5. Poisson Distribution
5. True/False: The Poisson Distribution is a continuous distribution.
6. We are running a simulation of people riding the bus. We are only concerned with dropping passengers off (like 28X inbound). Assume that each passenger has a 20% chance of getting off at any stop.
  1. Suppose there are 50 passengers on the bus. What is the chance that exactly 9 people got off at the next stop?
  2. What is the chance that a passenger stays on the bus for at least 5 stops?
  3. More generally, suppose there are P passengers on the bus, how should we simulate people getting off at bus stops?
7. Suppose the chance that a bus breaks down is 0.15% and that there are 500 buses in service. Calculate the chance that 10 buses broke down?
8. Suppose we are running a simulation to study the effectiveness of a tutoring service. Suppose a student has 70% chance of getting a question correct, and suppose the tutor stays with the student until they get 20 questions correct.
  1. What's the probability that the tutor will work with the student for 30 questions?
  2. What's the probability that the tutor will work with the student for 15 questions?
  3. How would you model the tutor time?
9. Consider the experiment of tossing a single die. Define X as the number of spots on the up face of the die after a toss. Assume the die is loaded so that the probability that a given face lands up is proportional to the number of spots showing.
  1. What is the probability that an even number is showing after one toss?
  2. After two tosses, what is the probability that the sum is even?
  3. After two tosses, what is the probability that the sum is odd?
10. Forty percent of assembled ink-jet printers at a plant are rejected.
  1. Find the probability that the first acceptable ink-jet printer is the third one inspected.
  2. Find the probability that the third printer inspected is the second acceptable printer.
11. A computer repair person is contacted each time there is a call for service. The number of contacts per hour occurs, on average, is 2 times/hour.
  1. What is the probability of being contacted twice in the next hour?
  2. What is the probability of being contacted two or more times in the next hour?
12. At some point in the season, assume the Pirates are currently winning 0.55 of their games and that there are 5 games in the next two weeks. What is the probability that they will win more games than they lose?
13. On average, an ocean-going vessel requires 12 hours to load, with a variance of 4 hours² . Assume that loading times are normally distributed.
  1. What is the probability that it will be loaded in less than 10 hours?
  2. What is the probability that it will take more than 14 hours to load?
Lecture 4: Continuous Probability Distributions & Pseudo Random Number Generators
1. For the Normal Distribution, the parameter σ represents:
  1. Expectation Value
  2. Probability
  3. Rate of Occurrence
  4. Standard Deviation
  5. Variance
2. For the Normal Distribution, the parameter μ represents:
  1. Expectation Value
  2. Probability
  3. Rate of Occurrence
  4. Standard Deviation
  5. Variance
3. True / False: The Exponential Distribution measures the time between Poisson events.
4. True / False: For the Exponential Distribution,the parameter λ represents the average time between events.
5. Which of the following is not true about requirements for pseudo-random number generators (select all that are not true):
  1. Numbers must be uniformly distributed between 0 and m (m is an integer upper bound parameter)
  2. Numbers must appear to be independent
  3. The period of the generator must be high
  4. Numbers must be uniformly distributed between 0 and 1
  5. Numbers must not have any correlation
  6. The density of the distribution must be high
6. A business owner wants to create a simulation for their restaurant to improve customer wait times. What distribution would best represent the time a customer must wait for a server to serve their food? Why?
7. Show that the Exponential Distribution is a memoryless distribution.
8. A shuttle that takes people between a park and a hotel does not leave the hotel until 10 people have boarded. People get on the shuttle at an average rate of 1 person each minute. What is the probability that the shuttle must wait at least 11 minutes before departing? What assumptions did you need to make to answer this question?
Lecture 5: Evaluating Uniformity and Independence & Generating Random Variates using Inverse Transform Method
1. True / False: The Kolmogorov-Smirnov Test is a test for Independence.
2. True / False: The Chi-Squared Test can test whether random numbers are uniformly distributed.
3. True / False: The Runs Test can use measures of whether sequences (runs) of numbers are above or below the observed mean.
4. True / False: The Runs Test can use measures of whether sequences (runs) of consecutive numbers are all increasing or decreasing.
5. True / False: The Inverse Transform Method for generating random variates basically starts by generating random numbers according to U[0, 1), plugs that number into the inverse Cumulative Distribution Function (cdf) for a distribution, and gets the random variate value.
6. True / False: For the Inverse Transform Method for generating random variates, you must determine the inverse function for the Probability Function (pdf or pmf). Then you generate random numbers according to U[0, 1), plug them into the inverse function, and get the random variate values out.
7. Why might you use the Kolmogorov-Smirnov Test over the Chi-Squared Test?
8. Calculate the inverse CDF for the Geometric Distribution.
Lecture 6: Generating Random Variates
1. True / False: Generating random variates according to the Continuous Uniform Distribution is best-implemented using the accept/reject technique.
2. Which special property do we make use of to generate random variates according to a homogeneous Poisson process?
3. Consider a pdf for the absolute value of the Standard Normal Distribution: f(x) = [2 / sqrt(2*π)] e^-x²/2 for x > 0. Generate random variates for this distribution using the Accept/Reject method. Bound the distribution with the Exponential Distribution with mean 1.
4. Generate three Poisson random variates with mean α = 0.2.
5. In the programming practice above, you were told to generate customer arrival times according to a uniform distribution. Now that we have covered both probability distributions and random variate generation, which distribution should you use and why? How would you generate random variates according to that distribution? What information would you need to generate the variates?
6. Which random variate generation technique is the slowest? Why?
Lecture 7: Introduction to Queueing Theory
1. Using the standard Queueing Notation: A/B/c/N/K, what does the "A" represent?
  - Inter-arrival distribution
  - Number of parallel servers
  - Service-time distribution
  - Size of calling population
  - System capacity
2. Using the standard Queueing Notation: A/B/c/N/K, what does the "N" represent?
  - Inter-arrival distribution
  - Number of parallel servers
  - Service-time distribution
  - Size of calling population
  - System capacity
3. How do probability distributions fit into Queueing Theory?
4. Identify the components of a Queueing Model in an ATM.
5. Identify the components of a Queueing Model in library.
Lecture 8: Queueing Theory - Performance, Stability, & Server Utilization
1. Does system capacity double when the number of servers doubles? Why or why not?
2. How does L^ (L with a carat above it, also called L-hat) differ from L?
3. Show the derivation for Little's Equation.
Lecture 9: Queueing Theory - Steady-State Behavior, Markov Properties, and Various Queueing Models; Introduction to Input Modeling
1. In Queueing Theory, which performance measure does L represent?
  - Long-run average number of customers in the system
  - Long-run average number of customers in the queue
  - Long-run average time spent in the system
  - Long-run average time spent in the queue
2. In Queueing Theory, which performance measure does w represent?
  - Long-run average number of customers in the system
  - Long-run average number of customers in the queue
  - Long-run average time spent in the system
  - Long-run average time spent in the queue
3. When a Queueing Model's arrival times or service times follow an Exponential distribution, it's said that the model follows a Markov Process. Why?
  - It's named after the person who discovered the Exponential distribution: Markov
  - The model’s transition from one state to another is (basically) memoryless
  - They aren’t said to follow a Markov Process
4. True or False: Input Modeling involves determining a distribution that describes the inputs to a simulation.
5. A local library has four people working at the check-out counter. On Saturdays, they have observed a sustained arrival rate of 34 library patrons borrowing books each hour that the library is open (it's open six hours on Saturdays). Due to budget cuts, the library's board of directors is thinking of reducing the number of people working at the check-out counters. If each employee takes an average of 1.5 minutes to check-out a patron, how many employees are needed to handle all of the patrons wishing to borrow books on Saturdays? What assumptions do you need to make to answer this question?
6. Years later, the library's budget increases and the board of directors decides to hire a book shelver since they have been relying on volunteers since the budget cuts. On average, three carts are ready to be reshelved every hour. Two of the volunteers are interested in the job: Ibery and Korisova. To help decide which volunteer to hire, the board asked the head librarian how quickly each one works. The head librarian has observed that Ibery can reshelve 13 carts of books during her shift (four hours long) and Korisova can reshelve 18 carts during her shift (three hours long). Ibery is asking for $9/hour and Korisova is asking for $15/hour. Which volunteer should they hire and why? What assumptions do you need to make to answer this question?
7. The library has been purchasing many books and is running out of room on the existing shelves to store all of them. It is considering replacing its physical card catalog with an online catalog to free up space for new shelves. The board of directors believes that this will be cheaper than moving into a larger space (moving would increase expenses by about 50%). The library has received a grant to develop the online catalog. It's expected that patrons will log on at around 375 per hour and remain connected for 20 minutes. How many concurrent users can they expect to have? It the library wants to have adequate capacity 99% of the time (as required by the grant), what capacity should they be prepared to handle?
8. The library is considering offering a new service to handicapped and elderly patrons: employees to find the books (requested by these patrons) in the library and bring them to the patrons. Based on information collected over the past month, the library suspects that about 15 handicapped/elderly patrons will arrive to use this service each hour. The head librarian believes that each employee involved in this service can find and return a requested book in 3 minutes on average. If the library provides three employees for this service, what is the chance of having no customers waiting for a book or waiting to place their request? What is the chance that all servers are busy? What is the time-average length of the waiting line? What is the time-average number of patrons in the system? What is the average time a patron spent waiting in the system? What assumptions do you need to make to answer these questions?

Lecture 10: Input Modeling

The Maximum Likelihood Estimate is used for ... (pick the most appropriate answer)
- determining which distribution is most likely to fit the observed data.
- estimating the parameter for a distribution given the observed data.
- maximizing the number of bins used for the Chi-Squared test.
True or False: The minimum expected frequency for each outcome/bin must be at least 3 in the Chi-Squared test.
True or False: Input modeling is used in simulations to generate realistic data for the simulation.
Use the Maximum Likelihood Estimate to derive the parameter estimation for the Binomial distribution.

During a final exam, a professor recorded the time each student turned in their exam. Below are the times, sorted by the students' last names; they're recorded as offsets from the start of the exam (e.g. the first time in the table represents 83.7 minutes after the start of the exam, the second time represents 51.3 minutes after the start of the exam). What distribution (including distribution parameters) do these turn-in times come from? Use α = 0.05.

83.7
51.3
78.9
70.1
59.4
79.1
67.9
96.3
98.2
90.1
105.7
84.1
55.6
78.2
50.4
72.3
77.0
82.9
32.1
62.3
74.4
63.6
67.1
104
38.0
87.6
103.0
68.0
80.5
65.2
92.1
63.0
92.2
88.0
105.7

Lecture 11: Input Modeling, Verification & Validation
1. Which of the following are not parts of verification (select zero or more):
  - Test structural assumptions
  - Have the operational model validated by a domain expert
  - Implement the model using well-written code
  - Use a debugger to validate that the program behaves as expected
  - Trace through the program to evaluate that the program behaves as expected
  - Evaluate data assumptions
  - Have someone other than a developer validate the operational model
2. True or False: The Chi-Squared test is used to compare generated and measured data during validation.
3. True or False: When validating model output, the hypothesized mean (μ₀) can be obtained from a domain expert.
4. Which of the following is a good example of a decision variable? Circle it and in two to five sentences, explain why (and what limits there are to it being a decision variable):
  - The rate patrons arrive at a library
  - How quickly a patron can find their desired book(s) in a library
  - How many self-checkout kiosks there are at a library
  - The service rate of a self-checkout kiosk in a library
5. How might you build a model with high face validity?
6. Describe in detail one method for testing data assumptions.
Lecture 12: Output Analysis
1. True or False: Transient behavior occurs when a simulation is terminated with a specific event.
2. True or False: When a simulation's output variable(s) of interest are no longer correlated with the initial conditions of the simulation, the simulation has reached steady state.
3. True or False: Confidence intervals must be used for output analysis.
4. True or False: It is possible to use hypothesis testing for output analysis, not just for validation.
5. True or False: When comparing alternative designs of a system and if it's possible, it is beneficial to design experiments to use correlated sampling (also called paired-t approach).
6. True or False: A disadvantage to correlated sampling (also called paired-t approach is that the output variables are correlated.
7. True or False: Your recommendation in the situation described below is good.
  A store is open every day from 10am to 10pm. The manager wonders whether patrons are waiting too long to make their purchase and decide to leave instead. You have been hired to run simulations to determine whether hiring another cashier would be beneficial (currently, there is only one cashier). After validating, implementing, and verifying your model of the store, you run a series of 10 trials with just one cashier (each run having a different set of random values). You then add a second cashier to your implementation and run another series of 10 trials (again, each run having different random values). Each trial begins when the store opens and ends when the store closes; each trial uses different random variables; and the output variable you measure is average wait time in the checkout line (assume there is just one line, regardless of the number of cashiers). You find wait times are significantly longer with only one cashier. You therefore recommend the owner hire a second cashier.
  
  Justify your True/False answer in a sentence or two.

Lecture 13: Output Analysis & Monte Carlo Simulations

True or False: It is always possible to perform correlated sampling when the same number of runs are performed for each model.
True or False: When comparing two models on an output variable, if the variances of that output variable for the two models are not equal, then it is not possible to compare the models.
True or False: Monte Carlo simulations can be used to determine an approximate value for hard calculations.

The table below shows the value of an output variable over 20 runs from two versions of a model. While both models are of the same system, the second model is far more complex than the first. Is there a significant difference between the two models? Evaluate the difference at the α = 0.05 level. Use a confidence interval.

Model 1	Model 2
11	9
12	13
13	13
10	7
3	12
11	11
12	12
12	4
8	12
7	4
11	13
3	7
7	4
11	8
13	8
11	9
5	8
11	4
4	10
7	11

Repeat the previous question, but use a hypothesis test.
In Homework 3, you explored three methods of calculating the cdf of the standard normal distribution. How might you use Monte Carlo simulations to compute the cdf? Use Monte Carlo simulations to compute the F(-1) for the standard normal distribution. What assumption(s) did you need to make? How might this method compare to the three explored in homework 3?

Lecture 14: Monte Carlo Simulations
1. Why are Bayesian networks useful?
2. Why is Rejection Sampling better than Direct Sampling?
3. Use Direct Sampling to calculate the probability Pr(Burglary=True | Earthquake=False, Mary=True). Generate at least 20 samples. Use the network below.
4. Use Rejection Sampling to calculate the probability Pr(Burglary=True | Earthquake=False, John=True). Generate at least 20 samples. Use the network from the question above.
5. Use Importance Sampling to calculate the probability Pr(Earthquake=True | Burglary=False, John=True). Generate at least 20 samples. Use the same network as above.
6. Use simulated annealing to find a short cycle through all of the cities below. For the time update function, decrease the time by 0.1 at each step.