Homework 2 (CS 1671)

Assigned: October 2, 2018

Due: October 16, 2018 (midnight)

2.1 HMM Decoding (Viterbi) (20 points)

A partial Viterbi calculation is pictured here. This calculation takes us up through t=2 where v2(1) and v2(2) are computed. In the picture, the index 1 is used for the state labeled C and the index 2 is used for the state labeled H. Compute v3(1) and v3(2). You will need the transition and observation probabilities given here.

Think of this as filling in a table where the columns are moments in time and the rows are states in the HMM. Filling in the table with the numbers computed in the diagram above, and adding a column for time t = 0, and showing all the probability cells, it looks like this:

Each cell in the Viterbi table is filled with one of the Viterbi values computed in the diagram. Like the diagram, the table is complete through t=2. The values in the cells represent Viterbi probabilities. The Viterbi probability written as v2(2) repesents the probability of the highest probability path that ends at state 2 at time 2.

• (10 points) Submit a completed version of the table above, together with the calculations you used to compute the Viterbi probabilities v3(1) and v3(2).
  • The calculations should show the products producing the path probabilities and the maximization that gives the final Viterbi value.
  • In addition, show how you would do all calculations in log (ln) space as well as directly as products of probabilities (recall Chapter 3).

• (10 points)
  • Report the best path through the HMM that fits the data.
  • Justify your answer by adding backtraces to your table, including the backtraces for column 3. This figure illustrates the idea. The dashed lines represent the best path associated with each Viterbi value. In your submission you can just explain textually how you would modify the figure, e.g., "Add a backtrace link (dashed line) to the backtrace figure going from STATE? at time t = 3 to STATE? at time t = 2."

2.2    CKY Parsing (60 Points)

Implement a non-probabilistic CKY parser.

• (40 points) Demonstrate the correctness of your implementation by running it with a NON-probabilistic version of the grammar below and the inputs "The flight includes a meal" and "Book the flight through Houston"

  CLARIFICATIONS: Your program will need to convert your grammar to CNF if needed (as with the grammar below). You can't do this manually due to the blind testing of your program.

  Here is an example grammar file (for the grammar below) that your program should be able to process. You can asssume that each word will have a separate rule (so you don't need to process disjunctions).

  You can also assume that terminals begin with lowercase and non-terminals with uppercase, as in the example grammar.

• (20 points) We will also test your implementation on other blind tests.

  CLARIFICATION: We will run your code on both new grammars and new test sentences.

0.80 S -> NP VP
0.15 S -> Aux NP VP
0.05 S -> VP
0.35 NP -> Pronoun
0.30 NP -> Proper-Noun
0.20 NP -> Det Nominal
0.15 NP -> Nominal
0.75 Nominal -> Noun
0.20 Nominal -> Nominal Noun
0.05 Nominal -> Nominal PP
0.35 VP -> Verb
0.20 VP -> Verb NP
0.10 VP -> Verb NP PP
0.15 VP -> Verb PP
0.05 VP -> Verb NP NP
0.15 VP -> VP PP
1.0 PP -> Preposition NP
Det -> that [0.10] | a [0.30] | the [0.60]
Noun -> book [0.10] | flight [0.30] | meal [0.15] | money [0.05] | flights [0.40] | dinner [0.10]
Verb -> book [0.30] | includes [0.30] | prefer [0.40]
Pronoun -> i [0.40] | she [0.05] | me [0.15] | you [0.40]
Proper-Noun -> houston [0.60] | twa [0.40]
Aux -> does [0.60] | can [0.40]
Preposition -> from [0.30] | to [0.30] | on [0.20] | near [0.15] | through [0.05]

Input/Output Requirements

Your script (for Python users) or executable jar (for Java users) must take two parameters:

• The grammar file
• The sentence to parse (which will be surrounded by double quotes)

If you use Python, your code will be tested as:

python cky.py cfg.txt "A test sentence"

If you use Java, your code will be tested as:

java -cp yourname.jar cs1671.hw2.CKY cfg.txt "A test sentence"

The output should be printed to the standard output stream. Print all of the parse trees for the sentence in the following bracket-based format:

[S [NP [Pronoun I]] [VP [Verb book] [NP [Det a] [Nominal [Noun flight]]] [PP [Preposition to] [NP [Proper-Noun houston]]]]]

(Copy and paste this string into mshang.ca/syntree to visualize it. You will find this tool very useful throughout this homework. )

What to Include in Submission?

For Python users, include:

• The python source file: cky.py.
• A readme.txt which includes:
  • Python version (2 or 3)
  • Any known issues that prevent your script from running.

For Java users, include:

• A yourname.zip archive which includes the Java source.
• A yourname.jar which is compiled from your source code. The jar should have a main class named cs1671.hw2.CKY.
• A readme.txt which includes:
  • Any known issues that prevent your code from running.

2.3    Probabilistic Parsing (20 Points)

The probabilistic grammar provided has rules such as VP -> Verb NP PP, which has more than two non-terminals on the right hand side.

• (10 points)
  • Explain how you would modify the algorithm for conversion to CNF to correctly handle rule probabilities.
  • Then show the binarized version of the grammar, with the probabilities. You should be very careful that the binarization should be done in a way that the probability stays the same for equivalent rules before and after binarization. That is, the CNF should assign the same total probability to each parse tree as the original grammar.

• (10 points) Consider the following PCFG:

  
  S -> NP VP 1.0
  PP -> P NP 1.0 
  VP -> V NP 0.7
  VP -> VP PP 0.3 
  P -> with 1.0 
  V -> saw 1.0 
  NP -> NP PP 0.4
  NP -> scientists 0.1
  NP -> chins 0.18
  NP -> saw 0.04
  NP -> moons 0.18
  NP -> telescopes 0.1
  

  • What is the probability of the sentence "scientists saw moons with chins"? Show not only the number but its computation.
  • If the sentence is ambiguous, also show the most likely parse using the bracket-based format.