The following are various announcements and pieces of advice to do with Assignment 2. The newest pieces of advice will go at the top of this file. Please check if your question is answered here before sending me mail.
For question 2b -- when you try out your grammar on the Mother Goose sentences -- it may be that it doesn't parse any of them at all, perhaps because you've given the words in your lexicon different categories to those in the Mother Goose lexicon. This is not a problem; whatever happens, you should just comment on how well your grammar does.
IF YOU WANT TO, you can try renaming the categories in the lexicon to see whether the sentences will parse. This is NOT REQUIRED, but if you're curious feel free to try this out.
generate (Wednesday 7:00pm)
I have just installed a new, improved version of generate
that should no longer produce "segmentation fault" errors. For people who
have complex grammars with a lot of rules, you may still find that
generate will produce VERY long sentences, often with
non-terminals still in them. If you want to try to tame it, you can try
putting the non-recursive rules before the recursive rules in your
grammar; for any non-terminal, generate will now prefer to
choose rules for that non-terminal that come earlier in the grammar.
However, if your grammar is just too big to move rules around like this,
or if moving doesn't seem to help with the massively long sentences, you
can do the following instead of using generate: give five
examples of sentences that your grammar and lexicon will allow that you
find ungrammatical in English. (If your grammar only allows sentences that
are grammatical English, then you can say so and impress us all!)
I've updated the "parse" program on all computer systems -- hopefully it will now no longer give you "too many parses" or "too many nodes" errors.
I've also edited the base sentences to take out a couple of PPs from the "West Ham" sentence, and to get rid of the subject-dropping in the "Williams picked up the injury" sentence".
I've *also* (busy, busy me!) added a link from the assignment page to an online English grammar site, for those who want a bit more information about grammar than is given in the textbook.
If you get "segmentation fault (core dumped)" from the "generate" program -- I'm debugging that one right now, so hang tight. :-)
By popular demand :-), we have provided a set of "base" sentences for question 2. You should make sure that your grammar can, as a minimum, parse the (19) sentences in this file. If your grammar is more powerful than this, you will be eligible for bonus marks on this question.
Many of you were having difficulty with writing the grammar for question 2, since the sentences in the texts we gave you were so long and complicated. So, there is now a new input file for the parser. This file contains all of the sentences of the previous files, but the complexity of the sentences has been reduced so that (a) it will hopefully be easier to write a grammar to parse more of them, and (b) the size and number of parse trees should no longer cause the parser program to die.
Since the sentences in the new file are just simplified versions of the sentences in the original files, those of you who have already written grammars to deal with the original files should be able to use what you have already written, and the task of writing a grammar for the remainder of the sentences should be easier.
Don't forget, you don't have to be able to parse all of the sentences in the input! In fact, we expect that there will be sentences your grammar does not parse, or that some of the sentences will be assigned multiple parses.
generate?
Some people may have found that generate would stop saying
"too many symbols" if they asked it to generate from their grammar. I
have installed a new version of generate that no longer has these
problems. However, you may still find that it produces sentences with
nonterminals in them (such as NP or S) instead of words. This happens if
the sentence is going to be too long (longer than 200 words). It may not
be possible to change your grammar to stop it trying to generate such
long sentences; if not, don't worry about it.
If your grammar overgenerates -- that is, accepts sentences that are not actually grammatical -- then you should try to reduce this as much as possible. I don't expect a perfect grammar, and you shouldn't knock yourself out trying to do too much... try to get a grammar that will parse most of the sentences in the input and that doesn't do too many stupid things.
You are free to modify the lexicon as much as you want, as well -- for example, adding new categories or moving words between categories.
The parser we're using can only parse simple context-free rules with no features, so you'll have to add just "plain vanilla" context-free rules. Hopefully this will make you realise just how useful features can be!
Just FYI, the value of Kappa that is computed here is a simpler version of Kappa than the one that people often use these days. If you're interested, I can give you references about the other Kappa (which can also be used to compare more than two classifiers). For this assignment, don't worry about this issue; you can just use my program to evaluate the taggers.
You will get warnings about "different number of words on lines" for a couple of the lines of output from the HMM tagger. This is because the tagger tokenizes slightly differently than the Treebank. Since it's just a couple of words, it probably won't have a big effect on the Kappa value, but you can comment in your answers on what effect you think this might have.
Make sure you follow the instructions very carefully -- and make sure
that you are following the correct instructions for the computer system
that you are using! In particular, students in DCS should log into a
Solaris machine such as canna (type ssh canna) before
running either of the taggers or the parser. Also, if the Brill tagger is
giving you messages like
"sh: start-state-tagger: not found",
you should also follow my advice about setting your PATH.
If you still can't get it running, let me know and I'll look into it -- it could be that something still isn't quite configured right on my end.