HOMEWORK 2 (CS 1573): Pittsburgh Food Eliza

Assigned: January 29, 2004

System: Due February 10, 2004

Evaluation: Due February 19, 2004

Project Description

The goal of this project is to use regular expressions to implement a version of Eliza tailored to talking about Pittsburgh food topics (restaurants, local specialities). As in the versions of Eliza discussed in class, your program should match a set of regular expression patterns against an input sentence, and then produce a corresponding response. The response should sometimes use subexpressions of the patterns.

Additionally, your program should maintain the conversational initiative. This means that the program deliberately places the user in the position of responding to it. If the program's response allows the initiative to be lost, it reclaims the initiative by introducing a new topic and asking a question.

Finally, your program should always try to get the user to talk about food topics of interest to people living in Pittsburgh.

System

Your program should operate in two modes. In interactive mode, your input and output should be from the screen; you should also produce a file (specified in the command line) that contains a transcript of the whole interaction when done. In batch mode, you should take input from an ascii file of user sentences (one per line).

Your program will be graded both on the range of regular expression capabilities exploited, and on the system's performance (in terms of percentage of system responses that are reasonable, and ability to hold the initiative) on a set of test utterances.

Please submit source code, a README file describing what your program does, what general classes of behavior it handles, how to run it, and a transcript highlighting your program's best features. Follow the submission procedure for the previous homework.

Also submit an Evaluation Plan, containing 1) a hypothesis that you will test during the Evaluation phase described below, and 2) a new performance measure for evaluating Pittsburgh Food Eliza agents.

Evaluation

Given that we will have 14 Pittsburgh Food Eliza agents, and that we are empirically inclined in this class, we will use these systems to generate some data for an experiment.

Everyone will be assigned two Pittsburgh Food Eliza agents (not your own) to converse with. You will talk to these two agents in a number of conservations, to get some data. You will analyze this data with respect to your proposed performance measures, to compare your agents and to test your hypothesis. Discuss limitations of your approach. Try to make Eliza respond inappropriately at least once. (The utterance should be a reasonable one within the context of a Pittsburgh food chat, however.) Explain how/why Eliza might be responding inappropriately.