EUROLAN 2007 Monday, July 30 Working Session ================================ Exercise 1: In this exercise, we'll work with the MPQA subjectivity annotation scheme. Your task will be to find the direct-subjective, objective-speech-event, and expressive-subjective-element expressions and their nested sources. Below, the "polarity" annotations are the annotations used in our work in contextual polarity [Wilson, Wiebe, Hoffmann EMNLP 2005]. The "attitude-type", "attitude-link", "target", and "target-link" are new annotations that will be in the next release of the corpus. They are part of Theresa Wilson's dissertation work. ==Sentence 1 [docid=non_fbis/08.40.56-18707 span=599,778] The State Department's annual human rights report, which was published Monday, said Russia had a poor record regarding the independence and freedom of the media and in Chechnya. GATE_objective-speech-event (599, 599) nested-source=w implicit=true [] The entire sentence is an objective speech event attributed to the writer. It is simply presented as true (without evaluation) by the writer, that the report said Russia... GATE_direct-subjective (679, 683) intensity=medium attitude-link=a520 expression-intensity=neutral nested-source=w,report ['say'] The "say" event is direct-subjective. The source is writer,report (according to the writer, according to the report, Russia had a poor record). It is subjective because, in the context of this article, it is only the report's opinion that Russia's human rights record is poor. [This is from a Russian newspaper that is critical of the US report] Also, this use of "poor" is evaluative. [There is no polarity field in this annotation because the expression-intensity is neutral. You'll find the negative polarity annotation on the annotation for "poor" below.] GATE_expressive-subjectivity (697, 701) nested-source=w,report polarity=negative intensity=medium ['poor'] "poor" is an expressive subjective element attributed to writer,report. Here are additional annotations: GATE_attitude (691, 777) intensity=medium id=a520 attitude-type=sentiment-neg target-link=t6 ['have', 'a', 'poor', 'record', 'regard', 'the', 'independence', 'and', 'freedom', 'of', 'the', 'media', 'and', 'in', 'chechnya'] GATE_target (684, 690) id=t6 ['russia'] These annotations are for the negative attitude about Russia that is attributed to the report by the writer. Note that it is linked via the id a520 to the direct-subjective annotation that is anchored to "said". And, the target is linked to it via the id t6. ===Sentence 2 docid=non_fbis/08.40.56-18707 span=270,595 "One gets the impression that its writers simply used old drafts, as if nothing had happened in either Russia or the United States in recent years, as if the events of Sept. 11, 2001 had not occurred and the international community had not closed ranks in the battle against terrorism," the ministry said in a statement. GATE_objective-speech-event (270, 270) nested-source=w implicit=true [] The entire sentence is an objective speech event attributed to the writer. GATE_expressive-subjectivity (271, 294) nested-source=w, ministry polarity=neutral intensity=medium ['one', 'get', 'the', 'impression'] Note that the polarity of this expression is neutral. Other expressive-subjectivity annotations capture the negative evaluation. GATE_expressive-subjectivity (312, 335) nested-source=w, ministry polarity=negative intensity=medium ['simply', 'use', 'old', 'draft'] GATE_expressive-subjectivity (337, 342) nested-source=w, ministry polarity=neutral intensity=low ['as', 'if'] GATE_expressive-subjectivity (420, 425) nested-source=w, ministry polarity=neutral intensity=low ['as', 'if'] This is a "spare" annotation of the "as if" expressive subjective element annotations. One could argue that the first should be 'as if nothing had happened" with a higher strength than low (and similarly for the second "as if"). GATE_direct-subjective (574, 594) intensity=medium attitude-link=a140 expression-intensity=neutral nested-source=w, ministry ['say', 'in', 'a', 'statement'] Here are additional annotations: GATE_attitude (271, 558) intensity=high repetition=yes id=a140 attitude-type=sentiment-neg target-link=t5 ['one', 'get', 'the', 'impression', 'that', 'its', 'writer', 'simply', 'use', 'old', 'draft', 'as', 'if', 'nothing', 'have', 'happen', 'in', 'either', 'russia', 'or', 'the', 'unite', 'state', 'in', 'recent', 'year', 'as', 'if', 'the', 'event', 'of', 'sept', 'have', 'not', 'occur', 'and', 'the', 'international', 'community', 'have', 'not', 'close', 'rank', 'in', 'the', 'battle', 'against', 'terrorism'] GATE_target (300, 311) id=t5 ['its', 'writer'] The text spans/anchors for the attitude and target annotations are: the portion of the sentence that the reader needs in order to perceive the attitude or target. So, we see here that the negative attitude is linked to the direct subjective annotation on ['say', 'in', 'a', 'statement'], and thus inherits the nested-source=w, ministry The attitude is negative toward the writer. This illustrates that the target of a direct-subjective annotation is often not the same as the target of an attitude tied to that annotation. The target of the direct-subjective annotation is what was said. But the target of the negative attitude is the writer of the report. The repetition=yes feature was introduced to help with future work. Intensity is often increased through repetition, but it isn't clear how this type of repetition can be recognized computationally. So, we added it as a feature. ===Sentence 3 docid=non_fbis/08.40.56-18707 span=50,266 The Foreign Ministry said Thursday that it was "surprised, to put it mildly" by the U.S. State Department's criticism of Russia's human rights record and objected in particular to the "odious" section on Chechnya. GATE_objective-speech-event (50, 50) nested-source=w implicit=true [] There is no subjectivity at the level of the writer. So, the top-level annotation is objective-speech event. GATE_direct-subjective (71, 75) nested-source=w,ministry expression-intensity=neutral attitude-link=a110 intensity=medium ['say'] GATE_direct-subjective (93, 126) polarity=negative intensity=high attitude-link=a110 expression-intensity=high nested-source=w,ministry,ministry ['be', 'surprise', 'to', 'put', 'it', 'mildly'] It's important that 'to put it mildly' is part of this text anchor, because it is an intensifying phrase --- the private state referred to is a high intensity "surprise" private state. As you'll see below, "to put it mildly" is also an expressive subjective element in its own right. Ask yourself now, what is the nested-source of "to put it mildly" as an expressive subjective element? GATE_attitude (93, 127) intensity=high id=a110 attitude-type=sentiment-neg target-link=t2 ['be', 'surprise', 'to', 'put', 'it', 'mildly'] GATE_target (131, 201) id=t2 ['the', 'u', 's', 'state', 'department', "'s", 'criticism', 'of', 'russia', "'s", 'human', 'right', 'record'] Interestingly, the negative attitude above is linked to two direct-subjective annotations: "say" with source (w,ministry) and "was surprised, to put it mildly" with source (w,ministry,ministry). The "say" direct-subjective annotation refers to a speech event in which what is said expresses a negative sentiment toward the US State Department's criticism. The "to put it mildly" expressive subjective element also expresses a negative sentiment toward the critcism. An interesting direction for future work would be to investigate targets in the context of "semantic entailment", a recent paradigm being pursued in Natural Language Processing. An interesting task would be to recognize cases where subjectivity toward an event entails an attitude toward the agent of that event. For example, we would like to recognize that a negative sentiment toward someone's criticism entails (at least plausibly) a negative sentiment toward that person him/herself. Another would be to recognize situations where referring to someone's private state may entail a negative sentiment toward that person: E.g., He is only interested in himself" entails a negative judgement of "He". GATE_expressive-subjectivity (109, 126) nested-source=w,ministry polarity=negative intensity=medium ['to', 'put', 'it', 'mildly'] GATE_direct-subjective (159, 168) polarity=negative intensity=medium attitude-link=a120 expression-intensity=medium nested-source=w,ministry,ministry,ussd ['criticism'] GATE_attitude (159, 168) intensity=medium id=a120 attitude-type=sentiment-neg target-link=t3 ['criticism'] GATE_target (173, 201) id=t3 ['russia', "'s", 'human', 'right', 'record'] GATE_agent (173, 179) id=russia ['russia'] GATE_direct-subjective (206, 214) polarity=negative intensity=high attitude-link=a130 expression-intensity=medium nested-source=w,ministry attitude-toward=ussd ['object'] GATE_attitude (206, 245) intensity=high id=a130 attitude-type=sentiment-neg target-link=t4 ['object', 'in', 'particular', 'to', 'the', 'odious'] GATE_target (246, 265) id=t4 ['section', 'on', 'chechnya'] GATE_expressive-subjectivity (238, 244) nested-source=w,ministry polarity=negative intensity=high ['odious'] ==Sentence 4 docid=temp_fbis/21.37.46-9337 span=1215,1317 The United States has not only made a procedural mistake, but the facts are incorrect, officials said. GATE_objective-speech-event (1215, 1215) nested-source=w implicit=true [] GATE_direct-subjective (1312, 1316) intensity=medium attitude-link=a210,a211,a212 expression-intensity=neutral nested-source=w,officials attitude-toward=US ['say'] GATE_attitude (1233, 1300) intensity=medium id=a210 attitude-type=sentiment-neg target-link=t15 ['have', 'not', 'only', 'make', 'a', 'procedural', 'mistake', 'but', 'the', 'fact', 'be', 'incorrect'] GATE_attitude (1233, 1271) intensity=medium id=a211 attitude-type=arguing-pos target-link=t15 ['have', 'not', 'only', 'make', 'a', 'procedural', 'mistake'] GATE_attitude (1273, 1300) intensity=low-medium id=a212 attitude-type=arguing-neg target-link=t99 ['but', 'the', 'fact', 'be', 'incorrect'] GATE_target (1215, 1232) id=t15 ['the', 'unite', 'state'] GATE_target (1277, 1286) id=t99 ['the', 'fact'] The annotator attached three attitudes to 'say': a negative sentiment of the officials toward the United States, a positive arguing about the United States (the officials are arguing for the proposition that the United States made a procedural mistake), and a negative arguing about the facts (the officials are arguing that the facts are not true). There's different options for what one might consider the target of attitudes, especially arguings. In the above annotations, what are marked are the topics of the arguing (the officials are arguing that something about the United States is true). Another option would be to identify the entire proposition as the target. GATE_expressive-subjectivity (1281, 1300) nested-source=w, officials polarity=negative intensity=medium ['fact', 'be', 'incorrect'] GATE_expressive-subjectivity (1237, 1245) nested-source=w,officials polarity=neutral intensity=low ['not', 'only'] Exercise 2: Now we turn to annotating word senses for subjectivity. Part 1: noun alarm 1 S Neg noun alarm 2 O noun alarm 3 O noun alarm 4 O noun assault 1 O noun assault 2 O noun assault 3 O noun assault 4 O There is a missing word sense, for verbal assault verb assault 1 B Neg The hypernym is "physical or emotional attack", so the subjective and objective usages are not teased apart verb assault 2 O verb assault 3 S Here is the subjective sense that is missing from "noun assault" noun blow 1 O noun blow 2 O noun blow 3 S Neg noun blow 4 S Neg noun blow 5 O noun blow 6 O We decided that this is an objective sense, assuming that this is just a name for drugs, not an evaluation of them. (But perhaps we are wrong.) noun blow 7 O noun adoration 1 S noun adoration 2 S noun adoration 3 S noun bomb 1 O noun bomb 2 O noun bomb 3 S Neg noun catch 1 S Neg noun catch 2 O noun catch 3 S Pos noun catch 4 O noun catch 5 S Both? noun catch 6 O noun catch 7 O noun catch 8 O noun catch 9 O noun catch 10 O "Catch" is interesting because it is only one of a couple words in our annotated word senses that has positive and negative subjective senses. While we found a number of words in our noun atmosphere 1 S Both One can have an atmosphere of excitement, but also an atmosphere of dread. noun atmosphere 2 O noun atmosphere 3 O noun atmosphere 4 O noun atmosphere 5 O noun atmosphere 6 S Both The examples include both positive and negative usages Part 2: noun alarm 1 S Neg noun alarm 2 O noun alarm 3 O noun alarm 4 O Though alarms are associated with negative events, it is easy to use the objective senses in a neutral way. In contrast, the subjective sense is intrinsically negative. noun assault 1 O Interestingly, one of the examples given is positive: "Grant won a desisive vitory..." But then, that sentence includes a member of the synset, not the word "assault" itself. Can "assualt" with this meaning be used neutrally or positively? It's not clear. noun assault 2 O Assault and battery are legal terms, so can be used neutrally ("He was charged with assault and battery, but the case was dismissed"). noun assault 3 O noun assault 4 O Again, this can be used neutrally as a legal term. verb assault 1 B Neg The subjective usages are negative. The objective uses may be negative or neutral. verb assault 2 O verb assault 3 S noun blow 1 O noun blow 2 O noun blow 3 S Neg noun blow 4 S Neg noun blow 5 O noun blow 6 O noun blow 7 O "Blow" is similar to "assualt": objective blows are often associated with negative polarity, but may be used neutrally. Where the subjective meanings of blow are intrinsically negative. noun adoration 1 S noun adoration 2 S noun adoration 3 S noun bomb 1 O Unfortuately, bombs can be good as as well as bad, depending on your perspective. noun bomb 2 O This sense seems intrinsically objective. noun bomb 3 S Neg This sense is intrinsically subjective. Of course, we are talking here about "prior polarity"; the polarity we would expect not considering any context. Context can change polarity, of course, as in "In all his years of standup comedy, he never once bombed during a show". noun catch 1 S Neg noun catch 2 O noun catch 3 S Pos noun catch 4 O I considered that this sense might be intrinsically positive, but "His catch that day was meager" illustrates that it is not. noun catch 5 S Both? noun catch 6 O noun catch 7 O noun catch 8 O noun catch 9 O noun catch 10 O This can be positive, from the officer's point of view; negative, from the criminal's point of view; neutral, from a legal point of view. noun atmosphere 1 S Both noun atmosphere 2 O noun atmosphere 3 O noun atmosphere 4 O noun atmosphere 5 O noun atmosphere 6 S Both The objective senses of 'atmosphere' seem intrinsically objective. Part 3: Subjectivity and especially polarity tend to be lost at higher levels of hypernyms. "good" has ancestor "quality", for example. "love"'s direct parent is "emotion", so the polarity is lost in only one level. It seems that subjectivity and polarity are clearest for "basic-level categories" and "subordinate-level categories". For example, "chair" is a basic-level category, furniture is a superordinate category, and "lounge chair" is a subordinate-level category: you can form an image of a chair, but not of furniture, etc. Here is more information about basic-level categories: http://www.wisegeek.com/what-is-a-basic-level-category.htm Part 4: We want to hear from you!