Paper: Stating with Certainty or Stating with Doubt: Intercoder Reliability Results for Manual Annotation of Epistemically Modalized Statements

ACL ID N07-2036
Title Stating with Certainty or Stating with Doubt: Intercoder Reliability Results for Manual Annotation of Epistemically Modalized Statements
Venue Human Language Technologies
Session Short Paper
Year 2007
Authors

Texts exhibit subtle yet identifiable mo- dality about writers’ estimation of how true each statement is (e.g. , definitely true or somewhat true). This study is an analy- sis of such explicit certainty and doubt markers in epistemically modalized statements for a written news discourse. The study systematically accounts for five levels of writer’s certainty (ABSOLUTE, HIGH, MODERATE, LOW CERTAINTY and UNCERTAINTY) in three news pragmatic contexts: perspective, focus, and time. The study concludes that independent coders’ perceptions of the boundaries be- tween shades of certainty in epistemically modalized statements are highly subjec- tive and present difficulties for manual annotation and consequent automation for opinion extraction and sentiment analysis. While stricter annotatio...