Paper: Factored Language Models And Generalized Parallel Backoff

ACL ID N03-2002
Title Factored Language Models And Generalized Parallel Backoff
Venue Human Language Technologies
Session Short Paper
Year 2003

We introduce factored language models (FLMs) and generalized parallel backoff (GPB). An FLM represents words as bundles of features (e.g. , morphological classes, stems, data-driven clusters, etc.), and induces a prob- ability model covering sequences of bundles rather than just words. GPB extends standard backoff to general conditional probability tables where variables might be heterogeneous types, where no obvious natural (temporal) backoff order exists, and where multiple dynamic backoff strategies are allowed. These methodologies were implemented during the JHU 2002 workshop as extensions to the SRI language modeling toolkit. This paper provides initial perplexity results on both CallHome Arabic and on Penn Treebank Wall Street Journal articles. Significantly, FLMs with GPB can produc...