Paper: On Combining Language Models: Oracle Approach

ACL ID H01-1058
Title On Combining Language Models: Oracle Approach
Venue Human Language Technologies
Session Main Conference
Year 2001

In this paper, we address the problem of combining several lan- guage models (LMs). We find that simple interpolation methods, like log-linear and linear interpolation, improve the performance but fall short of the performance of an oracle. The oracle knows the reference word string and selects the word string with the best per- formance (typically, word or semantic error rate) from a list of word strings, where each word string has been obtained by using a dif- ferent LM. Actually, the oracle acts like a dynamic combiner with hard decisions using the reference. We provide experimental results that clearly show the need for a dynamic language model combina- tion to improve the performance further. We suggest a method that mimics the behavior of the oracle using a neural network or a de- ci...