Paper: Evaluation And Extension Of Maximum Entropy Models With Inequality Constraints

ACL ID W03-1018
Title Evaluation And Extension Of Maximum Entropy Models With Inequality Constraints
Venue Conference on Empirical Methods in Natural Language Processing
Session Main Conference
Year 2003
Authors
  • Jun'ichi Kazama (University of Tokyo, Tokyo Japan)
  • Jun'ichi Tsujii (University of Tokyo, Tokyo Japan; CREST Japan Science and Technology Corporation, Saitama Japan)

A maximum entropy (ME) model is usu- ally estimated so that it conforms to equal- ity constraints on feature expectations. However, the equality constraint is inap- propriate for sparse and therefore unre- liable features. This study explores an ME model with box-type inequality con- straints, where the equality can be vio- lated to reflect this unreliability. We eval- uate the inequality ME model using text categorization datasets. We also propose an extension of the inequality ME model, which results in a natural integration with the Gaussian MAP estimation. Experi- mental results demonstrate the advantage of the inequality models and the proposed extension.