With a few exceptions, extensions to latent Dirichlet allocation (LDA) have focused on the distribution over topics for each document. Much less attention has been given to the un- derlying structure of the topics themselves. As a result, most topic models generate topics in- dependently from a single underlying distri- bution and require millions of parameters, in the form of multinomial distributions over the vocabulary. In this paper, we introduce the Shared Components Topic Model (SCTM), in which each topic is a normalized product of a smaller number of underlying component dis- tributions. Our model learns these component distributions and the structure of how to com- bine subsets of them into topics. The SCTM can represent topics in a much more compact representation than LDA and ach...

Click Here to View the Paper In XML Format