Conference

Improving effectiveness of query expansion using information theoretic approach

Automatic query expansion is a widely used method to enhance the performance of information retrieval systems. This paper proposes information-theoretic measures to improve the efficiency of co-occurrence-based automatic query expansion using a pseudo-relevance feedback approach. Expansion terms are selected from the top N retrieved documents and ranked using two information-theoretic methods: Kullback-Leibler divergence (KLD) and a variant of KLD. Experiments conducted on the TREC-1 dataset demonstrate that incorporating these measures can further enhance query expansion effectiveness. Additionally, extensive testing was performed to optimize key parameters, including the number of top N documents used for expansion.