Improving unsupervised query segmentation using parts-of-speech sequence information

Rishiraj Saha Roy, Yogarshi Vyas, Niloy Ganguly, Monojit Choudhury

Abstract: We present a generic method for augmenting unsupervised query segmentation by incorporating Parts-of-Speech (POS) sequence information to detect meaningful but rare n-grams. Our initial experiments with an existing English POS tagger employing two different POS tagsets and an unsupervised POS induction technique specifically adapted for queries show that POS information can significantly improve query segmentation performance in all these cases.