Compact and Fast Indexes for Translation Related Tasks

Jorge Costa, Luís Gomes, Gabriel Pereira Lopes, Luís M. S. Russo, Nieves R. Brisaboa

Abstract: Translation tasks, including bilingual concordancing, demand an efficient space/time trade-off, which is not always easy to get due to the usage of huge text collections and the space consuming nature of time efficient text indexes. We propose a compact representation for monotonically aligned parallel texts, based on known compressed text indexes for representing the texts and additional uncompressed structures for the alignment. The proposed framework is able to index a collection of texts in main memory, occupying less space than the text size and with efficient query response time. The proposal supports any type of alignment granularity, a novelty in concordancing applications, allowing a flexible environment for linguistics working in all phases of a translation process. We present two alternatives for self-indexing the texts, and two alternatives for supporting the alignment, comparing the alternatives in terms of space/time performance.