Boolean-aware Attention for Dense Retrieval

ACL ARR 2025 February Submission4429 Authors

Abstract: We present Boolean-aware attention, a novel attention mechanism that dynamically adjusts token focus based on Boolean operators (e.g., "and, "not"). Our model employs specialized Boolean experts, each tailored to amplify or suppress attention for operator-specific contexts. A predefined gating mechanism activates the corresponding experts based on the detected Boolean type. Experiments on two Boolean retrieval datasets demonstrate that integrating BoolAttn with BERT greatly enhances the model's capability to process Boolean queries.

Paper Type: Short

Research Area: Information Retrieval and Text Mining

Research Area Keywords: dense retrieval

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Publicly available software and/or pre-trained models

Languages Studied: English

Submission Number: 4429