Better entity matching with transformers through ensembles

Jwen Fai Low, Benjamin C. M. Fung, Pulei Xiong

Abstract: Highlights•Ensembles can be constructed with just one transformer architecture.•Single-architecture ensembles save on memory, making LLM ensembles viable.•Non-stochastic shuffling of LLM input tokens is sufficiently unique for ensembles.•More aggregator tokens equals more information preserved equals better EM performance.•Extra self-attention keeps model focused only on the informative aggregator tokens.