Context-aware resemblance detection for data deduplication with neural network

Open Webpage

Xuming Ye, Wenlong Tian, Yaping Wan, Ruixuan Li, Weijun Xiao, Zhiyong Xu

Published: 2025, Last Modified: 17 Jul 2025Eng. Appl. Artif. Intell. 2025EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: Highlights•Traditional schemes overlook chunk-context, limiting effectiveness in resemblance detection.•Proposing a chunk-context aware approach to improve the accuracy and efficiency in resemblance detection.•N-sub-chunk shingles and BP-neural network enabling qualified chunk representation.•Achieving up to 75.03% more data removal and 5.6x–86.7x faster detection than state-of-the-art work.