Dynamic Confidence-aware Truth Discovery on Unevenly Distributed Data Streams

Xiu Fang, Hao Chen, Xinyang Du, Ziqi Wei, Yang Zheng, Guohao Sun

Abstract: To resolve conflicts in multi-source data, truth discovery has been widely studied. However, existing methods ignore the uneven distribution of time-series data that evolves. The coverage of objects by sources may exhibit a long-tail phenomenon. Additionally, the distribution of objects across sources is often uneven, with some hot objects being claimed by multiple sources, while only a few sources claim other cold objects. Due to insufficient data support, the reliability evaluation of small sources and the truth estimation of cold objects are often unreasonable. Furthermore, unpredictably, the roles of big and small sources may be interchanged over time, and objects classified as hot and cold may switch places. Dynamic changes in data distribution further increase the complexity of estimation. To tackle this problem, we propose a novel dynamic truth discovery method(TDCITP). Based on data fluctuation, TDCITP adaptively chooses the stability method or the fluctuation method for truth estimation and source weight estimation. Confidence interval estimation is conducted for both source weight and truth estimation to distinguish small sources and cold objects. Experimental results demonstrate that our method outperforms state-of-the-art methods in terms of accuracy and efficiency.