Tight Lower Bounds for the Distinct Elements Problem

Piotr Indyk, David P. Woodruff

Abstract: We prove strong lower bounds for the space complexity of (/spl epsi/, /spl delta/)-approximating the number of distinct elements F/sub 0/ in a data stream. Let m be the size of the universe from which the stream elements are drawn. We show that any one-pass streaming algorithm for (/spl epsi/, /spl delta/)-approximating F/sub 0/ must use /spl Omega/(1//spl epsi//sup 2/) space when /spl epsi/ = /spl Omega/(m/sup -1/(9 + k)/), for any k > 0, improving upon the known lower bound of /spl Omega/(1//spl epsi/) for this range of /spl epsi/. This lower bound is tight up to a factor of log log m for small /spl epsi/ and log 1//spl epsi/ for large /spl epsi/. Our lower bound is derived from a reduction from the one-way communication complexity of approximating a Boolean function in Euclidean space. The reduction makes use of a low-distortion embedding from an l/sub 2/ to l/sub 1/ norm.