%\documentclass{uai2024} % for initial submission
\documentclass[accepted]{uai2024} % after acceptance, for a revised version; 
% also before submission to see how the non-anonymous paper would look like 
                        
%% There is a class option to choose the math font
% \documentclass[mathfont=ptmx]{uai2024} % ptmx math instead of Computer
                                         % Modern (has noticeable issues)
% \documentclass[mathfont=newtx]{uai2024} % newtx fonts (improves upon
                                          % ptmx; less tested, no support)
% NOTE: Only keep *one* line above as appropriate, as it will be replaced
%       automatically for papers to be published. Do not make any other
%       change above this note for an accepted version.

%% Choose your variant of English; be consistent
\usepackage[american]{babel}
\usepackage{amsthm}
\usepackage{amssymb}
\usepackage{comment}


\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}
\newtheorem{lemma}{Lemma}


\theoremstyle{definition}  
\newtheorem{example}{Example}
\newtheorem{definition}{Definition}

\usepackage{graphicx} % Required for inserting images
\usepackage{subfig}
\usepackage{algpseudocode}
\usepackage[ruled,vlined,linesnumbered]{algorithm2e}
\usepackage{breqn}
%\usepackage[noend]{algpseudocode}
%\usepackage{amssymb}
\usepackage{amsmath}

% \usepackage[british]{babel}

%% Some suggested packages, as needed:
\usepackage{natbib} % has a nice set of citation styles and commands
    \bibliographystyle{plainnat}
    \renewcommand{\bibsection}{\subsubsection*{References}}
\usepackage{mathtools} % amsmath with fixes and additions
% \usepackage{siunitx} % for proper typesetting of numbers and units
\usepackage{booktabs} % commands to create good-looking tables
\usepackage{tikz} % nice language for creating drawings and diagrams

%% Provided macros
% \smaller: Because the class footnote size is essentially LaTeX's \small,
%           redefining \footnotesize, we provide the original \footnotesize
%           using this macro.
%           (Use only sparingly, e.g., in drawings, as it is quite small.)

%% Self-defined macros
\newcommand{\swap}[3][-]{#3#1#2} % just an example

\title{Publishing Number of Walks and Katz Centrality under \\ Local Differential Privacy}

% The standard author block has changed for UAI 2024 to provide
% more space for long author lists and allow for complex affiliations
%
% All author information is authomatically removed by the class for the
% anonymous submission version of your paper, so you can already add your
% information below.
%
% Add authors
\author[1]{Louis Betzer}
\author[2]{\href{mailto:<vorapong@is.s.u-tokyo.ac.jp>?Subject=Your UAI 2024 paper}{Vorapong Suppakitpaisarn}{}}
\author[2]{Quentin Hillebrand}
% Add affiliations after the authors
\affil[1]{%
    Ecole Polytechnique\\
    Palaiseau, France
}
\affil[2]{%
    The University of Tokyo\\
    Tokyo, Japan
}
  \begin{document}
\maketitle

\begin{abstract}
  In our study, we present an algorithm for publishing the count of walks and Katz centrality under local differential privacy (LDP), complemented by a comprehensive theoretical analysis. While previous research in LDP has predominantly focused on counting subgraphs with a maximum of five nodes, our work extends this to larger subgraphs. The primary challenge in such an extension lies in managing the exponentially increasing noise associated with LDP as the size of the subgraph grows. Our solution involves an algorithm for publishing the count of walks originating from each node in the graph, which subsequently enables us to publish the Katz centrality of all nodes. This algorithm incorporates multiple communication rounds and employs a clipping technique. Through both theoretical and empirical evaluation, we demonstrate that our algorithm achieves a relatively small bias and variance, showing significant improvements over both the randomized response method and non-clipping algorithms. Additionally, our approach to estimating Katz centrality successfully identifies up to 90\% of the nodes with the highest centrality values.
\end{abstract}

\section{Introduction}\label{sec:intro}
As discussed in  \cite{narayanan2009anonymizing,backstrom2007wherefore,zheleva2009join}, preserving the privacy of social network users' information is gaining in importance, especially when disclosing data or applying data mining algorithms to these networks. The typical method of ensuring privacy involves the obfuscation of the original social networks or the results of data mining. Various privacy concepts have been established to ensure that these obfuscated networks or outcomes provide sufficient privacy for users. A number of these concepts, such as $k$-diversity discussed in \cite{campan2008data} and $\ell$-diversity discussed in \cite{zhou2011k}, are notions designed for tabular data. 

In the realm of tabular data, differential privacy discussed in  \cite{dwork2006differential,dwork2014algorithmic} is among the most widely adopted privacy notions, as it provides a quantifiable measure of the amount of user information disclosed in a given publication, referred to as the privacy budget. The broad interest in this concept comes from its relative simplicity in calculating this privacy budget, even for complex data mining operations and data publications as discussed in \cite{mcsherry2009privacy}.

Numerous variations of differential privacy such as~\cite{soria2017individual,mironov2017renyi} are presented in literature. Among them, local differential privacy (LDP) discussed in \cite{evfimievski2003limiting,cormode2018privacy} is one of the most prominent. In differential privacy, the default assumption is that unaltered data is aggregated at a central server, and the obfuscation is performed on this complete data. However, LDP aims to safeguard user information during its transmission to the central server. Therefore, the data obfuscation occurs locally. Because the central server does not have access to the unmodified data at any time, it is typically more challenging to apply any data mining algorithm to the data that is protected under LDP.

Edge LDP, an augmentation of LDP proposed in 
 \cite{qin2017generating}, has been put forth specifically for the publication of social network information. Under the protection of edge LDP, it becomes hard to discern the presence of an edge or relationship within the input social network based on the published information. Multiple graph data mining algorithms such as \cite{hidano2022degree, sajadmanesh2021locally,ye2020lf} have been developed within the edge LDP framework. These algorithms include algorithms for subgraph counting in \cite{imola2021locally,imola2022communication,imola2022differentially,hillbrand2023}.

To the best of our understanding, all existing LDP-based counting algorithms attempt to count subgraphs identifiable via the local information, such as adjacency vectors, of a single node or a small number of nodes. This includes subgraphs like $k$-stars, triangles, or 4-cycles. No work, however, has been conducted on subgraphs which require consideration of adjacency vectors of multiple nodes. This is attributable to the fact that in LDP, these vectors are obfuscated independently. Despite the low probability of addition or removal of an edge from an adjacency list, the chance of obfuscation of an edge in a larger subgraph can be quite significant, which can result in a considerable error in the counted number.  

\subsection{Our Contribution}

Our contribution in this paper is as follows:
\begin{quote}
We propose an algorithm to estimate the number of walks with specific length in a social network under LDP, and apply this algorithm to provide an estimation of Katz centrality (\cite{katz1953new}), a prevalent social network centrality measure. Additionally, we carry out a thorough theoretical analysis of the algorithm.
\end{quote}

Although walks with long lengths involve several nodes in the graph, we can estimate the number using multiple rounds of communications and local clipping method. Our algorithm is discussed in Section~3. 

While the utilization of the clipping method is not a new concept and has been previously employed in \cite{imola2022communication}, we are the first to offer a theoretical guarantee for multiple rounds of communications in Section~4. Here, we give upper bounds for the variance and bias of our algorithm. Both of the upper bounds are relatively small. Our analysis facilitates the proposal of the optimal parameter for the clipping. A key factor in our analysis is our assumption that only a small number of nodes possess a large degree. It is worth noting that several practical social networks, such as those discussed in \cite{stephen2009explaining, clauset2009power} meet our assumption. Several works in differential privacy such as \cite{kasiviswanathan2013analyzing} have been conducted under comparable assumptions.

Section~5 confirms our theoretical results through experimental validations. This section illustrates that the bias and variance in our estimation of the number of walks and Katz centrality significantly decrease compared to the classical randomized response technique in \cite{warner1965randomized}. Moreover, our Katz centrality estimation effectively recalls up to 90\% of the nodes with peak Katz centrality. Consequently, it provides precise recommendations of the most influential nodes in the social networks, as gauged by Katz centrality.

Calculating walk counts and Katz centrality present difficulties not only to the LDP notion, but also to the general concept of differential privacy. The reason for this is that the number of walks can undergo massive shifts with the addition or removal of a single edge. This results in high sensitivity and requires the addition of substantial noise to the count in order to protect user information. We have attempted to use similar proof methods as in Section~4 to arrive at a lower upper bound on sensitivity. However, despite the improved upper bound, all the differential privacy notion algorithms we experimented with failed to surpass our Section~3 algorithm. Hence, we believe that not only is this algorithm optimal for LDP, but it is also the best differential privacy algorithm for estimating the number of walks and Katz centrality. 

\subsection{Related Works}

The domain of graph data mining under LDP is comparatively new, whereas mining under differential privacy has been a subject of investigation for several researchers over the years. Some of those works include \cite{gupta2010differentially,olatunji2021releasing}. As discussed in \cite{imola2021locally}, except for special cases such as \cite{zhang2020differentially}, LDP generally only allows for the concealment of an edge or relationship, while differential privacy, as in \cite{hay2009accurate,raskhodnikova2016differentially}, can also be used to hide whether an individual or node is part of a social network. In essence, there exists edge differential privacy and node differential privacy, but the concept of node differential privacy is not applicable in the context of LDP.

There are algorithms publishing centrality of graphs under differential privacy such as \cite{laeuchli2022analysis,task2012guide}. The most notable one is the differentially private algorithm for publishing PageRank centrality in \cite{epasto2022differentially}. One might think that the publication of PageRank and Katz centrality are similar as both of them are based on the repetition of matrix multiplication. We believe that publishing PageRank centrality under local differential privacy presents a significant challenge. Despite the centrality's sensitivity being relatively low, the PageRank value at a given node is deeply influenced by the network's overall information. This dependency on global data complicates the computation of PageRank in LDP. Nevertheless, we are of the view that our efforts on Katz centrality could serve as a foundational step towards enabling PageRank calculation under local differential privacy. 

\section{Preliminaries}

\subsection{Notations}
An input social network is denoted by $G = ([n],E)$ when $[n] = \{1,2,...,n\}$ and $E \subset [n]^{2}$. We use $\mathbb{G}_n$ to represent the collection of graphs consisting of $n$ nodes. 
For every $v \in [n]$, denote $a_{v} \in \{0, 1\}^{n}$ as the adjacency vector of node $v$. In this context, $a_{v}[u]=1$ signifies that nodes $v$ and $u$ are neighbors, otherwise it is 0.
For each vector $a \in \{0, 1\}^{n}$, let $\Gamma(a) \subset \{0,1\}^{n}$ correspond to the collection of lists that are different from $a$ by a single bit. The set $\eta(v) \subset [n]$ denotes the set of nodes adjacent to $v$ and $deg(v) < n$ denotes the degree of $v$ in the graph $G$.


Two graphs, $G = ([n],E)$ and $G' = ([n],E')$, are said to differ by a single edge if an edge $e \in [n]^{2}$ exists such that $E=E'\cup \{e\}$ or $E'=E\cup \{e\}$. The set of all graphs differing from $G$ by one edge is represented as $\Gamma(G) \subset \mathbb{G}_{n}$.

For every $\delta \in \mathbb{R}_{\geq 0}$, the Laplacian noise centered at 0 with a scale of $\delta$ is represented as $Lap(\delta)$.
For any $k > 0$ and $a \in \mathbb{R}^{k}$, the 1-norm of $a$ is denoted by $|a|$.


\subsection{Local Differential Privacy for Graph Data Mining}

Definitions of the local differential privacy (LDP) for graph and social network information used in this paper are as follows: 

\begin{definition}[$\epsilon$-edge LDP in \cite{qin2017generating}]
Let us consider a positive real number, denoted as \(\epsilon\), a node \(v\) within the set \([n]\), a randomized algorithm \(R_{v}\) that maps \(\{0,1\}^n\) to a set \(\mathsf{S}\), and another algorithm \(A\) that maps \(\mathsf{S}^n\) to a set \(S\). Define an algorithm \(\mathcal{A}\) such that \(\mathcal{A}(a_1, \ldots, a_n) = A(R_1(a_1), \ldots, R_n(a_n))\). This algorithm \(\mathcal{A}\) is said to provide \(\epsilon\)-edge local differential privacy (LDP) if, for any node \(v\), for any two adjacent lists \(a_{v}\) and \(a'_{v}\) differing in only one bit, and for any subset \(S\) of \(\mathsf{S}\), the probability \(\mathbb{P}[R_{v}(a'_{v}) \in S]\) is at most \(\exp(\epsilon) \times \mathbb{P}[R_{v}(a_{v}) \in S]\).
\label{def:edgeLDP}
\end{definition}

Next, we give the definition of sensitivity:

\begin{definition}[Sensitivity]
Let $R$ be a deterministic algorithm of which the domain is $\{0,1\}^{n}$ and the range is $\mathbb{R}^{k}$ for $k > 0$, we say that $R$ has a sensitivity of $\sigma$ if 
$\max\limits_{a \in \{0,1\}^{n},a' \in \Gamma(a)} |R(a)-R(a')| \leq \sigma$
\end{definition}

In the next definition, we give an algorithm which satisfies the $\epsilon$-edge LDP. This algorithm is commonly referred to as the Laplacian mechanism.

\begin{definition}[Laplacian Mechanism in \cite{dwork2006calibrating}] 
Let $R_i: \{0, 1\}^{n} \rightarrow \mathbb{R}^{k}$ be a deterministic algorithm, let $\sigma_i$ be the sensitivity of $R_i$, and let $Y_i = (Y_{i1},\dots,Y_{ik})$ where the $Y_{ij}$ are drawn independently from $Lap(\sigma_i/\epsilon)$. We say that $R_i': \{0, 1\}^{n} \rightarrow \mathbb{R}^{k}$ is a publication of $R_i$ under the Laplacian mechanism if $R_i'(a_i) = R_i(a_i) + Y_i$.      
\end{definition}

The following theorem can be straightforwardly derived from Proposition 1 of \cite{dwork2006calibrating}.

\begin{theorem}  
For all $i$, let $R_i'$ be a publication of $R_i$ under the Laplacian mechanism. Then, for any algorithm $A: \left(\mathbb{R}^k\right)^n \rightarrow S$, an algorithm $\mathcal{A}$ such that $\mathcal{A}(a_1, \dots, a_n) = A(R'_1(a_1), \dots, R'_n(a_n))$ provides $\epsilon$-edge LDP. \label{thm:ldprdp}
\end{theorem}

Next, we introduce the composition theorem for the edge LDP. The result can be straightforwardly derived from \cite{dwork2010boosting}.
\begin{theorem}[Composition Theorem from \cite{dwork2010boosting}] Let $\mathcal{A}_1, \dots, \mathcal{A}_p$ be edge LDP mechanism with privacy budget $\epsilon_1, \dots, \epsilon_p$. Then, the mechanism $\mathcal{A}_p \circ \cdots \circ \mathcal{A}_1$ is $(\varepsilon_1 + \cdots + \varepsilon_p)$-edge LDP.
\end{theorem}

\subsection{Number of Walks and Katz Centrality}

For every \( v \in [n] \) and \( k \in \mathbb{N} \), our goal is to compute the vector \( P^{(k)} \) where \( P^{(k)}[v] \) denotes the number of walks of length \( k \) originating from \( v \). We can determine \( P^{(k)}[v] \) for each \( v \in [n] \) based on the principle that, for all \( k > 0 \) and \( v \in [n] \), \( P^{(k)}(v) = \sum\limits_{u \in \eta(v)} P^{(k-1)}(u) \).


Introduced in \cite{katz1953new}, Katz centrality is a widely recognized method for assessing the significance of nodes in networks. Let \( \alpha \) be a constant, referred to as the attenuation factor. For a node \( v \in [n] \), its Katz centrality is defined as $Katz[v] = \sum\limits_{k=1}^{\infty} \alpha^{k} P^{(k)}[v].$

For any positive integer \(i\) and a vertex \(v\) in a set of \(n\) vertices, define \(K^{(i)}[v]\) as \(\alpha^i P^{(i)}[v]\). It follows that \(K^{(i)}[v] = \alpha \cdot \sum_{u \in \eta(v)} K^{(i-1)}[u]\) and the Katz centrality of \(v\) is expressed as \(Katz[v] = \sum_{k = 1}^{\infty} K^{(k)}[v]\). Considering a finite number of steps \(S\), the Katz centrality can be approximated by \(Katz[v] = \sum_{k = 1}^{S} K^{(k)}[v]\). An instance of this calculation can be found in the appendix, specifically in Example~\ref{ex1}.
%The accuracy of this approximation for Katz centrality improves exponentially with the increase in the number of steps \(S\).


%Let $A$ be the adjacency matrix of the input graph $G$. It is known that, for the series to converge, \( \alpha \) must be less than the inverse of the maximum absolute eigenvalue of \( A \). 
One can calculate the Katz centrality vector as $Katz = ((I - \alpha A^{T})^{-1} - I) J,$
where \( I \) is the identity matrix and \( J \) is an \( n \)-dimensional vector filled with ones. This can be computed by a single matrix inversion, but the algorithm mentioned in the previous paragraph is easier to adapt to an LDP framework.
It is important to note that $K^{(i)} = P^{(i)}$ for all $i$ if we set $\alpha = 1$. 

\begin{comment}

\end{comment}

\section{Our Estimator}


Our estimation approach for Katz centrality values is presented in Algorithm \ref{alg:LDPKatz}, and an illustrative example of this can be found in Example \ref{ex1} within the appendix. If we set $\alpha = 1$, the algorithm gives us an estimator of the number of walks where $\widetilde{K}^{(i)}$ is an estimator of $P^{(i)}$.

The underlying principle of the algorithm recognizes that while Katz centrality inherently depends on the global graph topology, the iterative calculation of $K^{(i)}[v]$ for each node $v$ are autonomous, relying solely on its immediate neighbors. Consequently, the algorithm can be decentralized, allowing each node to perform its computations independently. In other words, since every node \( u \in [n] \) requires \( K^{(i-1)} \) to determine \( K^{(i)}[u] \), nodes relay their results after each step to a centralized server. This server, in turn, disseminates the entire vector \( K^{(i-1)} \) to all nodes before initiating step \( i \).
 

To ensure differential privacy, each node \( u \) incorporates Laplacian noise into \( K^{(i)}[u] \) prior to its transmission and also before it contributes to the Katz centrality estimation (as seen in line 11). The central server remains unaware of the graph's edge details and serves solely as a communication facilitator, ensuring that our algorithm is secure under the LDP notion.

The initial version of Algorithm~\ref{alg:LDPKatz}, excluding lines 12-13, represents our preliminary design and will serve as a comparison standard in our Section 5 experiments. However, this iteration presents an inherent flaw. The magnitude of the Laplacian noise must align with the sensitivity, denoted as \(\max\limits_{v \in [n]} |\widetilde{K}^{(i-1)}[v]|\). This becomes problematic as this magnitude can escalate considerably, potentially compromising the accuracy of our estimator from both theoretical and practical perspectives.


Motivated by \cite{epasto2022differentially}, we incorporated a clipping strategy (as presented in lines 12-13). By constraining \(K^{(i)}\) during the \(i\)th step, we aim to diminish the sensitivity when deriving \(K^{(i+1)}\) in the subsequent step. This adaptation holds potential to enhance the estimator's efficacy by minimizing noise and, consequently, variance, though it might introduce a certain bias. It is important to clarify that this clipping is executed subsequent to the incorporation of \(K^{(i)}\) into \(\widetilde{Katz}\) (as detailed in line 11). The primary intent behind the clipping is not the preservation of differential privacy during the \(i\)-th phase but rather the attenuation of sensitivity for the \(i+1\) step.


\begin{algorithm}
\SetAlgoLined
\small
\SetKwInOut{Input}{Input}\SetKwInOut{Output}{Output}

\Input{Graph $G = ([n],E)$, attenuation factor $\alpha$, clipping factor $X$, privacy budget $\epsilon$, number of step $S$}
\Output{Vector $\widetilde{Katz}$ of size $n$ where $\widetilde{Katz}[v]$ is the estimated Katz centrality of node $v \in [n]$ under $\epsilon$-edge LDP}
\BlankLine
\For{$v \in [n]$}{
\textbf{[User $v$]} $\widetilde{Katz}[v] \gets 0$\;
\textbf{[User $v$]} $\widetilde{K}^{(0)}[v] \gets 1$ \;
}
\For{$i = 1$ \KwTo $S$}{
    \textbf{[Server]} $\pi \gets \frac{\alpha S}{\epsilon} \cdot \max\limits_v 
    |\widetilde{K}^{(i - 1)} [v]|$ \;
    \textbf{[Server]} Distribute $\pi$ and $\widetilde{K}^{(i - 1)}$ to all users \;
    \For{$v \in [n]$}{
    \textbf{[User $v$]} $\widetilde{K}^{(i)}[v] \gets \alpha \cdot \sum\limits_{u \in \eta(v)} \widetilde{K}^{(i - 1)}[u]$\;
    \textbf{[User $v$]} $\widetilde{K}^{(i)}[v] \gets \widetilde{K}^{(i)}[v] + Lap(\pi)$ \;
    \textbf{[User $v$]} $\widetilde{Katz}[v] \gets \widetilde{Katz}[v] + \widetilde{K}^{(i)}[v]$\;
    \textbf{[User $v$]} $\widetilde{K}^{(i)}[v] \gets \min\{\widetilde{K}^{(i)}[v], (\alpha X)^i\}$ \;
    \textbf{[User $v$]} $\widetilde{K}^{(i)}[v] \gets \max\{\widetilde{K}^{(i)}[v], -(\alpha X)^i\}$ \;
    \textbf{[User $v$]} Communicate $\widetilde{K}^{(i)}[v]$ to the central server.
    }
}
\Return $\widetilde{Katz}$
\caption{Algorithm to estimate Katz centrality under \( \epsilon \)-edge LDP}
\label{alg:LDPKatz}
\end{algorithm}

\begin{comment}
\begin{example}

\end{example}
\end{comment}

It is evident that the magnitude of noise appended to \( \tilde{K}^{(i)}[v] \) can be significantly high in comparison to its original value determined at line 9. Specifically, for a node \( v \) with only one neighbor, it follows that \( \tilde{K}^{(i)}[v] \) is bounded by \( \frac{S}{\epsilon} \times \pi \). Given that \( \frac{S}{\epsilon} \) typically exceeds one, the standard deviation of the added noise is often greater than the original value. Conversely, a node \( v \) with a higher degree exhibits a larger \( \tilde{K}^{(i)}[v] \), where its magnitude usually surpasses that of the noise. As a result, the algorithm delivers precise Katz centrality calculations for nodes with higher centrality values, yet it becomes less accurate for nodes with lower centrality. This characteristic of the algorithm makes it suitable for identifying nodes with the top \( k \) centrality values, but it is not as effective for ranking the centrality of all nodes.

In practical applications, Algorithm~\ref{alg:LDPKatz} functions as a distributed algorithm. Considering \( D \) as the highest degree of the input graph, each user faces a computational complexity of \( O(D) \) in every step of the algorithm.
As each node has to upload one real number to the server and download $O(n)$ real numbers per iteration, 
the communication complexity per step is \( O(n) \). Typically, the number of steps is set to \( O(\log n) \), leading to an overall computational complexity for each user of \( O(D \log n) \) and a communication complexity of \( O(n \log n) \). The communication complexity could be reduced by the sampling technique proposed in \cite{hillebrand2023communication}, but that reduction is out of scope for this work. 

When $\alpha$ is 1, $\tilde{K}^{(i)}[v]$ can be regarded as an estimate of the number of walks with length $i$ beginning from node $v$.

\subsection{Privacy}

From the following lemma, we show that Algorithm~\ref{alg:LDPKatz} is $\epsilon$-edge LDP. We begin by discussing the privacy of the communication at Line 14 of the algorithm.

\begin{lemma}
The communication of $\widetilde{K}^{(i)}[v]$ at Line 14 of Algorithm~3 is ($\epsilon/S$)-edge LDP. \label{lem:pri}
\end{lemma}
\begin{proof}
Consider $a_v$ and $a_v'$ as adjacency vectors of node $v$, differing by a single bit. Let $\widetilde{K}_{\text{max}} := \max\limits_v |\widetilde{K}^{(i-1)}[v]|$, and suppose $\widetilde{K}^{(i)}[v]$ and $\widetilde{K'}^{(i)}[v]$ are the computation results acquired from Lines 9 and 12-13 of the algorithm when the adjacency vector is $a_v$ and $a_v'$. We find that $|\widetilde{K}^{(i)}[v] - \widetilde{K'}^{(i)}[v]| \leq \alpha \cdot \widetilde{K}_{\text{max}}$. Therefore, the sensitivity of transmitting $\widetilde{K}^{(i)}[v]$ is $\sigma = \alpha \cdot \widetilde{K}_{\text{max}}$. Given that we set the Laplacian noise parameter to $\frac{\sigma S}{\epsilon}\cdot\widetilde{K}_{\text{max}}$, the Laplacian mechanism at Line 10 exhibits $(\epsilon/S)$-edge LDP. Any post-processing not related on the edge set will not alter the privacy outcome. Hence, the communication at Line 14 ensures ($\epsilon/S$)-edge LDP, despite the post-processing at Lines 11-13 of the algorithm.
\end{proof}

We are now ready to show the privacy of Algorithm~\ref{alg:LDPKatz} in the following theorem. 

\begin{theorem}
Algorithm \ref{alg:LDPKatz} is an $\epsilon$-edge LDP. \label{thm:pri}
\end{theorem}

\begin{proof}
 The private information \(\tilde{K}^{(i)}[v]\) is communicated \(S\) times. According to Lemma \ref{lem:pri}, each communication ensures \((\epsilon/S)\)-edge LDP. Applying the composition theorem, it follows that Algorithm \ref{alg:LDPKatz} is $\epsilon$-edge LDP.
\end{proof}

\section{Loss of Our Estimator}

In the following discussion, we conduct a theoretical evaluation of the accuracy of the algorithm proposed in the previous section. We observed that in the majority of social networks, a handful of nodes exhibit a considerably larger degree than the rest, as affirmed several works including \cite{stephen2009explaining, clauset2009power}. This observation motivates our assumption in the following analysis. Here, we assume that the maximum degree of the input graph $G = ([n],E)$ is $D$, and there is at most $N \ll n$ nodes exhibit a degree greater than $d \ll D$.

Let us revisit the clipping factor $X$ as outlined in the preceding section. For this section, we select parameters $d$ and $X$ such that they satisfy the conditions $NX + Dd \leq X^2$ and $X \leq D$. As previously stated, we operate under the assumption that $N$ and $d$ is small, thus intuitively setting $X$ to $O\left(\sqrt{D}\right)$. 

It is noteworthy that it is always possible to identify parameters $d$ and $X$ that fulfil the condition, specifically when $X = d = D$. This results in $N = 0$, and consequently, both of the conditions are satisfied. However, by assigning $X = D$, the computation at lines 12-13 of Algorithm~2 becomes nearly insignificant as $\widetilde{K}^{(i)}[v]$ is typically less than $(\alpha D)^i$. For the computation to be utilized effectively, we generally aim to set the parameter of $X$ to the lowest possible value. The most ideal situation is when we can assign $X$ a value approximately equal to $\sqrt{D}$.

\begin{comment}
\begin{example}
In the context of the path graph described in Example \ref{ex1}, the diameter \( D \) is 2. Setting \( d = 1 \) results in \( N = 3 \). To meet the stipulated assumption, the clipping factor \( X \) should fulfill the condition \( 3X + 2 \leq X^2 \). Consequently, it follows that \( X \) must be greater than 3.56.
\end{example}
\end{comment}

\subsection{Bias of Algorithm~\ref{alg:LDPKatz}}

In this section, we give an upper bound of the bias of Algorithm~\ref{alg:LDPKatz} as an estimator of Katz centrality.  Let $\phi$ be the golden ratio. The main results of this section are as follows:

\begin{theorem} [Bias of counting number of walks]
\label{thm:biaswalk} For Algorithm~\ref{alg:LDPKatz}, when considering an attenuation factor of \(\alpha=1\), a number of steps \(S\) such that \(S \geq i\), and satisfying the condition \(X^{2}/D + X \leq X^{2}\), the bias of the estimator for the number of walks of length \(i\) is given by
$\max\limits_{v \in [n]} \mathbb{E}[P^{(i)}[v]-\widetilde{K}^{(i)}[v]] \leq 2 (\phi X)^{i-1} D \frac{S}{\epsilon}$.
\end{theorem}

For the node with large Katz centrality, the number of walks of length \(i\) originating from node \(v\) can scale as \(D^i\). When \(X \ll D\), it is evident that the upper bound presented in Theorem \ref{thm:biaswalk} is significantly less than the trivial upper bound. Thus, while the clippings in lines 12-13 do introduce a certain level of bias, this bias is not large compared to the genuine number of walks.


\begin{theorem} \label{thm:bias}
 The bias of the Katz centrality estimator in Algorithm~\ref{alg:LDPKatz} with attenuation factor $\alpha < 1/ \phi X$, number of step $S$, and privacy budget $\epsilon$ such that $S/\epsilon \geq 1$  and $X^{2}/D + X \leq X^{2}$ is $\max\limits_{v \in V} \mathbb{E}[Katz(G)[v] - \widetilde{Katz}(G)[v]] \leq \frac{\alpha S}{\epsilon}\left(1 + \frac{2 \alpha \phi D X}{1- \alpha \phi X}\right).$
\end{theorem}

For values of \(\alpha\) less than \(1/2\phi X\), the expression \(\alpha\left(1 + \frac{2 \alpha \phi D X}{1- \alpha \phi X}\right)\) approaches a minimal constant. Consequently, the bias of the Katz centrality estimator becomes linearly proportional to \(S/\epsilon\). This suggests that, for those $\alpha$, our bias does not escalate rapidly with additional steps in Algorithm~\ref{alg:LDPKatz} and with a better level of differential privacy. The proof of Theorem \ref{thm:biaswalk} and \ref{thm:bias} can be found in the appendix.

It is established that the factor \(\alpha\) must be less than \(1/\lambda_n\), where \(\lambda_n\) is the largest eigenvalue of the adjacency matrix (see Page 78 of \cite{junker2008analysis}). There are currently no known methods to estimate \(\lambda_n\) in a Local Differential Privacy (LDP) setting. However, \(\lambda_n\) can be upper bounded by the maximum degree \(D\) of the graph (refer to Lemma 3.4.1 of \cite{lecturenote1}), which can be approximated using the method described in \cite{hillebrand2023communication}. Consequently, \(\alpha\) should not exceed \(1/D\). Additionally, according to Theorem \ref{thm:bias}, our method permits \(\alpha\) to reach up to \(1/(2\phi X)\), also estimable through the method in \cite{hillebrand2023communication}. This value is significantly greater than \(1/D\) since \(X = O(\sqrt{D})\). Thus, our approach not only minimizes bias in the estimation but also permits a larger \(\alpha\) value in estimating Katz centrality under LDP.


\subsection{Variance of Algorithm~\ref{alg:LDPKatz}}
Let $L = \max(N D,X^{2})$. The following theorems provide the upper bound of the variance for the number of walks and Katz centrality that Algorithm~\ref{alg:LDPKatz} publishes.

\begin{theorem} \label{thm:walkvariance}
For the estimator of the number of walks of length \( i \) obtained from Algorithm~\ref{alg:LDPKatz}, given an attenuation factor \( \alpha=1 \), number of steps \( S\geq i \), and satisfying the condition \( X^{2}/D+X \leq X^{2} \), the variance is bounded by 
$\max\limits_{v \in [n]} \mathrm{Var}[\widetilde{K}^{(i)}[v]] \leq \frac{16 S^{2} (D^{2} + X^{2}) (4 L)^{i-2}}{\epsilon^{2}}.$
\end{theorem}


The theorem suggests that the standard deviation of our publication scales as \( (2X)^{i - 1} \). Given that \( X \ll D \) and the typical walk can scale as \( D^i \), the upper bound of the standard deviation is not large compared to the actual walk count.


\begin{theorem} \label{thm:katzvariance}
The variance of the Katz estimator published by Algorithm~\ref{alg:LDPKatz} using attenuation factor $\alpha \leq 1/(2\sqrt{L})$, number of step $S$ and privacy budget $\epsilon$ is $\max\limits_{v \in V} \mathrm{Var}[\widetilde{Katz}(G)[v]] \leq \frac{4 S^{2} \alpha^{2}(D^{2} + X^{2})}{ L \epsilon^{2} (1 - 2 \alpha \sqrt{L})^{2}}.$
\end{theorem}

For values of \( \alpha \) such that \( \alpha \leq \frac{1}{4\sqrt{L}} \), the term \( \frac{8 \alpha^2 (D^2 + X^2)}{L(1 - 2\alpha\sqrt{L})^2} \) tends toward a negligible constant. As a result, the standard deviation of the Katz estimator produced by our algorithm aligns with the order of \( \frac{S}{\epsilon} \). This observation indicates that, for these values of \( \alpha \), the variance remains relatively stable even as the step count \( S \) and privacy parameter \( \epsilon \) vary. The proofs of Theorem \ref{thm:walkvariance} and \ref{thm:katzvariance} can be found in the appendix.

In Theorems~\ref{thm:walkvariance} and \ref{thm:katzvariance}, the standard deviation increases linearly with $S/\epsilon$. In contrast, Theorem~\ref{thm:noclip} in Section \ref{sec:withoutClip} of the appendix shows an exponential growth in noise. The size of the noise is $\Omega\left((\log n)^S\right)$ for any value of $\alpha$ and $\epsilon$. This suggests that the unclipped variant will have considerably greater variance. Even though clipping introduces a bias, its presence greatly diminishes the algorithm's variance. As a result, our algorithm is anticipated to perform notably better than its unclipped counterpart.

Theorem \ref{thm:katzvariance} confirms that the variance of our algorithm remains relatively low when \(\alpha \leq 1/2\sqrt{L} = O(1/X) = O(1/\sqrt{D})\). This finding aligns with the results from Theorem \ref{thm:bias}. Together, these theorems assure us that the algorithm can accommodate a significantly large factor \(\alpha\).

\section{Experiments}

In this section, we enhance our theoretical insights with empirical evaluations of our algorithm. Specifically, we examine its efficacy in estimating the Katz centrality and the number of walks.  

\noindent \textbf{Dataset : } We conducted our experiments using two graphs sourced from the Stanford Network Analysis Platform (SNAP) in  \cite{snapnets}. The first graph represents the social circles from Facebook, as described in \cite{leskovec2012learning}. This undirected graph has 4,039 nodes and 88,234 edges. Its average degree is at 43.69, with the highest degree is 1,045, and a maximum eigenvalue of $E_{F} = 162.37$. The second graph illustrates voting patterns on Wikipedia, based on references \cite{leskovec2010signed,leskovec2010predicting}. This directed graph has 7,115 nodes and 103,663 edges. Its average degree is 14.57, with the highest degree of 1,167, and its maximum eigenvalue is $E_{W} = 45.14$. In this manuscript, for ease of reference, we refer to the first graph as the ``Facebook graph'' and the second graph as the ``Wikipedia graph.'' Besides these two graphs, we have also carried out experiments on various subgraphs derived from them. We specifically selected these two graphs as they exemplify two distinct types of social networks. The Facebook graph typifies a social network characterized by multiple clusters, whereas the Wikipedia graph is representative of social networks that revolve around a few central nodes.

%\noindent \textbf{Baseline : } To our understanding, this work pioneers the LDP technique for estimating number of walks and Katz centrality. For benchmarking purposes, both for Katz centrality and the number of walks, we employ our algorithm, excluding the clipping mechanism (specifically, Algorithm~\ref{alg:LDPKatz} without lines 12-13). This comparative analysis aims to ascertain the tangible impact of the clipping concept on the efficiency of our method.

\noindent \textbf{Privacy Budget:} Unless specified otherwise, we adopted a privacy budget set at $\epsilon = 0.5$, a commonly accepted benchmark \cite{imola2021locally}. Based on our theoretical findings, we anticipate analogous outcomes for different values of $\epsilon$.


\noindent \textbf{Attenuation Factor:} As discussed after Theorem \ref{thm:bias}, the reciprocal of the graph's maximum eigenvalue acts as an upper limit for the attenuation factor. We opted for attenuation factors near this threshold, setting $\alpha_{F} = 0.85/E_{F}$ and $\alpha_{W} = 0.85/E_{W}$. Such values render the Katz centrality estimation more challenging. Specifically, for small values of $\alpha$, the relation $Katz[v] \approx \alpha \times deg(v)$ holds true for all nodes $v \in [n]$.

\noindent \textbf{Number of Steps:} We set the number of steps \(S\) for our algorithm at 5, except the experiments in Figures \ref{fig:SandLoss}, \ref{fig:SandRecall}, and~\ref{fig:walk}.

\subsection{Comparisons with the Randomized Response Technique}

In the absence of existing research on publishing the number of walks and Katz centrality under LDP, we opted to benchmark our results against the conventional randomized response technique, as described in \cite{warner1965randomized}. We define this technique as \( \mathcal{A}(a_1, \ldots, a_n) = A(R_1(a_1), \ldots, R_n(a_n)) \), where \( R_i([a_{i,1}, \ldots, a_{i,n}]^t) \) results in \( [a^{RR}_{i,1}, \ldots, a^{RR}_{i,n}]^t \). Each element \( a^{RR}_{i,j} \) retains its original value \( a_{i,j} \) with a probability of \( \frac{e^\epsilon}{1 + e^\epsilon} \) and switches to \( 1 - a_{i,j} \) with a probability of \( \frac{1}{1 + e^\epsilon} \). The function \( A \) then publishes the number of walks and Katz centrality for the graph \( G' \), which is represented with the adjacency vector \( (a'_{i,j})_{1 \leq i,j \leq n} \). For \( i < j \), the values of \( a'_{i,j} \) and \( a'_{j,i} \) are set to \( a^{RR}_{i,j} \).

In all the figures, we refer to the randomized response algorithm as ``randomized'', while refer to our algorithm as ``clipping''. 


\begin{figure}
  \centering 
  \subfloat[Varying the number of steps $S$ on the Facebook graph]{\includegraphics[width=3.9cm]{loss_facebook.png}\label{fig:f11}}
  \hfill
  \subfloat[Varying the number of steps $S$ on the Wikipedia graph]{\includegraphics[width=4.3cm]{loss_wiki_randomized.png}\label{fig:f12}}
  \captionsetup{justification = centering,font = small}
  \caption{Loss and variance of our Katz estimators (Algorithm~\ref{alg:LDPKatz}) compared with the randomized response technique}
  \label{fig:SandLoss}
\end{figure}

In our analysis, we initially evaluate the loss and variance of our method relative to the randomized response technique, as illustrated in \textbf{Figure} \ref{fig:SandLoss}. The ``loss'' and ``variance'' are defined as the sum of the $\ell_2$-loss and the sum of variances across all nodes in the graph, respectively. For each node \(v\), let \(Katz[v]\) denote its Katz centrality and \(\widetilde{Katz}[v]\) denote the centrality estimated by our algorithm. The reported loss is given by \(\sum\limits_v (Katz[v] - \widetilde{Katz}[v])^2\), while the reported variance is \(\sum\limits_v Var[\widetilde{Katz}[v]]\).
Across all the number of steps and for both input graphs, our algorithm consistently achieves a notably lower loss and variance compared to the conventional approach.

Our variance results matches with Theorem \ref{thm:katzvariance}. The variance grows linearly with $S^2$ in both of the social networks. On the other hands, contrary to the upper limit set in Theorem \ref{thm:bias}, the bias of our algorithm — which represents the discrepancy between the loss and the variance — generally diminishes with an increasing number of steps. 

The variance and bias of the randomized response algorithm exhibit an increasing trend with the growth of \( S \). This effect is particularly pronounced in the Wikipedia graph, where both variance and loss demonstrate exponential growth in relation to \( S \). Such a pattern suggests that, while Katz centrality converges in the original input graph \( G \), it fails to do so in the randomized graph \( G' \). This outcome is likely a frequent occurrence with the traditional technique, given the substantial alterations it introduces to the adjacency matrix.

\begin{figure}
  \subfloat[Varying the number of steps $S$ on the Facebook graph]{\includegraphics[width=4.1cm]{Detection_Facebook_Steps.png}\label{fig:f21}}
  \hfill
  \subfloat[Varying the number of steps $S$ on the Wikipedia graph]{\includegraphics[width=4.1cm]{Detection_Wiki_Steps.png}\label{fig:f21}}
  \captionsetup{justification = centering,font = small}
  \caption{Recall of nodes with largest Katz centrality by our estimator (Algorithm~\ref{alg:LDPKatz}) compared with the randomized response technique}
  \label{fig:SandRecall}
\end{figure}

While the previous results provide valuable insights, in many practical scenarios, the primary concern is not the estimator's loss but its capability to identify the top $k$ nodes with the highest Katz centrality. Our next experiment focuses on this aspect. We ranked nodes based on the true Katz centrality values and compared them to rankings from our estimators. For specific values of $k$, we evaluated the percentage of top $k$ nodes, according to the real Katz centrality, that also appeared in the top $k$ nodes of each estimator. \textbf{Figure} \ref{fig:SandRecall} display the recall of these top $k$ nodes, considering $k$ values of 10 and 100, along with confidence intervals. 
%For \textbf{figure 2.a} and \textbf{2.c}, we applied a clipping factor $X$ equivalent to $E_{F}$ and $E_{W}$, respectively. Meanwhile, \textbf{figure 2.b} and \textbf{2.d} were based on 5 steps and clipping factors $X$ that varied between $0.5 E_{F}$ to $2 E_{F}$ and $0.5 E_{W}$ to $2 E_{W}$, respectively.

With our Katz centrality estimator applied to the Facebook graph, we successfully identified approximately 90\% of the top 100 nodes and around 80\% of the top 10 nodes. For the Wikipedia graph, our estimator achieved a 91\% identification rate for both the top 10 and top 100 nodes. These recall rates are significantly higher than those achieved using the randomized response technique. In the Facebook graph, the recall of the standard technique for the top 10 and top 100 nodes is only about 20\% and 0\%, respectively. For the Wikipedia graph, the standard technique's recall rates are 89\% for the top 10 nodes and 79\% for the top 100 nodes. The number of iterations \( S \) plays a crucial role in influencing recall rates. In the Facebook graph, we achieve maximum recall at \( S = 5 \), while in the Wikipedia graph, the peak recall is attained at \( S = 9 \).

As noted in Section 3.1, our methodology accurately computes the Katz centrality for nodes with higher centrality values, whereas it tends to be less precise for nodes of lesser importance. Consequently, our estimator is not ideally suited for ranking the centrality of all nodes in a network, but rather for identifying those with the highest centrality. As discussed in \cite{olsen2014efficient,bergamini2019computing}, pinpointing the top \( k \) nodes is crucial, as it holds significant relevance for numerous applications in social networks.

\begin{figure}
  \subfloat[Loss and Variance]{\includegraphics[width=4.1cm]{loss_facebook_by_nodes.png}\label{fig:f31}}
  \hfill
  \subfloat[Recall of the top $k$ nodes]{\includegraphics[width=4.1cm]{Detection_Per_Nodes.png}\label{fig:f31}}
  \captionsetup{justification = centering,font = small}
  \caption{Results of our estimator (Algorithm~\ref{alg:LDPKatz}) compared with the randomized response technique varying the graph size}
  \label{fig:numberofNodes}
\end{figure}

In \textbf{Figure} \ref{fig:numberofNodes}, we explored whether our results are scalable with the size of the graph. The figure showcases an analysis where we selected subsets of nodes from the Facebook graph through random walks and computed the Katz centrality on the subgraphs induced by these node sets. We then compared the performance of our method in terms of loss, variance, and recall of the top \( k \) nodes against the randomized response technique. While our approach exhibits a higher variance compared to the classical technique in these subgraphs, it demonstrates significantly lower loss and larger recall. The plot also reveals that, as the subgraph size increases, the performance of the randomized response deteriorates, whereas our results remain consistent across various subgraph sizes. This suggests that the improvements we reported in Figures \ref{fig:SandLoss} and \ref{fig:SandRecall} are likely to be even more pronounced in larger graphs.

We opted for generating subgraphs through random walks due to our belief that this method effectively maintains the graph's structural integrity, especially in the context of Katz centrality. Random sampling techniques tend to yield much sparser graphs, potentially degrading the performance of all estimators. In contrast, using breadth-first search for subgraph generation often results in subgraphs dominated by a few high-degree nodes. These high-degree nodes are typically those with the highest Katz centrality, rendering the task of identifying top nodes overly simplistic. Therefore, random walks strike a balance by preserving the essential characteristics of the original graph, which is crucial for our analysis.

\begin{figure}
  \subfloat[Loss and Variance]{\includegraphics[width=4.1cm]{loss_vs_randomized_epsilon.png}\label{fig:f41}}
  \hfill
  \subfloat[Recall of the top $k$ nodes]{\includegraphics[width=4.1cm]{Detection_Per_Eps.png}\label{fig:f41}}
  \captionsetup{justification = centering,font = small}
  \caption{Results of our estimator (Algorithm~\ref{alg:LDPKatz}) compared with the randomized response technique varying the privacy budget}
  \label{fig:epsilon}
\end{figure}

Finally, as illustrated in \textbf{Figure} \ref{fig:epsilon}, our algorithm consistently outperforms the randomized response across all privacy budget values $\epsilon$. Notably, when the privacy requirement is more stringent (indicated by a smaller $\epsilon$), the performance of the randomized response tends to decline due to an increase in the flipping of relationships within the graph. In contrast, the performance of our algorithm remains relatively stable regardless of the $\epsilon$ value. Consequently, our approach shows a more pronounced improvement, especially at lower values of $\epsilon$.

%Finally, by matching the results from \textbf{figure} \ref{fig:SandLoss} and \ref{fig:SandRecall}, we notice that the best performance for detecting the top $k$ nodes are not achieved for the smallest loss of the estimator but rather for smaller value of bias even at a cost of larger value of variance and overall loss. Indeed, for ranking nodes, a larger bias does not affect the performance as much as a larger variance since a larger bias means the Katz centrality estimation of every node have a similar negative error, possibly not changing the ranking, while adding noise almost certainly changes the ranking.

\subsection{Results Related to Clipping Factor}


\begin{figure}
  \centering 
  \subfloat[Loss and variance on the Facebook graph]{\includegraphics[width=4cm]{loss_clipping.png}\label{fig:f41}}
  \hfill
  \subfloat[Loss and Variances on Wikipedia graph]{\includegraphics[width=4cm]{loss_clipping_wiki.png}\label{fig:f42}}
  \hfill
  \subfloat[Recall on the Facebook graph]{\includegraphics[width=4cm]{dectection_clipping.png}\label{fig:f43}}
  \hfill
  \subfloat[Recall on the Wikipedia graph]{\includegraphics[width=4cm]{detection_clipping_wiki.png}\label{fig:f44}}
  \captionsetup{justification = centering,font = small}
  \caption{Performance evaluation of Algorithm~\ref{alg:LDPKatz} across various clipping factor \(X\) values } \label{fig:VaryX}
\end{figure}

This section is dedicated to verifying the impact of the clipping process implemented in lines 12-13 of Algorithm~\ref{alg:LDPKatz}. We aim to determine the most optimal value for the clipping factor \( X \). Our findings are presented in \textbf{Figures}~\ref{fig:f41} and \ref{fig:f42}.

As anticipated, a smaller clipping factor \( X \) results in reduced variance. This occurs because the noise added in our process is relatively minor in such cases. However, on the flip side, both the loss and bias are significantly higher due to the aggressive clipping of results when \( X \) is small. As we increase \( X \), a higher variance and lower bias are observed. Initially, we notice a rapid decrease in bias with a comparatively slower increase in variance, leading to a reduction in overall loss for larger values of \( X \). Yet, beyond a certain threshold, the bias ceases to decrease, while the variance continues to escalate, causing the loss to increase as \( X \) is further augmented. In summary, there exists an optimal value of the clipping factor \( X \) that minimizes the loss for both the Facebook and Wikipedia networks.
We notice that the optimal point is around the maximum eigenvalues (denoted by $E_F$ and $E_W$) in both of the input graphs.

In Algorithm~\ref{alg:LDPKatz}, if the clipping is omitted in lines 12-13, the outcome is equivalent to setting $X \rightarrow \infty$. This is supported by the evidence in Figure \ref{fig:VaryX}, which shows a deterioration in results as the value of $X$ surpasses the maximum eigenvalues. A significant decline in performance is expected without clipping, as elaborated in Section \ref{sec:withoutClip} of the appendix. Specifically, with an increased number of steps $S$, the results tend to degrade. For instance, at $S = 9$, the loss in the Facebook graph is approximately tripled without clipping. Furthermore, in the Wikipedia graph, the loss escalates by over $10^4$ times.


%Firstly, we have tested the performance of our algorithms by computing their loss and variance as estimators of the Katz centrality. The results are shown on \textbf{figure 1}. For \textbf{figure 1.a} and \textbf{1.c}, we used a clipping factor of $X = E_{F}$ and $X = E_{W}$ respectively and for \textbf{figure 1.b} and \textbf{1.d} we used a number of steps of $S=5$.



%In \textbf{figure 1.a} and \textbf{1.c}, it is evident that with just one step, both algorithms exhibit a pronounced bias. This is because they only account for the number of walks of length 1 when estimating Katz centrality. As a result, they display minimal variance due to the limited noise introduced. As the step count grows, meaning more walks are computed and more noise is integrated, we observe a decline in bias and a rise in variance across both algorithms. With increasing steps \(S\), the bias of the non-clipping algorithm (illustrated in orange) approaches zero since the Laplacian noise does not inherently introduce any bias. Yet, its variance surges sharply. Our theoretical evaluation from section 4.3 forecasts an exponential surge in variance when \( \alpha S H_{n}/\epsilon > 1\). This is corroborated in the experimental data, manifesting for \(S \geq 12\) in the Facebook graph and \(S \geq 4\) in the Wikipedia graph. As anticipated, the clipping mechanism (depicted in blue) effectively moderates the variance's escalation. While the bias does not reduce to zero as seen in the algorithm without clipping, it is evident that the bias remains sufficiently low. Consequently, the overall loss of our algorithm is minimal, making our publication impactful.

%In \textbf{figure 1.b} and \textbf{1.d}, it is evident that the clipping factor helps lower the variance but at the expense of a higher bias. Specifically, when $X$ is small, there is a noticeable gap between the loss and the variance, which indicates a larger bias. In contrast, with larger values of $X$, the variance decreases. The algorithm without clipping is equivalent to having $X \rightarrow \infty$.





\begin{comment}
\begin{figure}
  \centering \vspace{-0.7cm}
  \subfloat[Varying the number of steps $S$ on the Facebook graph]{\includegraphics[width=4cm]{detection_steps.png}\label{fig:f5}}
  \hfill
  \subfloat[Varying the clipping constant $X$ on the Facebook graph]{\includegraphics[width=4cm]{dectection_clipping.png}\label{fig:f6}}
  \hfill
  \subfloat[Varying the number of steps $S$ on the Wikipedia graph]{\includegraphics[width=4cm]{detection_step_wiki.png}\label{fig:f7}}
  \hfill
  \subfloat[Varying the clipping constant $X$ on the Wikipedia graph]{\includegraphics[width=4cm]{detection_clipping_wiki.png}\label{fig:f8}}
  \captionsetup{justification = centering,font = small}
  \caption{Comparison of the top $k$ nodes determined by Katz centrality and those identified by our estimator} \vspace{-0.5cm}
\end{figure}
\end{comment}



\subsection{Number of walks}

Our assessment included an analysis of the algorithm's proficiency in estimating walk counts. As depicted in Figure \ref{fig:walk}, a notable exponential increase in loss is observed relative to the walk lengths within both networks. This trend is likely attributable to the term \(X^i\) mentioned in Theorem \ref{thm:biaswalk}, which introduces a significant bias. However, it is important to note that since the value of \(X\) remains considerably lower than the maximum degree \(D\), the relative loss of our algorithm remains modest.
A key observation is the better performance of the algorithm when clipping is applied, compared to its absence. In scenarios where \(\alpha S H_{n} /\epsilon > 1\), the variance of the algorithm without clipping escalates exponentially. This sharp increase is specifically seen when \( S \geq 12 \) in the Facebook network and \( S \geq 4 \) in the Wikipedia network.


\begin{figure}[htp] 
  \centering 
  \subfloat[On the Facebook graph]{\includegraphics[width=4cm]{path10_facebook.png}\label{fig:f9}}
  \hfill
  \subfloat[On the Wikipedia graph]{\includegraphics[width=4cm]{path10_wiki.png}\label{fig:f10}}
  \captionsetup{justification = centering,font = small}
  \caption{Loss in estimators of the number of walks}
  \label{fig:walk}
\end{figure}

\section{Conclusion}

 In this study, we developed an estimator to calculate both the number of walks and Katz centrality, leveraging multiple communication rounds and a clipping method. This approach maintains local differential privacy while effectively managing error. We provided an upper bound for the bias and variance associated with certain attenuation factor values, denoted as $\alpha$. Our findings also highlighted that, without the clipping method, the variance of our algorithm can escalate exponentially, even on the simplest graphs. Our experiments further demonstrated that our algorithm performs well in ranking tasks, successfully identifying up to 90\% of the top $k$ nodes with the highest Katz centrality—a key metric in our research.

 \section*{Acknowledgement}

 Vorapong Suppakitpaisarn is partially supported by KAKENHI Grant 23H04377.
Quentin Hillebrand is supported by KAKENHI Grant 20H05965.  A portion of this research was carried out while Louis Betzer was undergoing an internship under the supervision of Prof. Phillipe Codognet. The authors extend their gratitude to Prof. Codognet for hosting the internship. They also wish to express their thanks to the anonymous reviewers whose valuable feedback greatly enhanced the quality of this paper.

% References
\bibliography{uai2024-template}

\newpage

\onecolumn

\title{Publishing Number of Walks and Katz Centrality under \\ Local Differential Privacy (Appendix)}
\maketitle

\section{Example}

We give an example how the Algorithm \ref{alg:LDPKatz} works in this section.
\begin{example}
Assuming the input graph \( G \) is a path graph with length 5, that is \( G = ([5], \{\{1,2\}, \{2,3\}, \{3,4\}, \{4,5\}\}) \), and the attenuation factor \( \alpha \) is 0.1. The number of walks of length 1 from each node, represented as \( P^{(1)}[1], \ldots, P^{(1)}[5] \), are 1, 2, 2, 2, 1 respectively. For walks of length 2, denoted as \( P^{(2)}[1], \ldots, P^{(2)}[5] \), the counts are 2, 3, 4, 3, 2. Similarly, for walks of length 3, indicated by \( P^{(3)}[1], \ldots, P^{(3)}[5] \), the values are 3, 6, 6, 6, 3. Consequently, we have that \( [K^{(1)}[1], \ldots, K^{(1)}[5]] = [0.1, 0.2, 0.2, 0.2, 0.1] \), \( [K^{(2)}[1], \ldots, K^{(2)}[5]] = [0.02, 0.03, 0.04, 0.03, 0.02] \), and \( [K^{(3)}[1], \ldots, K^{(3)}[5]] = [0.003, 0.006, 0.006, 0.006, 0.003] \). When the number of steps $S = 3$, the approximate Katz centrality for the entire graph, denoted as \( Katz \), is \( [0.123, 0.236, 0.246, 0.236, 0.123] \). 

Now, let us assume the privacy budget \( \epsilon = 1 \), the clipping factor \( X = 2 \), and the number of steps \( S = 3 \). Initially, \( \tilde{K}^{(0)}[v] = 1 \) for all vertices \( v \), leading to a noise magnitude of \( \alpha S / \epsilon = 0.3 \) at the first step. Each user \( v \) then independently computes \( \tilde{K}^{(1)}[v] \). Focusing on user 2, at line 9 of the algorithm, they find \( \tilde{K}^{(1)}[2] = 0.2 \). Subsequently, at line 10, the user adds a randomly chosen noise of magnitude \( 0.3 \); suppose this noise is \( 0.7 \). Therefore, the updated value of \( \tilde{K}^{(i)}[v] \) at line 10 becomes \( 0.9 \). This result updates the \( \widetilde{Katz}[2] \) value to \( 0.9 \) at line 11. However, since the value of \( (\alpha X)^i \) at lines 12-13 is \( 0.2 \), the \( \tilde{K}^{(1)}[2] \) value is clipped to \( 0.2 \).

Assuming the values of \( \tilde{K}^{(1)} \) sent to the central server at line 14 are \([-0.2, 0.2, 0.18, 0.2, -0.2]\), the noise magnitude for the second step becomes \( \alpha S / \epsilon \times \max_v |\tilde{K}^{(i)}[v]| = 0.06 \). This information, along with the noise size, is communicated back to all users, including user 2. At line 9, user 2 calculates the initial value of \( \tilde{K}^{(2)}[2] \) as -0.002. Assuming the Laplacian noise generated is -0.01, the value, when added with noise, becomes -0.012, which remains unclipped due to the clipping factor \( (\alpha X)^2 = 0.04 \). Consequently, the value of \( \widetilde{Katz}[2] \) is updated to \( 0.9 - 0.012 = 0.888 \). The progression of Algorithm \ref{alg:LDPKatz} mandates that the user transmit the noisy value of \( \tilde{K}^{(2)}[2] \) to the central server prior to commencing the third iteration of the algorithm. Nonetheless, these specifics will not be elaborated in this manuscript. \label{ex2}
\label{ex1}
\end{example}

\section{Proofs of Theorem \ref{thm:biaswalk} and \ref{thm:bias}}

We skip the proof of Theorem \ref{thm:biaswalk}, as the proof can be obtained from the arguments of Theorem \ref{thm:bias}'s proof. The statement of Theorem \ref{thm:biaswalk} can be deduced by setting the value of \( \alpha \) in the upper bound \( B_i \), as outlined in the proof of Theorem \ref{thm:bias}, to \( 1 \).

Firstly, it is evident that \(0 \leq \mathbb{E}[\widetilde{K}^{(i)}[v]] \leq K^{(i)}[v]\) for all \(i > 0\) and \(v \in [n]\). This implies that both \(\widetilde{K}^{(i)}\) and \(\widetilde{Katz}\) possess a negative bias relative to the actual values. Given that the clipping at line 13 elevates \(\mathbb{E}[\widetilde{K}^{(i)}[v]]\), it reduces the bias (in terms of magnitude). As our objective is to present an upper boundary for this bias, we can disregard the effect of line 13.
 

The bias introduced by the Laplacian mechanism at Line 10 of Algorithm~\ref{alg:LDPKatz} is not easy to analyze. To facilitate analysis, we make a substitution throughout all analyses in this section. Specifically, we replace the Laplacian distribution with an alternative one that simplify our analysis and always yields a greater bias. This substitution allows us to establish an upper bound for the bias from the algorithm.

Recall that we draw the noise from the Laplacian distribution $Lap\left[\frac{ \alpha S}{\epsilon} \max\limits_{v} \tilde{K}^{(i-1)}[v]\right]$ in Algorithm~\ref{alg:LDPKatz}. Let $\mathcal{L}_{i,v}$ be the noise we have drawn. The noise is clipped to $\min\{\mathcal{L}_{i,v}, (\alpha X)^i - \widetilde{K}^{(i)}[v] \}$ at Line 12 of the algorithm. It is straightforward to see that the clipped noise does not introduce more bias than $\min\{\mathcal{L}_{i,v}, 0\}$. Hence, to facilitate the proof in this section, we assume that the alternative noise is obtained by the Laplacian distribution clipped by $0$.
 We know that $\zeta_{i,v} \leq 0$.
With the alternative noise, we obtain the following lemma:

\begin{lemma} For all $i \geq 2$ and $v \in V$, $\max\limits_{v \in V}\widetilde{K}^{(i)}[v] < (\alpha X)^{i}.$ \label{lem:largestK}
\end{lemma}
\begin{proof}
To prove this lemma, we proceed by induction on the number of step $i$. 

Define $M_{i} := \max\limits_{v \in V}\widetilde{K}^{(i)}[v]$ and 
$m_{i} = \max\limits_{v \in V : deg(v) \leq d}\widetilde{K}^{(i)}[v]$.
After the calculation at line 12 of Algorithm~\ref{alg:LDPKatz}, we have $M_{1} \leq \alpha X$ and $m_{1} \leq \alpha d$. Therefore, for all $v \in V$,
\[\begin{split}
\widetilde{K}^{(2)}[v] & = \sum_{u \in \eta(v)} \alpha \widetilde{K}^{(1)}[u] + \zeta_{i,v} \\
& \leq \sum_{u \in \eta(v) \mid deg(u) > d} \alpha \widetilde{K}^{(1)}[u] + 
\sum_{u \in \eta(v) \mid deg(u) \leq d} \alpha \widetilde{K}^{(1)}[u] \\
& \leq \alpha N M_{1} + \alpha D m_{1} \\ & \leq \alpha^{2} (N X + D d) \\ & \leq (\alpha X)^{2}
\end{split}\]
This proves that $M_{2} = \max\limits_{v \in V}\widetilde{K}^{2}[v] \leq (\alpha X)^{2}$. 

Then, for all $i > 1$ and $v \in \{u \in [n]: deg(u) \leq d\}$, we have that 
$$\widetilde{K}^{(i)}[v] = \sum_{u \in \eta(v)} \alpha \widetilde{K}^{(i-1)}[u] + \zeta_{i,v} \leq \alpha d M_{i-1}.$$ 
This means that $m_{i} \leq \alpha d M_{i-1}$. 

To show the induction step,  we assume that $\widetilde{K}^{(j)}[v] < \alpha^j X^j$ for all $j < i$ and $v \in [n]$. By the assumption, for all $i > 2$ and $v \in [v]$: 
\[\begin{split}
\widetilde{K}^{(i)}[v] & = \sum_{u \in \eta(v)} \alpha \widetilde{K}^{(i-1)}[u] + \zeta_{i,v} \\
& \leq \sum_{u \in \eta(v) \mid deg(u) > d} \alpha \widetilde{K}^{(i-1)}[u] + 
\sum_{u \in \eta(v) \mid deg(u) \leq d} \alpha \widetilde{K}^{(i-1)}[u] \\
& \leq \alpha N M_{i-1} + \alpha D m_{i-1} \\ & \leq \alpha N M_{i-1} + \alpha^{2} d D M_{i-2} \\
& \leq \alpha N (\alpha X)^{i-1} + \alpha^{2} d D (\alpha X)^{i - 2} \\ & \leq \alpha^iX^{i - 2} (NX + d D ) \\ & \leq (\alpha X)^{i}
\end{split}\]
\end{proof}

The previous lemma implies that, by the alternative noise used in this section, the calculation at Line 12 changes the results only at the first step. We are now ready to prove our main theorem.

\begin{proof}[Proof of Theorem \ref{thm:bias}]
The expected value of the alternative noise is $\mathbb{E}[\zeta_{i,v}] \geq - \alpha^{i} X^{i-1} S/\epsilon$. We obtain that, for $i > 1$, $$\mathbb{E}[{K}^{(i)}[v] - \widetilde{K}^{(i)}[v]] \leq 
\sum_{u \in \eta(v)} \alpha \mathbb{E}[{K}^{(i-1)}[u] - \widetilde{K}^{(i-1)}[u]] + 
\frac{\alpha^{i} X^{i-1} S}{\epsilon}.$$
Let $b_{i} = \max\limits_{v \in V \mid deg(v) \leq d} \mathbb{E}[{K}^{(i)}[v] - \widetilde{K}^{(i)}[v]]$
and  $B_{i} = \max\limits_{v \in V} \mathbb{E}[{K}^{(i)}[v] - \widetilde{K}^{(i)}[v]]$. By using the similar argument as in the proof of Lemma \ref{lem:largestK}, we can obtain that, for all $i > 1$, 
$$B_{i} \leq \alpha (N B_{i-1} + D b_{i-1}) + \frac{\alpha^{i} X^{i-1} S}{\epsilon}$$ and, for all $i > 2$,
$$b_{i - 1} \leq \alpha d B_{i-2} + \alpha^{i-1} X^{i-2} S/\epsilon.$$ Combining the above two equations, we obtain that, for all $i > 2$, 
$$B_{i} \leq \alpha N B_{i-1} + \alpha^{2} d D B_{i-2} + 
\alpha^{i} (X^{i-1} + D X^{i-2}) \frac{S}{\epsilon}.$$
We will now prove by induction that $B_{i} \leq 2 \alpha^{i} (\phi X)^{i-1} D S/\epsilon$. By the assumption that $S/\epsilon \geq 1$, we have  $$B_{1} \leq \alpha (D-X) + \alpha S/\epsilon
\leq 2 \alpha D S/\epsilon.$$
Recall the assumption that $NX + Dd \leq X^2$ and $X \leq D$. The assumptions imply that $N \leq X$ and $d \leq X$. It follows that $b_1 \leq \alpha S / \epsilon$. Hence, by $\alpha < 1/\phi X$:
\begin{dmath*}B_{2} \leq \alpha (N B_{1} + (D-N) b_{1}) + \alpha^{2} X \frac{S}{\epsilon} \leq \alpha^2 N(D - X) + \alpha^2 D \frac{S}{\epsilon} + \alpha^2 X \frac{S}{\epsilon}  \leq \alpha^{2} N (D-X) + \alpha^{2} (D+X) S/\epsilon. \end{dmath*} 
Recall that $N \leq X$ and $1 \leq S/\epsilon$. We obtain that $N(D-X) \leq ND \leq XDS/\epsilon$. Also, recall the assumption that $X^2/D + X \leq X^2$. We obtain that $X + D \leq XD$. Hence, 
$$B_2 \leq \alpha^{2} X D S/\epsilon + \alpha^{2} X D S/\epsilon \leq 2 \alpha^{2} \phi X D S/\epsilon.$$

We will now consider the case when $i \geq 3$. Assume by induction that, for all $k < i$, $B_{k} \leq 2\alpha^{k} (\phi X)^{k-1} D S/\epsilon$, then 
\[
\begin{split}
B_{i} & \leq \alpha N B_{i-1} + \alpha^{2} d D B_{i-2} +  \alpha^{i} (X^{i-1} + D X^{i-2}) S/\epsilon \\
& \leq \frac{2 \alpha^{i} S D (\phi X)^{i-3}}{\epsilon} \left[ N \phi X + d D + \frac{(X^{2}+D X)}{D \phi^{i-2}}\right] \\
& \leq \frac{2 \alpha^{i} S D (\phi X)^{i-3}}{\epsilon}( N (\phi-1) X + 
(N X + d D) + (X^{2}/D + X)) \\
& \leq \frac{2 \alpha^{i} S D (\phi X)^{i-3}}{\epsilon}( (\phi-1) X^2 + 
X^2 + X^2) \\
& \leq \frac{2 \alpha^{i} S D \phi^{i-3} X^{i-1}}{\epsilon}( \phi + 1) \\ & = 
\frac{2 \alpha^{i} S D (\phi X)^{i-1}}{\epsilon}.
\end{split}
\]
This concludes the induction. 

The discrepancy in the Katz centrality estimated by our algorithm is from three components:
\begin{enumerate}
\item The bias derived from the initial step, which does not exceed $\alpha S/\epsilon$;
\item The bias from the second up to the $S$-th step, which is not larger than the sum $\sum\limits_{i = 2}^S B_i$; and
\item The bias resulting from the limitation that our computation does not extend past the $S$-th calculation step.
\end{enumerate}
Therefore, 
\[
\begin{split}
\max_{v \in V} \mathbb{E}[Katz(G)[v] - \widetilde{Katz}(G)[v]] & \leq \frac{\alpha S}{\epsilon} + 
\sum_{i=2}^{S} B_{i}  + \sum_{i= S + 1}^{\infty} K^{(i)}[v] \\
& \leq \frac{\alpha S}{\epsilon} +
\sum_{i=2}^{S} \frac{2 \alpha D S (\alpha \phi X)^{i-1}}{\epsilon}
+ \sum_{i=S + 1}^{\infty} (\alpha X)^{i}
\\
& \leq \frac{\alpha S}{\epsilon} + \frac{2 \alpha^{2} D S \phi X}{\epsilon} \sum_{i=0}^{S-2} (\alpha \phi X)^{i} + \frac{(\alpha X)^{S+1}}{1-\alpha X} \\
& \leq \frac{\alpha S}{\epsilon} + 
\frac{2 \alpha^{2} D S \phi X(1 - (\alpha \phi X)^{S-1})}{\epsilon(1- \alpha \phi X)} + 
\frac{(\alpha X)^{S+1}}{1-\alpha X} \\
& \leq \frac{\alpha S}{\epsilon}(1 + \frac{2 \alpha \phi D X}{1- \alpha \phi X}) -
\frac{2 \alpha D S (\alpha \phi X)^{S}}{\epsilon (1 - \alpha \phi X)} +
\frac{(\alpha X)^{S+1}}{1-\alpha X}.
\end{split}
\]
By $2S/\epsilon > 1$, $X \leq D$, and $\phi > 1$, we obtain the theorem statement from the previous derivation because 
$$\frac{2 \alpha D S (\alpha \phi X)^{S}}{\epsilon (1 - \alpha \phi X)} \geq \frac{(\alpha X)^{S+1}}{1-\alpha X}.$$
\end{proof}

\section{Proofs of Theorems \ref{thm:walkvariance} and \ref{thm:katzvariance}}

This section gives only the proof of Theorem \ref{thm:katzvariance}. We can show Theorem \ref{thm:walkvariance} by the upper bound of $\mathcal{V}_i$ provided in the proof of Theorem \ref{thm:katzvariance}. 

\begin{proof}[Proof of Theorem \ref{thm:katzvariance}]
First, let us examine the variance of the number of walks, $\tilde{K}^{(i)}[v]$. It is clear that the computation at Lines~12-13 in Algorithm~\ref{alg:LDPKatz} can only decrease the variance, so we can disregard this step when establishing the upper bound for the variance. Consequently, the upper bound for $Var[\tilde{K}^{(i)}[v]]$ is made up of two components:
\begin{enumerate}
    \item The variance of the Laplacian noise added at Line 10: By Lemma \ref{lem:largestK}, we have that $$\pi \leq \frac{\alpha S}{\epsilon} \cdot \max\limits_{v \in V}  \widetilde{K}^{(i-1)}[v] < \frac{\alpha^{i}X^{i-1} S}{\epsilon}.$$ Hence, the variance of the Laplacian noise at Line 10 is not larger than $2 \cdot \pi^2 = 4 \alpha^{2 i} X^{2 i -2} S^{2}/\epsilon^{2}$.
    \item The collective sum of covariances between the variable $\alpha \cdot \widetilde{K}^{(i-1)}[u]$ and $\alpha \cdot \widetilde{K}^{(i-1)}[w]$ for every $u,w$ within $\eta(v)$: Let  $\mathcal{V}_i = \max\limits_{v\in V}\mathrm{Var}[\widetilde{K}^{(i)}[v]]$ and 
$\nu_i = \max\limits_{v\in V : deg(v) \leq d}\mathrm{Var}[\widetilde{K}^{(i)}[v]]$. By the Cauchy-Schwartz inequality in \cite{cauchy1821cours}, we obtain that
\begin{equation*}
    \mathrm{Cov}[\widetilde{K}^{(i)}[u], \widetilde{K}^{(i)}[w]] \leq 
    \begin{cases} 
      \nu_i & \text{if } deg(u), deg(w) \leq d, \\
      \sqrt{\nu_i \cdot \mathcal{V}_i} & \text{if } \min\{deg(u),deg(w)\} \leq d, \\ 
      \mathcal{V}_i & \text{Otherwise.}
   \end{cases}
\end{equation*}
Given that the maximum number of nodes in $\eta(v)$ is $D$, and among these $D$ nodes, at most $N$ nodes have a degree exceeding $d$:
\begin{dmath*}
    \sum_{u,w \in \eta(v)} \mathrm{Cov}[\alpha \cdot \widetilde{K}^{(i)}[u], \alpha \cdot \widetilde{K}^{(i)}[w]] 
    \leq \alpha^{2} (N^{2} \mathcal{V}_{i-1} + 
2 N D \sqrt{\nu_{i-1}\mathcal{V}_{i-1}} + D^{2} \nu_{i-1}).
\end{dmath*}
\end{enumerate}
There is no need to account for the covariance between $\widetilde{K}^{(i)}[u]$ and the Laplacian noise, since the noise is generated independently of the value of $\widetilde{K}^{(i)}[u]$.

 Hence, for all $i > 1$,
$$\mathcal{V}_i \leq \frac{4 \alpha^{2 i} X^{2 i -2} S^{2}}{\epsilon^{2}} + \alpha^{2} \left(N^{2} \mathcal{V}_{i-1} + 
2 N D \sqrt{\nu_{i-1}\mathcal{V}_{i-1}} + D^{2} \nu_{i-1}\right),$$
and, for all $i > 2$,
    $$\nu_{i-1} \leq \frac{4 \alpha^{2 (i-1)} X^{2 i -4} S^{2}}{\epsilon^{2}} + \alpha^{2} d^{2} \mathcal{V}_{i-2}.$$

By combining the above inequalities and by $\nu_i \leq \mathcal{V}_i$, we have that, for all $i >2$, 
\[
\mathcal{V}_{i} \leq \alpha^{2}(N^{2} + 2 N D) \mathcal{V}_{i-1} + \alpha^{4} d^{2} D^{2} \mathcal{V}_{i-2} 
+ \frac{4 \alpha^{2 i} X^{2 i - 4} S^{2}}{\epsilon^{2}}(D^{2} + X^{2}).
\]
By defining $L = \max\{X^{2},ND\}$, we will now prove by induction that, for all $i \geq 1$,
$\mathcal{V}_{i} \leq \frac{16 S^{2} \alpha^{2 i}}{\epsilon^{2}} (D^{2} + X^{2}) (4 L)^{i-2}$.

First, because the variance at the first step $\mathcal{V}_1$ comes only from the Laplacian noise, we have that 
$$\mathcal{V}_{1} = \frac{4 \alpha^{2} S^{2}}{\epsilon^{2}} \leq \frac{16 \alpha^{2} S^{2} (D^{2}+ X^{2})}{\epsilon^2 (4L)}.$$
By that, the covariance sum at the second step is no more than $\alpha^2 D^2 \mathcal{V}_1 = 4\alpha^4S^2D^2/\epsilon^2$. Because the variance of the Laplacian noise is $4\alpha^4X^2S^2 / \epsilon^2$, we have that 
$$\mathcal{V}_{2} = \frac{4 \alpha^{4} S^{2} (D^{2} + X^{2})}{\epsilon^{2}} < \frac{16 \alpha^{4} S^{2} (D^{2} + X^{2})}{\epsilon^{2}}.$$
For $i > 2$, we assume that, for all $k < i$,
$\mathcal{V}_{k} \leq \frac{16 S^{2} \alpha^{2 k}}{\epsilon^2} (D^{2} + X^{2}) (4 L)^{k-2}$, then, by $N^2 \leq X^2 \leq L$, $dD \leq X^2 \leq L$, and $ND \leq L$: 
\[
\begin{split}
\mathcal{V}_{i} & \leq \frac{16 S^{2} \alpha^{2 i} (4L)^{i-4} (D^{2} + X^{2})}{\epsilon^{2}}\left(4 L (N^{2} + 2 N D) + d^{2} D^{2} +X^{4}/4^{i-3}\right) \\
& \leq \frac{16 S^{2} \alpha^{2 i} (4L)^{i-4}(D^{2} + X^{2})}{\epsilon^{2}} \left(12 L^{2} + L^2 + L^{2}\right) \\
& \leq \frac{16 S^{2} \alpha^{2 i}}{\epsilon^{2}} (D^{2} + X^{2}) (4 L)^{i-2}.
\end{split}
\]
Finally, considering that \( \widetilde{Katz}(G)[v] = \sum_{i=1}^{S} K^{i}[v] \), and leveraging the Cauchy-Schwartz inequality, combined with the understanding that \( \sum_{i=0}^{\infty} \sum_{j=0}^{\infty} x^{i+j} = \frac{1}{(1-x)^2} \) for all \( x \) in \( \mathbb{R} \), we deduce the following:
\[
\begin{split}
\mathrm{Var}[\widetilde{Katz}[v]] & = \sum_{i=1}^{S} \sum_{j=1}^{S} \mathrm{Cov}[K^{(i)}[v],K^{(j)}[v]]\\
& \leq \sum_{i=1}^{S} \sum_{j=1}^{S} \sqrt{\mathrm{Var}[K^{(i)}[v]]\mathrm{Var}[K^{(i)}[v]]} \\
& \leq \frac{16 S^{2} \alpha^{2}}{4 L \epsilon^{2}} (D^{2} + X^{2}) 
\sum_{i=0}^{S-1} \sum_{j=0}^{S-1} (2 \alpha \sqrt{L})^{i} (2 \alpha \sqrt{L})^{j} \\
& \leq  \frac{16 S^{2} \alpha^{2}}{4 L \epsilon^{2}} (D^{2} + X^{2}) 
\sum_{i=0}^{\infty} \sum_{j=0}^{\infty} (2 \alpha \sqrt{L})^{i} (2 \alpha \sqrt{L})^{j} \\
& \leq \frac{4 S^{2} \alpha^{2}(D^{2} + X^{2})}{L \epsilon^{2} (1 - 2 \alpha \sqrt{L})^{2}} 
\end{split}
\]
\end{proof}


\section{Variance of the Algorithm without Clipping}

We will now proceed to evaluate the algorithm without clipping, essentially examining Algorithm \ref{alg:LDPKatz} while omitting lines 12 and 13. Given that the bias of this algorithm converges to 0 as the number of steps \( S \) approaches infinity, our analysis will predominantly concentrate on its variance. We will explore its variance over the graph \( G_0 = ([n],\emptyset) \) — a representation with \( n \) nodes and devoid of edges. Our theorem indicates that, even with this simple graph structure, the variance amplifies at such a rate that the utility of the publication becomes questionable.


Let \(N_{i}\) represent the scale of the noise at the \(i^{th}\) step, as seen in line 6 of Algorithm~\ref{alg:LDPKatz}. The underlying principle here is that when drawing \(n\) Laplacian noises of scale \(N_{i}\), it is highly probable that one of them will be considerably large, causing \(N_{i+1}\) to also be large. If we consider \((L_{i}(v))_{v \in [n]}\) as the \(n\) Laplacian noises drawn at the \(i^{th}\) step, each with a scale of \(N_{i}\), and given that \(G_0\) lacks any edges, we can deduce that \(K^{(i)}[v] = L_{i}(v)\). This leads to the expression 
$N_{i+1} = \frac{\alpha S}{\epsilon} \max\limits_{v \in [n]} |L_{i}(v)|$.
We will employ the subsequent lemma for further analysis:
 

\begin{lemma} [\cite{maxexpo}]
Let $n>0, \delta>0 $ and $(L_{v})_{v \in [n]}$ be n independent Laplacian noise with scale $\delta$. Then $\mathbb{E}[ \max\limits_{v \in [n]} |L_{v}|] = \delta H_{n}$ where $H_{n} = \sum_{i = 1}^{n} 1/i$ is the Harmonic series with n terms. \label{maxLap}
\end{lemma}

Given the algorithm without clipping applied to graph \(G_0\), we can now determine the expected noise at step \(i\).


\begin{theorem}
Considering the algorithm without clipping, and given parameters \(G_0\), \(\alpha\), \(S\), and \(\epsilon\). Define \(H_{n} = \sum_{i = 1}^{n} 1/i\) as the Harmonic series with \(n\) terms, where \(n\) represents the total number of nodes. The anticipated noise for step $i$ is expressed as $(\alpha S/\epsilon)^{S} H_{n}^{S-1}$. \label{thm:noclip}
\end{theorem}

\begin{proof}
We proceed with the proof of the theorem using induction. For the base case, consider step 1. Given that \(\widetilde{K}^{(0)}[v] = 1\) for every \(v \in [n]\), it follows that \(\mathbb{E}[N_{1}] = \alpha S/\epsilon\).

Now, let us assume for some arbitrary step \(i>0\) that \(\mathbb{E}[N_{i}] = (\alpha S/\epsilon)^{i} H_{n}^{i-1}\). Given that \(N_{i+1} = \frac{\alpha S}{\epsilon} \max\limits_{v \in [n]} |L_{i}(v)|\) where each \(L_{i}(v)\) represents an independent Laplacian noise with scale \(N_{i}\), and by employing Lemma \ref{maxLap}, we can express \(\mathbb{E}[N_{i+1}|N_{i}]\) as \(\frac{\alpha S}{\epsilon} H_{n} N_{i}\). Consequently, we have 
$$\mathbb{E}[N_{i+1}] = \mathbb{E}[\mathbb{E}[N_{i+1}|N_{i}]] = \frac{\alpha S}{\epsilon} H_{n} \mathbb{E}[N_{i}] = ( \alpha S/\epsilon)^{i+1} H_{n}^{i}.$$
This establishes the induction hypothesis, completing the proof.
\end{proof}

\label{sec:withoutClip}


\end{document}
