

\begin{figure*}[t!]
  \centering
  \includegraphics[width=0.95\textwidth]{figures/nc.pdf}
  \caption{
  Illustration of \emph{representation reliability} ($\mathsf{Reli}$) and \emph{neighborhood consistency} ($\mathsf{NC}$). For a test point $\bx^*$ and a class of pre-trained backbone models $\mathcal{H} = \{h_1, \cdots, h_M\}$, the representation reliability is defined as the average performance of downstream models when using the representations of $\bx^*$ provided by the backbones in $\mathcal{H}$.
  \textbf{Our $\mathsf{NC}$ estimates $\mathsf{Reli}$ without requiring any prior knowledge of the downstream tasks.} It operates by measuring the number of consistent neighbors of $\bx^*$ among reference points across different representation spaces.}
  \label{fig:repr_reli}
\end{figure*}


\section{Quantifying Representation Reliability}




\hao{In this section, we introduce a framework for assessing the reliability of representations assigned by pre-trained models. We discuss limitations of existing UQ frameworks in supervised learning when applied to representation spaces. 
Then we present an ensemble-based method that examines the consistency of neighboring points in the representation space.
Our method effectively captures the representation reliability without the need for a priori knowledge of downstream tasks. 
}












\subsection{Representation Reliability} \label{sec:defin}

We introduce a formal definition of \emph{representation reliability} by examining its impact on various downstream tasks. Intuitively, a reliable representation assigned by a pre-trained model should consistently yield accurate outcomes when the model is adapted to these downstream tasks.


We introduce some notations that will be used in our definition. We define $\mathcal{H}$ as a class of embedding functions trained by a self-supervised learning algorithm (e.g., SimCLR \citep{chen2020simple}) using a pre-trained dataset (e.g., ImageNet \citep{deng2009imagenet}). Each embedding function $h:\mathcal{X} \to \mathcal{Z}$ maps a data point (e.g., an image) to an abstract representation. Here the representation space $\mathcal{Z}$ can be either a real $d$-space $\Reals^d$ or a unit hyper-sphere $\mathcal{S}^{d-1}$. 



We consider a collection of downstream tasks (e.g., classifications), denoted as $\mathcal{T}$. Each task is associated with a set of downstream heads (i.e., additional layers added on top of the embedding function) and a population risk function that assesses the performance of each downstream head. For each task $t$, we optimize the risk function to obtain an optimal head $g_{h,t}$. This way, we can eliminate the impact of downstream training processes on our definition. The representation reliability of a new test point $\bx^*$ is measured by the (average) performance of these downstream models on $\bx^*$. A formal definition is provided below. 










\begin{defn}
\label{defn::rep^rel}
Let $\mathcal{T}$ be a collection of downstream tasks. For each task $t \in \mathcal{T}$, we take an embedding function $h$ (uniformly at random) from $\mathcal{H}$ and find an optimal downstream head $g_{h,t}$ based on $h$. We define the \emph{representation reliability} for a test point $\bx^* \in \cX$ as:
\begin{equation} \label{eq:rep_reli}
    \mathsf{Reli}(\bx^*; \cH, \mathcal{T}) \defined \frac{1}{|\cT|} \sum_{t \in \cT}{\myPerf{t}}
\end{equation}
where $\Perf{\cdot}$ measures the performance of the predictive model $g_{h,t} \circ h$ on $\bx^*$ for task $t$. %
\end{defn}
For classification downstream tasks with $C$ classes, an example of $\myPerf{t}$ is the negative Brier score, defined as:
\begin{align*}
-\sum_{c=1}^{C} \Big( \frac{1}{|\cH|}\sum_{h \in \mathcal{H}}{g_{h,t} \circ h(\bx^*)}_{[c]} - y_{t[c]}^* \Big)^2.
\end{align*}
Here $y_t^* \in \{0,1\}^C$ represents the label of $\bx^*$ on task $t$. For additional examples of $\Perf{\cdot}$, please refer to Appendix~\ref{app:performance}.


 
The above definition assumes that the set of downstream tasks (and ground-truth labels) are accessible. 
In practice, this may not always be the case (see Broad Impact and Implication in Section~\ref{sec::intro} for examples). Next, we discuss how to estimate the representation reliability based on the properties of the representation itself, without prior knowledge of the downstream tasks.




\subsection{First Attempt: Representation Consistency} 
\label{sec:consistency}









Our first attempt is directly applying standard supervised-learning techniques (see Section~\ref{sec:bg}) to estimate the representation reliability. Recall that if multiple predictive models give different predictions for the same test point, then it is likely that their predictions are uncertain. One may wonder whether the same idea can be applied to estimate the representation reliability. We present a negative result, showing that even if different embedding functions produce completely different representations, their downstream predictions can still be consistent. Below, we provide an (informal) theorem and defer a more rigorous statement along with its proof to Appendix~\ref{append::sl_count_ex}.


\begin{thm}
\label{thm::sl_count_exam_inf}
For any constant $A$ and a test point $\bx^*$, there exist embedding functions $h_1,\cdots,h_{M} \in \mathcal{H}$ such that $\Varr{i\sim [M]}{h_i(\bx^*)} \geq A$ but $\Varr{{i\sim[M]}}{g_{i,t} \circ h_i(\bx^*)} = 0$ for any downstream task $t$. Here $g_{i,t}$ is an optimal downstream head for $h_i$ under task $t$. 
\end{thm}







The key insight behind our proof is that an input point's representation is not unique (e.g., rotated spaces are in fact equivalent). In other words, even if different embedding functions assign distinct representations to the same test point, downstream heads built on these embedding functions can also vary, ultimately leading to similar predictions.

\subsection{Proposed Framework: Neighborhood Consistency} \label{sec:nb_consistency}

To address the aforementioned issue, we propose the idea of using an ``anchor'' point to align different representation spaces. The anchor point serves as a bridge that transforms different representation spaces into the same space. The graphical visualization of this idea is illustrated in Figure~\ref{fig:repr_reli} and Figure~\ref{fig:sketch_proof}. We formalize this intuition more rigorously in the following theorem. It states that if a test point has a (reliable) consistent neighboring point across all representation spaces, then uncertainty of its downstream predictions are bounded above.

\begin{thm}
\label{thm::nb_consistency}
For a test point $\bx^*$, suppose that there exists a \textbf{consistent} neighbor $\bx^r$ across all embedding functions $\mathcal{H} = \{h_1, \cdots, h_M\}$, satisfying
\begin{equation} \label{eq:nb}
    \lVert h_i(\bx^r) - h_i(\bx^*)) \rVert_2 \le \epsilon_{nb} ~, ~ \forall i \in [M] . 
\end{equation}
Suppose the downstream heads $g_{i,t}$ are Lipschitz continuous. Then, for any downstream task $t$, the variance of downstream prediction at $\bx^*$ is bounded above by:
\begin{equation} \label{eq:ens_var}
\Varr{i\sim[M]}{g_{i,t} \circ h_i(\bx^*)} \le (\sqrt{2} L_t \epsilon_{nb} + \sigma_{r,t})^2.
\end{equation}
where $\sigma_{r,t}^2=\Varr{i\sim[M]}{g_{i,t} \circ h_i(\bx^r)}$ is the reliability of $\bx^r$ as measured by variance, the function $g_{i,t}$ is the optimal downstream head for task $t$ built upon $h_i$, whose Lipschitz constant is $L_{i, t}$, and $L_t = \max_i L_{i, t}$.\footnote{When spectral normalized weights are employed with 1-Lipschitz continuous activation functions (e.g., identity, ReLU, sigmoid, or softmax), the constant $L_t = 1$.}


\end{thm}


See Appendix~\ref{pf::nb_consistency} for proof. The key takeaway is that if we can find a reference point consistently close to the test point, it can serve as an anchor point that helps align representation spaces with distinct semantic meanings. 
Plus, the reliability of the reference point along with its relative distance to the test point ensures a lower bound on the representation reliability of the test point.

In practice, identifying a reliable reference point is challenging without prior knowledge of the downstream tasks. Instead of searching for a single point as the anchor, we draw a set of reference points, denoted as $\bX_{\text{ref}} = \{\bx^{(l)}\}_{l=1}^n$. We then compute the number of consistent neighboring points within $\bX_{\text{ref}}$ and use it to estimate the representation reliability. The rationale behind this is that a test point with more consistent neighbors is more likely to have a reliable and consistent neighbor.


  





\paragraph{Our Algorithm.} Given an ensemble of embedding functions $h_1,\cdots, h_M$ and $\bX_{\text{ref}}$, we define the \textbf{Neighborhood Consistency (NC)} of a test point $\bx^*$ as:
\begin{equation} \label{defn::NC}
    \mathsf{NC}_{k}(\bx^*) = \frac{1}{M^2} \sum_{i < j} \Sim{k\text{-NN}_i \big( \bx^* \big), ~ k\text{-NN}_j \big( \bx^*\big) }
\end{equation}
where $k\text{-NN}_i(\bx^*)$ is the index set of $k$-nearest neighbors of $h_i(\bx^*)$ among $\{h_i(\bx) \mid \bx \in \bX_{\text{ref}}\}$ 
and $\Sim{\cdot, \cdot}$ is a measure of similarity between sets (e.g., \emph{Jaccard Similarity}).



\begin{rem}
\rev{The parameter $k$ involves a trade-off between two factors: choosing a neighborhood closer to the test point and increasing the chance of incorporating a more reliable neighbor. As shown in Theorem~\ref{thm::nb_consistency}, the reliability of a test point's representation hinges on whether it has a nearby reliable neighbor. Hence, a large $k$ may result in selecting a reliable neighbor, but it could be far from the test point.
On the other hand, since the upper bound in Theorem~\ref{thm::nb_consistency} holds using any one of the neighbors as the anchor point, the reliability of the test point is bounded using the neighbor with the smallest $\sigma_{r,t}$. 
Consequently, a small $k$ may compromise the reliability of the selected neighboring point.
We present an empirical study about this trade-off in Section~\ref{sec:ablation}.}
\end{rem}


\rev{Finally, our NC requires a set of embedding functions for computing consistent neighbors. We extend this algorithm to evaluate the reliability of a single embedding function in Section~\ref{sec:individual}.}
















