


\section{Introduction}
Computer-aided medical imaging systems have achieved significant progress in recent years,
with a substantial part of this progress made possible by the advancements of deep learning models 
\cite{rajpurkar2018deep,hannun2019cardiologist,rajpurkar2017chexnet,guan2021domain}. However, a major limitation for their adoption in clinical environments is given by their restricted generalization capacity across unseen data distributions~\cite{zhou2022domain}.
The reason for this is distribution shifts that can, for example, be caused by variations in scanner types or imaging protocols~\cite{roschewitz2023automatic}.
To address this, \textit{test-time adaptation and generalization} arose as methods to optimize a trained source model on new incoming target data. Unlike \textit{domain adaptation and generalization} techniques, test-time generalization can consecutively optimize the model on unlabelled data during the test phase without the requirement to access source data, fostering privacy-preserving adaptation to the target domain. Additionally, it allows the model to be optimized continuously without interrupting the inference process, proving especially beneficial in time-sensitive applications where maintaining a flow of real-time decision-making is imperative.
Furthermore, the capability of test-time generalization to process data in batches is reflective of real-world scenarios where medical data is also available serially.
This aspect enhances its applicability in dynamic clinical workflows.

As shown in Figure~\ref{fig1:intro}(a), test-time generalization \cite{wang2018deep,sun2020test,liu2021ttt++,liang2020we,niu2022efficient,huang2023fourier,ambekar2023learning} methods focus on fine-tuning of the source model based on source model predictions, surrogate models or task predictions. This optimization often involves computation of gradients with norm-based losses followed by finetuning of batch norm layers \cite{wang2021tent}, all the parameters of the model \cite{liang2020we} or a linear classification layer \cite{wang2021tent}. A more recent approach \cite{jang2022test} utilizes parameterized ensembles with backpropagation to optimize the last layers of the source model. Even though it is possible to only fine-tune the batch norm parameters \cite{wang2021tent}, gradient-based fine-tuning of model weights, in general, is resource and time-intensive. This increases computational costs and leads to slower adaptation processes, making such methods less practical for real-time applications such as dynamic contrast-enhanced imaging, real-time tumor classification, and rapid stroke identification. Gradient-based finetuning \cite{wang2021tent,niu2022efficient,jang2022test,liang2020we} often relies on maximum a posteriori estimation to obtain the target model, which can lead to overfitting when adapting to target data that involves multiple distribution shifts. This reduces the model's generalizability, making it less robust to diverse shifts. 
Moreover, the potential to converge to local minima and a susceptibility to hyperparameter selections limits these methods efficacy. 







\begin{figure}[t]
\centering

\includegraphics[width=0.6\linewidth]{figures/TNN_Fig2.pdf}
\caption{{\textbf{Data and model interaction scheme.} (a) Common test-time generalization techniques utilize norm-based losses for gradient-based finetuning of batch norm layers ($\beta_{t}$ and $\gamma_{t}$), full model ($\theta_{t}$), or linear layers ($\Phi_{t}$) to obtain target predictions $\y_{t}$. {These methods feature memory and compute constraints and require precise hyperparameter selection with several rounds of backpropagation. }(b) Non-parametric approaches such as T3A~\cite{iwasawa2021test} and ours, obtain $\y_{t}$ via techniques that operate on frozen source model predictions $\bar{x}_{t}$ in a feed-forward manner. This neglects the need for additional computational resources and simplifies the generalization process.}}

\label{fig1:intro}

\end{figure}






By the nature of their feedforward design, non-parametric methods bypass overfitting and negate the need for loss-based gradient finetuning, therefore offering clear advantages, Fig.~\ref{fig1:intro}(b).
T3A \cite{iwasawa2021test} {computes class representations or prototypes based on the source model's weights} and adjusts the classifier utilizing an entropy threshold.
{However, by relying only on the {entropy of samples}, information from the target domain is not fully utilized in the method. 
A more nuanced approach would be given by application of the source model to directly identify target samples with analogous characteristics to the source features.}
Such a method has the power to increase the utilization of target information and to align closely with the intrinsic data distribution. 

Building on these insights, our work introduces a novel, non-parametric method coined Test-time Non-parametric Neighbors (TNN). We leverage neighborhood information between the source prototypes and target data without the need for finetuning. In summary, our contributions are:
\begin{itemize}
     \item We propose utilizing target neighborhood information with dynamic voting to adjust source-trained classifiers in a non-parametric manner for test-time generalization. 
     \item {Our proposed method (TNN) is simple and does not modify the source training process.} Yet, it is effective across datasets and requires minimal computation at test time due to its feedforward nature.   
    \item We adopt several state-of-the-art test-time generalization techniques for medical imaging and perform exhaustive comparisons to our approach.   
\end{itemize}

\noindent Through comprehensive experiments and ablation studies, we demonstrate the efficacy and potential of TNN in medical imaging contexts, an area where such non-parametric approaches have been underexplored. 


