\section{Additional details for the WSI Datasets}
Table~\ref{tab:supp_data} presents in more detail the description of the different datasets used in our study. Details about the number of samples and classes are included, together with a small description of each of the digital pathology whole slide image data. More details can be identified in the original papers.
\input{tabs/appendix_data_310}


\section{Implementation Details about Attention MIL}


We provide additional details here on the attention mechanism used in 
Attention-Based Multiple Instance Learning ~\cite{ilse2018attentionbaseddeepmultipleinstance} used in our work. 
A bag $B = \{x_i\}_{i=1}^{N}$ consists of $N$ instances, each of which is first transformed into a 
low-dimensional embedding:

\begin{equation}
\label{eq:inst_embed_sup_supp}
\mathbf{z}_i = f(x_i), 
\qquad i = 1,\dots,N.
\end{equation}

These embeddings are aggregated using an attention operator to obtain a bag-level representation:

\begin{equation}
\label{eq:bag_agg_sup}
\hat{\mathbf{Z}} = \sum_{i=1}^{N} a_i \mathbf{z}_i .
\end{equation}

The bag prediction is then obtained by applying a classifier to the aggregated representation:
\begin{equation}
\label{eq:bag_pred_sup}
\hat{Y} = \varphi(\hat{\mathbf{Z}}).
\end{equation}

We adopt a gated attention (GA) mechanism ~\cite{gated_att_dauphin2017language} to produce more expressive attention scores. 
The unnormalized attention score for instance $i$ is computed as:

\begin{equation}
\label{eq:unnorm_att_sup}
u_i = 
\mathbf{w}^\top
\left(
\tanh(\mathbf{V}_1 \mathbf{z}_i)
\odot
\sigma(\mathbf{V}_2 \mathbf{z}_i)
\right),
\end{equation}

where
\begin{itemize}
    \item $\mathbf{V}_1, \mathbf{V}_2 \in \mathbb{R}^{L* M}$ are learnable projection matrices,
    \item $\mathbf{w} \in \mathbb{R}^{L}$ is a learnable attention vector,
    \item $\odot$ denotes element-wise multiplication,
    \item $\sigma(\cdot)$ denotes the sigmoid nonlinearity.
\end{itemize}

The normalized attention value is obtained using a softmax over all instances:

\begin{equation}
\label{eq:ga_softmax_sup}
a_i = 
\frac{
\exp(u_i)
}{
\sum_{j=1}^{N} \exp(u_j)
}.
\end{equation}

Substituting Eq. \ref{eq:ga_softmax_sup} into Eq. \ref{eq:bag_agg_sup}, we obtain the gated attention 
aggregation:

\begin{equation}
\label{eq:ga_agg_final_sup}
\hat{\mathbf{Z}}
=
\sum_{i=1}^{N}
\frac{
\exp(u_i)
}{
\sum_{j=1}^{N} \exp(u_j)
}
\mathbf{z}_i.
\end{equation}

\section{Perturbation Analysis}

To ensure a fair and meaningful assessment of attention faithfulness, all perturbation-based analyses are performed \textbf{only on correctly classified slides}. This choice follows standard practice in explainability evaluation, as perturbation curves computed on incorrect predictions may reflect model failure rather than the quality of the explanation. Restricting the analysis to correctly predicted bags ensures that attention faithfulness is evaluated conditional on correct model reasoning.
Unless stated otherwise, patches are progressively removed according to Eq.~\ref{eq:perturb_system}, and the model output $s(k)$ is recorded at each perturbation step $k$. Empty bags resulting from complete removal are represented by zero-valued inputs.


For quantitative comparison across models and tasks (Tables ~\ref{tab:tcga_scores_rebuttal},~\ref{tab:C16-UNI},~\ref{tab:ablation_rebuttal} and Fig.~\ref{fig:auc_aupc}), we report AUPC as a relative area, normalized with respect to the unperturbed prediction. This normalization accounts for differences in baseline confidence across models and tasks, enabling more meaningful cross-model comparisons.

For visualization purposes (box plots of perturbation curves on \ref{fig:aupc_brca}), we report the raw, unnormalized perturbation trajectories $s(k)$ without dividing by $s(0)$. This choice allows a direct and interpretable comparison of how predictive confidence degrades as a function of the perturbation level, without rescaling the curves. Importantly, this visualization choice does not affect the relative ordering of methods, but provides a more intuitive depiction of the perturbation curves in the overall figure.
