%
\documentclass[accepted, breaklinks,colorlinks=true,linkcolor=BrickRed, urlcolor=blue, anchorcolor=Blue,citecolor=Blue,backref=page]{uai2023} %
                                    %
                                    %
                                    %
%
%
                                         %
%
                                          %
%
%
%

%
\usepackage[american]{babel}
%

%
\usepackage[authoryear,round]{natbib} %
\bibliographystyle{abbrvnat}
\renewcommand{\bibsection}{\subsubsection*{References}}
\usepackage{mathtools} %
\usepackage{amssymb}
\usepackage{multirow}
\usepackage[table,dvipsnames]{xcolor}
\usepackage{amsthm}
\usepackage{booktabs}
\usepackage{bm}
%
\usepackage{booktabs} %
\usepackage{tikz} %
\usepackage{notation}
\usepackage{hyperref}
\usepackage{subcaption}
\usepackage{cleveref}
\crefname{figure}{Fig.}{Figs.}
\crefname{definition}{Def.}{Defs.}
\crefname{corollary}{Cor.}{Cors.}
\crefname{proposition}{Prop.}{Props.}
\crefname{theorem}{Thm.}{Thms.}
\crefname{remark}{Remark}{Remarks}
\crefname{principle}{Principle}{Principles}
\crefname{lemma}{Lemma}{Lemmata}
\crefname{claim}{Claim}{Claims}
\crefname{table}{Tab.}{Tabs.}
\crefname{section}{\S}{\S\S}
\crefname{subsection}{\S}{\S\S}
\crefname{subsubsection}{\S}{\S\S}
\crefname{assumption}{Assumption}{Assumptions}
\crefname{appendix}{App.}{App.}
\crefname{equation}{}{}
\crefname{example}{Example}{Examples}
%
%
%
%
%

%
\newcommand{\ulim}[1]{\underset{#1 \rightarrow \infty}{\text{lim}}} %
\newcommand{\cons}{\xrightarrow{\text{P}}} %
\newcommand{\asconv}{\xrightarrow{\text{a.s.}}} %
\newcommand{\sampleiid}{\stackrel{\text{i.i.d.}}{\sim}} %
\newcommand{\indep}{\perp \!\!\! \perp} %
%
\newcommand{\limsupm}{\limsup\limits_{m \rightarrow \infty}} %

\newtheorem{theorem}{Theorem}[section]
\newtheorem{definition}{Definition}[section]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{remark}[theorem]{Remark}
\renewcommand*{\backref}[1]{}
\renewcommand*{\backrefalt}[4]{%
    \ifcase #1%
          \or [Cited on page~#2.]%
          \else [Cited on pages~#2.]%
    \fi%
    }
%
%
\title{Causal Effect Estimation from Observational and Interventional Data \\ Through Matrix Weighted Linear Estimators}
%
%
%
%
%
%
%
%
\author[1,2]{Klaus-Rudolf Kladny%
}
\author[2,3]{Julius von K\"ugelgen}
\author[1,2]{Bernhard Sch\"olkopf}
\author[2]{Michael Muehlebach}
%
\affil[1]{%
    Department of Computer Science\\
    ETH Z\"urich\\
    Switzerland
}
\affil[2]{%
    Max Planck Institute for Intelligent Systems,
    T\"ubingen, Germany
}
\affil[3]{%
    Department of Engineering, University of Cambridge, United Kingdom
    
    \texttt{\{kkladny, jvk, bs, michaelm\}@tue.mpg.de}
    }
\begin{document}
\maketitle

\begin{abstract}
%
We study causal effect estimation from a mixture of observational and interventional data in a confounded linear regression model with multivariate treatments. 
%
%
%
We show that the statistical efficiency in terms of expected squared error can be improved by combining estimators arising from both the observational and interventional setting. 
%
To this end, we derive methods based on matrix weighted linear estimators and prove that our methods are asymptotically unbiased in the infinite sample limit. This is an important improvement compared to the pooled estimator using the union of interventional and observational data, for which the bias only vanishes if the ratio of observational to interventional data tends to zero. 
%
Studies on synthetic data confirm our theoretical findings. In settings where confounding is substantial and the ratio of observational to interventional data is large, our estimators outperform a Stein-type estimator and various other baselines.
    %
\end{abstract}
\section{Introduction}\label{sec:intro}
%
%
Estimating the causal effect of a treatment variable on an outcome of interest is a fundamental scientific problem that is central to disciplines such as econometrics, epidemiology, and social science~\citep{angrist2009mostly,morgan2014counterfactuals,imbens2015causal,hernan2020causal}.
A fundamental obstacle to this task is the possibility of hidden confounding: unobserved variables that influence both the treatment and the outcome may introduce additional 
%
associations between them~\citep{reichenbach1956direction}.
As a result, estimators purely based on observational (passively collected) data can be biased and typically do not recover the true causal effect.

%
This contrasts experimental studies such as randomized controlled trials~\citep[RCTs;][]{neyman1923application,fisher1936design}, where the treatment assignment mechanism is modified through an external intervention, thus breaking potential influences of confounders on the treatment.
For this reason, RCTs have become the gold standard for causal effect estimation.
However, obtaining such interventional data is difficult in practice because the necessary experiments are often infeasible, unethical, or very costly to perform.

\begin{figure}[tb] 
\centering
\begin{subfigure}{0.5\columnwidth}
\centering
\includegraphics[width=10em]{visualisations/scm_obs.pdf}
\caption{observational}
\label{fig:setting_obs}
\end{subfigure}%
\begin{subfigure}{0.5\columnwidth}
\centering
\includegraphics[width=10em]{visualisations/scm_int.pdf}
\caption{interventional}
\label{fig:setting_int}
\end{subfigure}
\caption{\textbf{Overview.} We estimate the causal effect~$\alphab$ of a continuous multi-variate treatment~$\Xb$ on a scalar outcome~$Y$ in a linear Gaussian structural equation model with unobserved confounders~$\Zb$. 
Given a large sample from the observational setting in (a) and a small sample from the interventional setting in (b), we establish an optimal, sample-size dependent matrix weighting scheme for combining the biased, low-variance observational estimator with the unbiased, high-variance interventional estimator.}
%
\label{fig:scm}
%
\end{figure}

%
In contrast, observational data is usually cheap and abundant, motivating the study of causal inference from observational data~\citep{rubin1974estimating,pearl2009causality}.
In fact, in certain situations causal effects can be identified and estimated from purely observational data, even under hidden confounding, e.g., in the presence of natural experiments~\citep[instrumental variables;][]{angrist1996identification} or observed mediators~\citep[front-door adjustment;][]{pearl1995causal}.
However, this does not apply to the general case in which a treatment~$\Xb$ and an outcome~$Y$ are confounded by an unobserved variable~$\Zb$ as shown in~\cref{fig:setting_obs}.

%
\looseness-1 In the present work, we study treatment effect estimation in this general setting under the assumption that we have access to both observational and interventional data. The latter can be viewed as sampled from the setting shown in~\cref{fig:setting_int}, where the arrow $\Zb\to\Xb$ has been removed as a result of the intervention on $\Xb$~\citep[graph surgery;][]{spirtes2000pc}, and is thus unbiased for our task. 
Due to small sample size, however, the estimator based only on interventional data may exhibit high variance. 
Our main idea is therefore to use the (potentially large amounts of) observational data for variance reduction---at the cost of introducing some bias. This is achieved by forming a combined estimator, which is superior to the purely interventional one in terms of 
%
mean squared error. 

%
We make the key assumption that both the treatment $\Xb \to Y$ and confounding $\Zb\to\{\Xb,Y\}$ effects are linear, but allow for treatment~$\Xb$ and unobserved confounder~$\Zb$ to be continuous and multi-variate. 
We then consider a class of estimators of the causal effect parameter vector that combine the unbiased, but high-variance interventional estimator and the biased, but low-variance observational estimator through weight matrices---akin to a multi-variate convex combination.
%
We study the statistical properties of these estimators, establish theoretical optimality results, and investigate their empirical behavior through simulations. 



In summary, we highlight the following contributions:
\begin{itemize}[topsep=0em,itemsep=0em,leftmargin=*]
    \item We introduce a new framework of weighing linear estimators using matrices and show that several existing approaches fall into this category~(\cref{sec:weighting_matrix}).
    \item \looseness-1 We prove that, unlike pooling observational and interventional data~(\cref{prop:MSE_greater_zero}), our matrix weighting approaches 
    %
    achieve vanishing mean squared error in the interventional sample limit~(\cref{prop:weight_matrix_cons,theo:no_bias_in_the_limit}) if the ratio between observational and interventional data is non-vanishing.
    %
    \item \looseness-1 We discuss two practical approaches for variance reduction in estimating optimal weight matrices (\cref{sec:inductive_biases}; \cref{prop:weak_weight_consistency}), and demonstrate through simulations that our estimators outperform 
    %
    baselines and existing methods
    %
    in situations where confounding is substantial~(\cref{sec:experiments}).
\end{itemize}
%
\section{Related Work}\label{sec:related_work}
%
%
Causal reasoning, i.e., inferring a causal query such as a causal effect, can be split up into the tasks of (i) identification and (ii) estimation.
Step (i) operates at the population level and seeks to answer whether a causal question can---at least in principle---be answered given infinite data.
If the answer is positive and a valid estimand is provided, step (ii) then aims to construct a statistically efficient estimator.

%
%
%
%
A causal query is identified from a set of assumptions if it can be expressed in terms of the available distributions (e.g., a mixture of different observational and interventional distributions). 
%
To this end, \citeauthor{pearl2009causality}'s do-calculus~(\citeyear{pearl1995causal,pearl2009causality}) provides an axiomatic set of rules for manipulating causal expressions based on graphical criteria.
The identification task has been studied extensively~\citep{tian2002general,pearl2014external,bareinboim2016causal} and has by now been solved for many settings of interest: In these cases, the do-calculus---and its extensions~\citep{correa2020general}---are sound and complete in that they provide a valid estimand if and only if one exists~\citep{huang2006identifiability,shpitser2006identification,bareinboim2012causal,sanghack2020gID}.
%

%

%
%

\looseness-1 In our setting from~\cref{fig:scm}, the causal effect~$\alphab$ is not identifiable from observational data, but is trivially identified by intervening on $\Xb$.
Yet, this leaves open the question of \textit{how to estimate $\alphab$ from finite data in the best possible way}.
In contrast to the plethora of works on identification, there is much less
%
prior literature about statistical efficiency of causal parameter estimation, particularly for confounded settings. 
%
%
%
%
%

\begin{figure}[tb]
    \centering
    \includegraphics[width=\columnwidth]{visualisations/relation_to_shrink.pdf}
    \caption{\textbf{Relation between shrinkage 
    %
    and estimator weighting in 2D.} Classical shrinkage methods like ridge regression  (yellow) shrink the interventional estimator $\alphaI$ toward the origin. Scalar estimator weighting (red) instead shrinks toward the observational estimator~$\alphaO$. 
    Dashed circles show the covariances of $\alphaO$ and $\alphaI$, here assumed isotropic. $\bm\Delta$ (green) is the confounding-induced bias of $\alphaO$.
    %
    }
    \label{fig:relation_to_shrink}
    %
\end{figure}

A common source of inspiration for both prior work and our approach is that of shrinkage estimation. 
In light of the bias-variance decomposition of the mean squared error~\citep[e.g.,][p.~24]{hastie2009elements}, shrinkage can yield a strictly better (``dominating'') estimator by reducing variance, at the cost of introducing some bias.
These ideas were first introduced in frequentist statistics by~\citet{stein1956inadmissibility,james1961estimation} who showed that the maximum likelihood estimate of a multivariate mean is dominated by shrinking towards a fixed point such as the origin.
Similar ideas are also at the heart of empirical Bayes analysis~\citep{robbins1964empirical,efron1973stein,efron2012large}.
%
%
For estimating a parameter vector~$\alphab$ in a linear model, as is the focus of the present work, a classical shrinkage method is ridge regression~\citep{hoerl1970ridge}.
%

%
Instead of shrinking towards the origin, an intuitive idea for causal effect estimation
%
is to \textit{shrink towards the observational estimator}. The hope is that the latter constitutes a better attractor
%
if the confounding bias is not too large---despite  a slight increase in variance compared to shrinking toward a constant.
%
We refer to this approach as scalar estimator weighting. \Cref{fig:relation_to_shrink} shows a visual comparison to classical shrinkage estimation.
The most closely related work on estimator weighting is that of
\citet{strawderman1991stein, green2005improved} and \citet{rosenman2020combining}. The former two consider general biased and unbiased estimators. The latter propose weighting schemes for estimating vectors of multiple \textit{binary} treatment effects. These works are strongly inspired by James-Stein shrinkage estimators
%
and minimize a generalized version of Stein's unbiased risk estimate~\citep[p.~150]{wasserman2006allofnonparam}. \Citet{rosenman2020combining}
%
show optimality among scalar weights with respect to minimizing the true risk as the %
dimensionality of the estimated treatment effects goes to infinity. However, these theoretical results rely on knowledge of the true covariance matrix of the interventional estimator (which is typically unknown in practice), and 
%
the behavior of their estimators %
in the infinite sample limit is not analyzed. 

%

Other work that focuses on combining observational and interventional data to estimate  causal effects of \textit{binary} treatments includes, e.g., \citet{kallus2018removing,cheng2021adaptive,ilse2021combining,rosenman2022propensity,hatt2022combining}, see~\citet{colnet2020causal} for a comprehensive survey.

\Citet{pmid33088006} also study combining estimators of binary treatment effects. However, in their framework an estimator with less bias in addition to a second error-prone estimator is computed from a second observational ``validation set'', in which all confounders are measured. Our framework, in contrast, does not require measurements of the confounders.
%
%
%
%

%


%
In the present work, we consider a
%
general
linear regression setting with continuous (rather than binary) multi-variate treatments. 
To combine observational and interventional data, we introduce a new class of matrix (rather than scalar) weighted estimators, of which ridge regression and data pooling are special cases.
Instead of employing Stein's unbiased risk estimate, we develop and analyze estimates for the theoretically optimal weight matrix, without making assumptions about the covariance structure of estimators.

%
Most approaches to causal estimation, including the present work, assume that the causal structure among variables is known and takes the general form of the directed acyclic graph in~\cref{fig:scm}.
For prior work on leveraging observational and interventional data for causal discovery, or structure learning, see, e.g., \citet{wang2017permutation}.


%
%



%

%

%
%
%

%
%

%
%

%

%

\section{Setting \& Preliminaries}\label{sec:setup}
%
\paragraph{Notation.} Upper case $Y$ denotes a scalar random variable, lower-case $y$ a scalar, bold lower-case $\xb$ a vector, and bold upper-case $\Xb$ either a matrix or random vector. 
%
The spectral norm of a matrix $\Xb$ is denoted by $\norm{\Xb}_2$.

\paragraph{Causal Model.} 
To formalize our problem setting, 
we adopt the structural causal model framework of~\citet{pearl2009causality}. Specifically, we assume that the causal relationships between the $d$-dimensional confounder~$\Zb$, the $p$-dimensional treatment~$\Xb$, and the scalar outcome~$Y$ are captured by the following linear Gaussian structural equation model~(SEM):
%
\begin{align} \label{equ:multivarSCMconfounder}
    \mathbf{Z}  &\; \leftarrow \; \mathbf{N}_{\mathbf{Z}}, 
    &&\Nb_\Zb\sim\Ncal(\bm{\mu}_{\Nb_\Zb}, \Sigmab_{\Nb_\Zb})\\
    \Xb  &\; \leftarrow \;  \mathbf{B} \mathbf{Z}+ \mathbf{N}_{\Xb}, \label{equ:SA_X} 
    &&\Nb_{\Xb}\sim\Ncal(\bm{\mu}_{\Nb_\Xb}, \Sigmab_{\Nb_\Xb})\\ 
    Y  &\; \leftarrow \; \mathbf{Z}^{\top} \bm\gamma+ \Xb^{\top} \bm\alpha+N_Y,
    &&N_Y\sim\Ncal(\mu_{N_Y}, \sigma^2_{N_Y}) \label{eq:outcome}
\end{align}
with $\Bb\in\RR^{p\times d}$, $\gammab\in\RR^d$, $\alphab\in\RR^p$, and $(\Nb_\Zb, \Nb_\Xb, N_Y)$ mutually independent exogenous noise variables. 
%
%
%
%
%
%
%
%
%
%
The SEM in~\cref{equ:multivarSCMconfounder}--\eqref{eq:outcome} induces an observational distribution over $(\Zb,\Xb, Y)$ which is referred to as $\Pobs$, see~\cref{fig:setting_obs}.

To model the interventional setting, we consider a soft intervention~\citep{eberhardt2007interventions},
which randomizes the treatment~$\Xb$ by replacing the assignment in~\cref{equ:SA_X} with
\begin{equation} \label{equ:replaced_SA}
    \Xb \; \leftarrow \; \widetilde{\mathbf{N}}_{\Xb},\qquad \qquad \widetilde{\mathbf{N}}_{\Xb}\sim\PP_{\widetilde{\mathbf{N}}_{\Xb}}, \qquad 
\end{equation}
%
%
%
%
%
%
%
%
%
where $\widetilde{\mathbf{N}}_{\Xb}$ is mutually independent of $\Nb_\Zb$ and $N_Y$. We note that $\widetilde{\mathbf{N}}_{\Xb}$ may be non-Gaussian. The modified interventional SEM consisting of~\cref{equ:multivarSCMconfounder,equ:replaced_SA,eq:outcome} induces a different, interventional distribution over $(\Zb,\Xb,Y)$, which we refer to as $\Pint$, see~\cref{fig:setting_int}. 

For ease of notation and for the remainder of this work, we assume without loss of generality that all noise variables are zero-mean. Details on how to extend our method to non zero-mean noise variables are provided in App.~D.
%
%
%
%
%
%

\paragraph{Data.} We assume access to two separate datasets of observations of $(\Xb,Y)$ of size $n$ and $m$, each sampled independently from the observational and interventional distributions (i.i.d.), respectively:
\begin{align*}
    (\xb_i, y_i) \; &\sampleiid \; 
    %
    \Pobs, \quad i=1,...,n,\\
    (\xb_i, y_i) \; &\sampleiid \; 
    %
    \Pint,\quad i=n+1,...,n+m,
\end{align*}
where $\Pobs$ and $\Pint$ denote the distributions of $(\Xb, Y)$ in the observational and interventional settings, respectively.
%
We note that the confounder $\Zb$ remains unobserved.
We concatenate the observational sample in a treatment matrix $\XO=(\xb_1, ..., \xb_n)^\top\in\RR^{n\times p}$ and outcome vector $\YO=(y_1,...,y_n)^\top\in\RR^n$, and similarly with $\XI,\YI$ for the interventional sample.
%
Finally, we denote the pooled data by  $\XP=(\XO, \XI)\in\RR^{(n+m)\times p}$ and $\YP=(\YO,\YI)\in\RR^{n+m}$. 

\paragraph{Goal.} Our objective is to obtain an accurate estimate of the parameter vector $\bm{\alpha}$, which characterizes the linear causal effect of $\Xb$ on $Y$ in~\cref{eq:outcome}. Formally, it is given by
%
%
%
\begin{align*}
    \bm{\alpha} = \nabla_{\xb} \mathbb{E} [Y|\mathrm{do}(\Xb \leftarrow \xb)],
\end{align*}
%
where the $\mathrm{do}(\cdot)$ operator denotes a manipulation of the treatment assignment akin to~\cref{equ:replaced_SA}, and the expectation is taken with respect to the corresponding conditional distribution. 

%

\paragraph{Confounding Issues.}In the general case with non-zero $\Bb$ and $\gammab$, the observational setting is confounded, meaning %
\begin{equation*}
    %
    \Pobs(Y | \Xb = \xb) \neq 
    %
    \PP(Y | \mathrm{do}(\Xb \leftarrow \xb))=\Pint(Y | \Xb = \xb),
\end{equation*}
which complicates the use of observational data.
Specifically, for our assumed model~\cref{equ:multivarSCMconfounder}--\eqref{eq:outcome} the conditional expectation of $Y$ under $\Pobs$ is given by
%
the following perturbed linear model~\citep{cevid2020spectral}:
%
\begin{equation} \label{eq:pert_model}
    \mathbb{E}_\text{obs}
    %
    [Y | \Xb = \xb]=(\bm{\alpha} + \bm{\Delta})^{\top} \xb,
\end{equation}
where $\Deltab\in\RR^p$ denotes the \textit{confounding bias}, which 
%
%
%
%
%
%
is given explicitly in terms of the model parameters as
%
\begin{equation}\label{equ:Delta}
    \bm{\Delta} = (\Sigmab_{\mathbf{N}_{\Xb}} + \mathbf{B} \Sigmab_{\Nb_\mathbf{Z}} \mathbf{B}^{\top})^{-1} \mathbf{B} \Sigmab_{\Nb_\mathbf{Z}} \bm{\gamma}.
\end{equation}
It can be seen from~\cref{equ:Delta} that the confounding bias~$\Deltab$ is zero if $\Bb$
 or $\gammab$ are zero (i.e., $\Zb$ only affects either $\Xb$ or $Y$).
 Furthermore, we have that, in general,
\begin{equation}\label{eq:cond_vars}
      \text{Var}_{\text{obs}}(Y | \Xb)=\condvar \neq \intvar=\text{Var}_\text{int}(Y|\Xb)\,.
\end{equation}
%
%

%

\paragraph{Assessing Estimator Quality.}
We rely on
%
mean squared error with respect to the true parameter $\bm{\alpha}$ as a measure for comparing different estimators.
\begin{definition}[MSE]
Let $\widehat{\bm{\alpha}}$ be any function of the pooled data $(\XP,\YP)$ taking values in $\mathbb{R}^p$. Then
%
\begin{equation*}
    \MSE(\widehat{\bm{\alpha}}) \coloneqq \mathbb{E}\left[\norm{\widehat{\bm{\alpha}} - \bm{\alpha}}_2^2\right],
\end{equation*}
where the expectation is taken over $\XP, \YP$.
\end{definition}
We note that the mean squared error can also be written as follows:
%
\begin{equation}
\label{eq:bias_variance}
    \MSE(\widehat{\bm{\alpha}}) = \norm{\mathbf{Bias}(\widehat\alphab)}_2^2
    %
    + \mathrm{Tr}(\mathbf{Cov}(\widehat{\bm{\alpha}}))\, ,
\end{equation}
where
\begin{equation*}
    \begin{aligned}
    \mathbf{Bias}(\widehat\alphab)&=\EE[\widehat\alphab] -\alphab\, ,\\
    \mathbf{Cov}(\widehat\alphab)&=\EE[(\widehat\alphab-\EE[\widehat\alphab])(\widehat\alphab-\EE[\widehat\alphab])^\top]\, .
    \end{aligned}
\end{equation*}
This decomposition highlights that biased estimators can dominate unbiased ones through variance reduction.
%

\paragraph{Pure Estimators.}
We study estimators for $\alphab$ that are linear combinations of the following ordinary least squares estimators  obtained on the two data sets individually.
%
\begin{definition}[Pure Estimators]
\label{def:pure_estimators}
For non-singular moment matrices $\XO^{\top} \XO$ and $\XI^{\top} \XI$,
the \emph{pure estimators} based only on the observational/interventional sample are given by:
\begin{align*}
    \alphaO &\coloneqq (\XO^{\top} \XO)^{-1} \XO^{\top} \YO,\\
    \alphaI &\coloneqq (\XI^{\top} \XI)^{-1} \XI^{\top} \YI.
\end{align*}
\end{definition}
Recall that $\alphaI$ is unbiased while $\alphaO$ has bias $\Deltab$. Their covariances conditionally on $\XO$ and $\XI$ are given by
\begin{equation}
\label{eq:estimator_covariances_true}
\begin{aligned}
    \COV(\alphaO)&=(\XO^{\top}\XO)^{-1}\condvar,
    \\
    \COV(\alphaI)&=(\XI^{\top}\XI)^{-1}\intvar.
\end{aligned}
\end{equation}
%
%
Unlike previous work~(see \cref{sec:related_work}), we do not make assumptions about the covariance structure of either estimator.



\paragraph{Almost sure convergence.} To analyze the behavior of estimators in the infinite sample limit, we will employ the following characterization known as \textit{almost sure convergence}.
\begin{definition}[Almost Sure Convergence]
\label{def:consistency}
Let $\mathbf{M}$ be a random matrix with realizations in $\mathbb{R}^{p \times p}$. We say a sequence of random matrices $\widehat{\mathbf{M}}_m$ indexed by $m \in \mathbb{N}$ \emph{converges almost surely} to ~$\mathbf{M}$, denoted $\widehat{\mathbf{M}}_m \asconv \mathbf{M}$, if and only if
\begin{equation*}
    \lim_{m\to\infty} \text{P} \left( \widehat{\mathbf{M}}_m  = \mathbf{M} \right) = 1,
\end{equation*}

where $\text{P}$ denotes probability.
%
%
%
%
\end{definition}

\section{Matrix Weighted Linear Estimators}\label{sec:weighting_matrix}

%
We now introduce our class of matrix weighted linear estimators, which combine the two pure estimators from~\cref{def:pure_estimators} using a weight matrix $\Wb$
%
to obtain a new (better) estimator.

\begin{definition}[$\Wb$-weighted Linear Estimator] \label{def:mat_weight_est}
Let $\Wb \in \mathbb{R}^{p \times p}$ (possibly random). The \emph{$\Wb$-weighted linear estimator} for $\alphab$ is given by
\begin{equation*}
    \alphaWeight{\Wb} \coloneqq \Wb \alphaI + (\mathbf{I}_p - \Wb)  \alphaO.
\end{equation*}
We furthermore refer to $\Wb$ as a weight matrix.
\end{definition}
We will generally think of $n$ as a function of $m$, where we sometimes even explicitly write $n(m)$. However, to simplify notation we index estimators by $m$ only, omitting the dependence $n(m)$.

Note that the purely interventional estimator is a special case of a $\Wb$-weighted estimator with $\Wb=\Ib_p$.
However, while unbiased, it may be subject to high variance if $m$ is very small.\footnote{E.g., consider a one-dimensional setting with $x_i = 1$ if $i$ is even 
and $-1$ otherwise. Then, for odd $m$, $\text{Var}(\alphaI|\XI) \propto (\sum_i x_i^2)^{-1}=\frac{1}{m}$.
%
} Hence, we generally prefer to employ the observational data as well and choose $\Wb\neq \Ib_p$.

\subsection{Existing Methods as Special Cases}

First, we show that several standard approaches can be viewed as special cases of matrix-weighted estimators.

\paragraph{Data Pooling.}  %
A straightforward approach for combining both data sets 
%
is to 
%
compute an estimator on the pooled data. 
%
The resulting least-squares estimator $\alphaP$ is: 
%
%
\begin{equation}
\begin{aligned} \label{equ:data_pooling}
    \alphaP :=&\, (\XP^{\top}\XP)^{-1} \XP^{\top} \YP \\ 
    =&\, (\XO^{\top}\XO + \XI^{\top} \XI)^{-1} (\XO^{\top} \YO + \XI^{\top} \YI) \\
    =&\, \WP  \alphaI + (\mathbf{I} - \WP)  \alphaO,
\end{aligned}
\end{equation}
where 
\begin{equation}\label{eq:W_data_pooling}
    \WP \coloneqq (\XO^{\top}\XO + \XI^{\top} \XI)^{-1} \XI^{\top} \XI.
\end{equation}
We see that
%
$\alphaP$ indeed qualifies as a valid matrix weighted estimator in the sense of~\cref{def:mat_weight_est}.

However, data pooling can lead to highly undesirable limiting behavior in cases where the amount of observational data~$n(m)$ does not vanish in the limit of infinite interventional data~$m \rightarrow \infty$. 
An example for this is given in the following proposition.
%
\begin{proposition} \label{prop:MSE_greater_zero}
Let $\lim_{m\to\infty} \frac{n(m)}{m} = c$ for some $c > 0$ and $\bm{\Delta} \neq \mathbf{0}$. Then, it holds that
\begin{equation*}
    \lim_{m\to\infty} \MSE \left(\alphaP \right) > 0.
\end{equation*}
\end{proposition}
The proof of~\cref{prop:MSE_greater_zero} is provided in App.~A.1. We note, however, that this does not happen for a vanishing amount of observational data, that is $\lim_{m\to\infty}\frac{n(m)}{m} = 0$ (see Prop.~4.2 in App.~A.2).

%

\paragraph{Ridge Regression.} %
%
The ridge regression estimator on the interventional data, which shrinks~$\alphaI$ towards the origin (see~\cref{sec:related_work} and~\cref{fig:relation_to_shrink}),
%
is given by
%
\begin{align*}
    \widehat{\bm{\alpha}}^m_{\text{ridge}} &= (\XI^{\top}\XI + \lambda \mathbf{I}_p)^{-1} \XI^{\top} \YI \\
    &=  \Wridge \alphaI + (\mathbf{I}_p - \Wridge) \mathbf{0},
\end{align*}
where
\begin{equation}
\label{eq:W_ridge}
    \Wridge \coloneqq (\XI^{\top}\XI + \lambda \mathbf{I}_p)^{-1} \XI^{\top}\XI.
\end{equation}
Hence, $\widehat{\bm{\alpha}}^m_{\text{ridge}}$ can also be seen as a special case of a matrix weighted estimator with no observational data and $\alphaO=\mathbf{0}$.
Further, comparing~\cref{eq:W_data_pooling,eq:W_ridge} suggest an interpretation of
%
ridge regression 
%
as a \textit{poor man's} data pooling since access to observational data is replaced by a positive definite data matrix $\lambda \mathbf{I}_p$.
%
However, $\lambda$ is a constant, and therefore $\lim_{m\to\infty} \MSE (\widehat{\bm{\alpha}}^m_{\text{ridge}}) = 0$ even in the setting of ~\cref{prop:MSE_greater_zero}, which contrasts data pooling.


\subsection{Optimal Weighting Schemes} \label{sec:optimal_weighting_matrix}
%
We now establish theoretically optimal weighting schemes that minimize the mean squared error of $\Wb$-weighted linear estimators $\widehat\alphab^m_\Wb$ for different classes of weight matrices~$\Wb$ by exploiting the specific structure of our problem setting~(\cref{sec:setup}).
%

\paragraph{Optimal Scalar Weight.}
First, we consider the special case of scalar estimator weighting by considering weight matrices of the form $\Wb=w\Ib_p$
%
with weight $w \in [0, 1]$.
%
The optimal scalar weight is then derived as follows:
\begin{align*}
    &\frac{\partial}{\partial w} \MSE\left(\alphaWeight{w\Ib_p}\right) \overset{!}{=} 0
    \\
    \stackrel{\eqref{eq:bias_variance}}{\iff} &\frac{\partial}{\partial w} \left( \norm{\mathbb{E}[\alphaWeight{w\Ib_p} - \bm{\alpha}]}_2^2 + \text{Tr}\left(\mathbf{Cov}\left(\alphaWeight{w\Ib_p}\right)\right)\right) \overset{!}{=} 0
    \\
    \implies &\wopt = \frac{\text{Tr}(\mathbf{Cov}(\alphaO)) + \norm{\bm{\Delta}}_2^2}{\text{Tr}(\mathbf{Cov}(\alphaI)) + \text{Tr}(\mathbf{Cov}(\alphaO)) + \norm{\bm{\Delta}}_2^2}.
\end{align*}
%

\paragraph{Optimal Diagonal Weight Matrix.}
A more general case is to weigh each dimension individually by different scalars 
%
$w^{(k)} \in [0, 1], \; k = 1, ..., p$, corresponding to a weight matrix of the form $\Wb=
%
\mathrm{diag}(\wb)$.
%
The optimal diagonal weighting $\text{diag}(\wb^{m}_{*}$) is then given by
\begin{equation*}\label{equ:theor_optimal_diagonal}
    w_*^{m (k)} = \frac{\mathrm{Cov}^{(k, k)}(\alphaO) + \Delta^{(k) \, 2}}{\mathrm{Cov}^{(k, k)}(\alphaI) + \mathrm{Cov}^{(k, k)}(\alphaO) + \Delta^{(k) \, 2}}\,, 
\end{equation*}
for $k = 1, ..., p$.
The derivation is analogous to that for the optimal scalar weight above, with the only difference being that we optimize over each dimension separately.

\paragraph{Optimal Weight Matrix.}
Finally, we can also determine the optimum weighting as follows:
\begin{align}
\begin{split} \label{equ:theor_optimal_matrix}
    \Wopt = &\left( \COV(\alphaO) + \bm{\Delta} \bm{\Delta}^{\top} \right) \\ &\left( \COV(\alphaI) + \COV(\alphaO) + \bm{\Delta} \bm{\Delta}^{\top} \right)^{-1}.
\end{split}
\end{align}
A thorough derivation of the proposed weighting schemes can be found in App.~C. In addition, we elaborate on how this weighting scheme handles sample imbalance in App. ~E.

\begin{remark} \label{rem:recover_data_pooling}
If (i) $\bm{\Delta} = \mathbf{0}$ and (ii) $\condvar = \intvar$, then $\Wopt  = \WP$, i.e., data pooling corresponds to weighing with the optimal weight matrix under these two assumptions.
\end{remark}
\looseness-1 \Cref{rem:recover_data_pooling} can be verified by  simplifying~\cref{equ:theor_optimal_matrix} with assumptions (i) and (ii) and comparing to~\cref{eq:W_data_pooling}. It agrees with our intuition: Ordinary least squares relies on the assumption that $\mathbb{E}[Y | \Xb=\xb_i] = \alphab^\top \xb_i$ with equal variance, for all $i$. Thus, data pooling recovers the optimal estimator if these assumptions are true, i.e., the two conditional distributions $\Pobs(Y|X)$ and $\Pint(Y|\mathrm{do}(X))$ are identical. However, in general, they will not be identical and data pooling then amounts to model misspecification. This is likely to result in a non-vanishing mean squared error for $m \rightarrow \infty$ as highlighted in~\cref{prop:MSE_greater_zero}.


\subsection{Practical Estimators} \label{sec:practicality}
%
Unfortunately, the optimal weighting derived in~\cref{equ:theor_optimal_matrix} cannot be implemented directly, since the quantities 
$\bm{\Delta}$, $\COV(\alphaO)$, and $\COV(\alphaI)$ are unknown in practice.
%
%
To construct practical estimators informed by our theoretical insights, one option is thus to
%
rely on plug-in estimates of these unknown quantities. 
For $\COV(\alphaI)$ and $\COV(\alphaO)$, we use the standard estimators
\begin{equation*}
\begin{aligned}
    \widehat{\COV}(\alphaI) &= (\XI^{\top} \XI)^{-1} \widehat{\sigma}^2_{Y|\mathrm{do}(\Xb)},
    \\
    \widehat{\COV}(\alphaO) &= (\XO^{\top} \XO)^{-1} \widehat{\sigma}^2_{Y|\Xb},
\end{aligned}
\end{equation*}
which replace the conditional variances in~\cref{eq:estimator_covariances_true} by
\begin{equation*}
\begin{aligned}
    \widehat{\sigma}^2_{Y|\mathrm{do}(\Xb)} &= \frac{1}{m-1} \norm{ \YI - \XI \alphaI }_2^2,
    \\
    \widehat{\sigma}^2_{Y|\Xb} &= \frac{1}{n-1} \norm{ \YO - \XO \alphaO }_2^2.
\end{aligned}
\end{equation*}
%
For $\bm\Delta$, one may consider using the unbiased estimator
\begin{equation}
\label{eq:delta_plugin}
    \Deltaest = \alphaO - \alphaI.
\end{equation}
Substituting these into~\cref{equ:theor_optimal_matrix} then yields:
%
\begin{align} \label{eq:opt_matrix_estimator}
\begin{split}
    \Wopthat = &\left( \widehat{\COV}(\alphaO) + \Deltaest \Deltaest^{\top} + \epsilon \Ib_p \right) \\ &\left( \widehat{\COV}(\alphaI) + \widehat{\COV}(\alphaO) + \Deltaest \Deltaest^{\top} + \epsilon \Ib_p \right)^{-1}.
\end{split}
\end{align}
The regularization with $\epsilon > 0$ ensures that the inverse remains stable even in the large sample limit where $\widehat{\COV}(\alphaO)$ and $\widehat{\COV}(\alphaI)$ tend to zero. The reason for instability without such regularization is that $\Wopt$ is not uniquely defined in the infinite sample limit. With regularization, however, we can guarantee that $\Wopthat$  converges to $\Ib_p$ almost surely.

\begin{proposition}[Weight Matrix Convergence] \label{prop:weight_matrix_cons}
    Let $\lim_{m\to\infty} \frac{n(m)}{m}=c$, for some constant $c > 0$. Then, 
    %
    %
    %
    %
    %
    $\Wopthat$ from~\cref{eq:opt_matrix_estimator} converges almost surely to $\Ib_p$, i.e., $\Wopthat \asconv \Ib_p$.
\end{proposition}
%
The proof for~\cref{prop:weight_matrix_cons} is included in App.~A.3. We can show that this convergence
%
implies that the mean squared error vanishes asymptotically.
\begin{theorem}[Zero Mean Squared Error in the Sample Limit]\label{theo:no_bias_in_the_limit}
    Let $\West$ be any sequence of random weight matrices such that $\West \asconv \Ib_p$ and $\lim_{m\to\infty} \frac{n(m)}{m} = c$ for some constant $c>0$.
    %
    %
    %
    %
    Then,
    \begin{equation*}
        \ulim{m} \; \MSE \left( \alphaWeight{\West} \right) = 0,
    \end{equation*}
    where $\alphaWeight{\West}$ denotes the matrix-weighted linear estimator with weight matrix $\West$, as defined in~\cref{def:mat_weight_est}.
\end{theorem}
%
The proof of Thm. \ref{theo:no_bias_in_the_limit} is included in App.~A.4. 

\Cref{theo:no_bias_in_the_limit} has the following relevant implication:  
%
we can incorporate an arbitrarily large amount of biased observational data and are still guaranteed that the bias (and also variance) of $\alphaWeight{\Wopthat}$ will vanish in the infinite sample limit. Moreover, 
%
this guarantee is independent of $\bm{\Delta}$ and $| \sigma_{Y|\Xb}^2 - \sigma_{Y|\mathrm{do}(\Xb)}^2 |$. 

We also note that \Cref{theo:no_bias_in_the_limit} does not imply unbiasedness of $\alphaWeight{\Wopthat}$ for any finite sample size. 

Further, we note
%
that almost sure convergence of $\West$ to $\Ib_p$ may generally not
be the only option to achieve vanishing mean squared error.
%
For example, if $\Deltab = \mathbf{0}$ such that $\alphaO$ is unbiased, we also obtain vanishing mean squared error for almost sure convergence of $\West$ to $\mathbf{0}$.
%
%

%

%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%

\subsection{Suitable Inductive Biases}\label{sec:inductive_biases}
%
Despite the desirable performance established in~\cref{theo:no_bias_in_the_limit}, the
%
plug-in estimates from~\cref{sec:practicality} will often not perform very well in finite sample settings. The main issue is the estimation of $\bm{\Delta}$, which has a large variance when done according to~\cref{eq:delta_plugin}.
To see this, we first note that
\begin{equation}
\label{eq:delta_covar_decomp}
    \text{Tr}(\COV(\Deltaest)) =  \text{Tr}(\COV(\alphaI)) + \text{Tr}(\COV(\alphaO)),
\end{equation}
since the observational and interventional data are independent.
Now, if we only have a small interventional sample (as is typically the case), 
%
$\text{Tr}(\COV(\alphaI))$ and hence according to~\cref{eq:delta_covar_decomp} also $\text{Tr}(\COV(\Deltaest))$ will be large.
%

We therefore explore possible 
%
inductive biases in the form of additional assumptions on the type of confounding that lead to reduced variance when estimating $\Deltaest$. These inductive biases can be motivated from domain knowledge and validation techniques such as cross-validation \citep{schaffer1993selecting}. Specifically, the application itself may provide some prior knowledge about the nature of confounding, which can then be confirmed by a better validation score compared to 
%
the other inductive biases/methods proposed here.

To this end, we observe that~\cref{eq:delta_plugin} can be written
%
as the solution of the following two-step ordinary least squares procedure:
\begin{align}
    &\alphaO  &&\gets \; \mathrm{arg} \; \min_{\bm{\alpha} \in \mathbb{R}^p} \; \left\{ \norm{ \YO \; - \; \XO \bm{\alpha} }_2^2 \right\} \nonumber \\
    &\mathbf{r}  &&\gets \; \YI - \XI \alphaO \nonumber \\
    &\Deltaest  &&\gets \; \mathrm{arg} \; \min_{\bm{\Delta} \in \mathbb{R}^p} \; \left\{ \norm{ \mathbf{r} \; + \; \XI \bm{\Delta} }_2^2 \right\}.\label{equ:optimization}
\end{align}

\paragraph{Small $\norm{\bm{\Delta}}_2$.}
In some settings, we may be willing to assume that, despite the existence of unobserved confounders, the resulting confounding bias is rather weak, i.e., that its Euclidean norm $\norm{\bm{\Delta}}_2$ is small. 
Since this is precisely the assumption underlying ridge regression, 
%
we reformulate~%
\eqref{equ:optimization} using a regularizer $\lambda_{\ell^2} > 0$ as
%
\begin{equation*}
    \Deltaest^{\ell^2} \; \gets \; \mathrm{arg} \; \min_{\bm{\Delta} \in \mathbb{R}^p} \; \left\{ \norm{\mathbf{r} \; + \; \XI \bm{\Delta} }_2^2 \; + \; \lambda_{\ell^2} \norm{\bm{\Delta}}_2^2\right\}, 
\end{equation*}
for which a closed-form solution of the same computational complexity as least squares exists. 
We refer to the weight matrix estimate obtained by using $\Deltaest^{\ell^2}$ in place of $\Deltaest$ in~\cref{eq:opt_matrix_estimator} as~$\Wweak$. By~\cref{prop:weak_weight_consistency}, we still obtain the same limiting guarantees of~\cref{theo:no_bias_in_the_limit} for $\Wweak$, as long as $\lambda_{\ell^2}$ is fixed ($\lambda_{\ell^2}$ is independent of $m$, $\XP$, $\YP$).
%
\begin{proposition}\label{prop:weak_weight_consistency}
Let $\lim_{m\to\infty} \frac{n(m)}{m} = c$ and $\lambda_{\ell^2} > 0$ be fixed. Then,
    \begin{equation*}
        \ulim{m} \; \MSE \left( \alphaWeight{\Wweak} \right) = 0.
    \end{equation*}
\end{proposition}
The proof for Proposition \ref{prop:weak_weight_consistency} is given in App.~A.5.
\paragraph{Small $\norm{\bm{\Delta}}_0$.}
In other settings, we may have prior beliefs that only some treatment variables~$X_i$ are confounded, i.e., that the number of nonzero elements of $\bm{\Delta}$, denoted by $\norm{\bm{\Delta}}_0$, is small.
%
If we are unaware of
%
which treatments are confounded, but
%
$p$ is small, we can simply fit all $2^p$ possible models or use best subset selection~\citep[p. 205]{james2013introduction}. For larger $p$, a more efficient technique known as the LASSO employs $\ell^1$-regularization and has become a standard tool~\citep{tibshirani1996regression}. For the LASSO, approximate optimization techniques exist that have a computational complexity of $\mathcal{O}(p^2 n)$ \citep{efron2004lars}, which is of the same order as ordinary least squares. In this case, we reformulate
%
\eqref{equ:optimization} as
\begin{equation*}
    \Deltaest^{\ell^1} \; \gets \; \mathrm{arg} \; \min_{\bm{\Delta} \in \mathbb{R}^p} \; \left\{ \norm{ \mathbf{r} \; + \; \XI \bm{\Delta} }_2^2 \; + \; \lambda_{\ell^1} \norm{\bm{\Delta}}_1\right\},
\end{equation*}
for some $\lambda_{\ell^1} > 0$, and where $\norm{\; \cdot \;}_1$ denotes the $\ell^1$-norm. We refer to the weight matrix obtained by using $\Deltaest^{\ell^1}$ in place of $\Deltaest$ in~\cref{eq:opt_matrix_estimator} as~$\Wsparse$. 
%
%
%
%

\section{Experiments}\label{sec:experiments}
\begin{table*}[t]
    \centering
    \caption{\textbf{Mean squared error for the causal effect parameter $\alphab$ using various weighting schemes for different types of confounding.} 
    %
    The standard plug-in optimal weight matrix estimator $\Wopthat$ generally does not perform well, while $\Wweak$ and $\Wsparse$, which benefit from prior knowledge, outperform prior work. Note that $\Wopt$ is an oracle that is generally not computable in practice. Numbers correspond to mean $\pm$ std.\ dev.\ over 1000 runs; the best method is highlighted in bold.}
    \label{tab:sim_experiment_outcomes}
    \resizebox{\textwidth}{!}{
    \small
    \begin{tabular}{ c  c  c  c  c  c  c  c  c }
        \toprule
         & & $\wrose$ & $\WI$ & $\WP$ & $\Wopthat$ & $\Wsparse$ & $\Wweak$ & $\textcolor{gray}{\Wopt}$ \\
        \midrule
         \multirow{2}{1cm}{spread conf.} & $\gamma = 1$ & $\mathbf{0.07}\pm0.02$ & $0.21\pm 0.06$ & $\mathbf{0.07} \pm 0.01$ & $0.21 \pm 0.06$ & $0.10\pm0.04$ & $0.08\pm0.03$ & \textcolor{gray}{$0.04\pm0.01$} \\
         \cline{2-9}
         &$\gamma = 5$ & $0.89\pm0.20$ & $2.79\pm0.78$ & $0.92\pm0.14$ & $2.77\pm0.77$ & $1.11\pm0.42$ & $\mathbf{0.76}\pm0.29$ & $\textcolor{gray}{0.10\pm0.03}$  \\
        \midrule
          \multirow{2}{1cm}{sparse conf.} & $\gamma = 1$ & $0.12\pm0.02$ & $0.21\pm0.06$ & $0.13\pm0.02$  & $0.21\pm0.06$ & $\mathbf{0.10}\pm0.04$ & $0.16\pm0.05$ & \textcolor{gray}{$0.05\pm0.01$}  \\
         \cline{2-9}
          &$\gamma = 5$ & $1.80\pm0.37$ & $2.79\pm0.78$ & $2.42\pm0.24$ & $2.77\pm0.77$ & $\mathbf{0.95} \pm0.48$ &  $2.28\pm0.63$ & \textcolor{gray}{$0.30\pm0.08$}  \\
          \bottomrule
    \end{tabular}
    }
\end{table*}
\begin{figure*}[t]    \centering
    %
%
\begin{tikzpicture}[x=1pt,y=1pt]
    \definecolor{fillColor}{RGB}{255,255,255}
    \path[use as bounding box,fill=fillColor,fill opacity=0.00] (0,0) rectangle (476.98,166.22);
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{255,255,255}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] (  0.00,  0.00) rectangle (476.98,166.22);
    \end{scope}
    \begin{scope}
    \path[clip] (  5.50,  5.50) rectangle (250.66,160.72);
    \definecolor{drawColor}{RGB}{255,255,255}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] (  5.50,  5.50) rectangle (250.66,160.72);
    \end{scope}
    \begin{scope}
    \path[clip] ( 55.94, 61.64) rectangle (137.50,138.65);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] ( 55.94, 61.64) rectangle (137.50,138.65);
    \definecolor{drawColor}{gray}{0.92}
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] ( 55.94, 66.77) --
        (137.50, 66.77);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] ( 55.94, 86.99) --
        (137.50, 86.99);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] ( 55.94,106.32) --
        (137.50,106.32);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] ( 55.94,125.66) --
        (137.50,125.66);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] ( 66.19, 61.64) --
        ( 66.19,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] ( 88.00, 61.64) --
        ( 88.00,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (109.81, 61.64) --
        (109.81,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (131.62, 61.64) --
        (131.62,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 55.94, 76.88) --
        (137.50, 76.88);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 55.94, 97.10) --
        (137.50, 97.10);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 55.94,115.55) --
        (137.50,115.55);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 55.94,135.77) --
        (137.50,135.77);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 77.09, 61.64) --
        ( 77.09,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 98.90, 61.64) --
        ( 98.90,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (120.71, 61.64) --
        (120.71,138.65);
    \definecolor{drawColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] ( 59.64,132.72) --
        ( 68.37,110.09) --
        ( 81.45, 97.42) --
        ( 94.54, 90.33) --
        (107.62, 85.44) --
        (120.71, 81.36) --
        (133.80, 78.16);
    \definecolor{drawColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] ( 59.64,102.34) --
        ( 68.37, 87.52) --
        ( 81.45, 81.23) --
        ( 94.54, 78.37) --
        (107.62, 76.79) --
        (120.71, 75.71) --
        (133.80, 74.94);
    \definecolor{drawColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] ( 59.64,113.00) --
        ( 68.37, 95.53) --
        ( 81.45, 85.47) --
        ( 94.54, 79.72) --
        (107.62, 76.05) --
        (120.71, 72.62) --
        (133.80, 70.14);
    \definecolor{drawColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] ( 59.64,107.87) --
        ( 68.37, 90.54) --
        ( 81.45, 81.32) --
        ( 94.54, 76.22) --
        (107.62, 72.78) --
        (120.71, 70.03) --
        (133.80, 67.64);
    \definecolor{drawColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] ( 59.64,105.79) --
        ( 68.37, 89.30) --
        ( 81.45, 81.77) --
        ( 94.54, 77.19) --
        (107.62, 74.16) --
        (120.71, 71.53) --
        (133.80, 69.34);
    \definecolor{drawColor}{RGB}{0,176,246}
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 59.64,107.87) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 68.37, 90.54) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 81.45, 81.32) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 94.54, 76.22) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (107.62, 72.78) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (120.71, 70.03) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (133.80, 67.64) circle (  1.53);
    \definecolor{drawColor}{RGB}{0,191,125}
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 59.64,113.00) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 68.37, 95.53) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 81.45, 85.47) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 94.54, 79.72) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (107.62, 76.05) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (120.71, 72.62) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (133.80, 70.14) circle (  1.53);
    \definecolor{drawColor}{RGB}{248,118,109}
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 59.64,132.72) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 68.37,110.09) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 81.45, 97.42) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 94.54, 90.33) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (107.62, 85.44) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (120.71, 81.36) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (133.80, 78.16) circle (  1.53);
    \definecolor{drawColor}{RGB}{163,165,0}
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 59.64,102.34) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 68.37, 87.52) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 81.45, 81.23) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 94.54, 78.37) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (107.62, 76.79) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (120.71, 75.71) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (133.80, 74.94) circle (  1.53);
    \definecolor{drawColor}{RGB}{231,107,243}
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 59.64,105.79) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 68.37, 89.30) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 81.45, 81.77) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] ( 94.54, 77.19) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (107.62, 74.16) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (120.71, 71.53) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (133.80, 69.34) circle (  1.53);
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[fill=fillColor,fill opacity=0.20] ( 59.64,135.15) --
        ( 68.37,112.21) --
        ( 81.45, 99.57) --
        ( 94.54, 92.41) --
        (107.62, 87.53) --
        (120.71, 83.49) --
        (133.80, 80.22) --
        (133.80, 75.81) --
        (120.71, 78.92) --
        (107.62, 83.06) --
        ( 94.54, 87.96) --
        ( 81.45, 94.97) --
        ( 68.37,107.67) --
        ( 59.64,129.88) --
        cycle;
    
    \path[] ( 59.64,135.15) --
        ( 68.37,112.21) --
        ( 81.45, 99.57) --
        ( 94.54, 92.41) --
        (107.62, 87.53) --
        (120.71, 83.49) --
        (133.80, 80.22);
    
    \path[] (133.80, 75.81) --
        (120.71, 78.92) --
        (107.62, 83.06) --
        ( 94.54, 87.96) --
        ( 81.45, 94.97) --
        ( 68.37,107.67) --
        ( 59.64,129.88);
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[fill=fillColor,fill opacity=0.20] ( 59.64,104.27) --
        ( 68.37, 88.94) --
        ( 81.45, 82.27) --
        ( 94.54, 79.23) --
        (107.62, 77.55) --
        (120.71, 76.31) --
        (133.80, 75.48) --
        (133.80, 74.37) --
        (120.71, 75.08) --
        (107.62, 76.00) --
        ( 94.54, 77.46) --
        ( 81.45, 80.11) --
        ( 68.37, 85.97) --
        ( 59.64,100.16) --
        cycle;
    
    \path[] ( 59.64,104.27) --
        ( 68.37, 88.94) --
        ( 81.45, 82.27) --
        ( 94.54, 79.23) --
        (107.62, 77.55) --
        (120.71, 76.31) --
        (133.80, 75.48);
    
    \path[] (133.80, 74.37) --
        (120.71, 75.08) --
        (107.62, 76.00) --
        ( 94.54, 77.46) --
        ( 81.45, 80.11) --
        ( 68.37, 85.97) --
        ( 59.64,100.16);
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[fill=fillColor,fill opacity=0.20] ( 59.64,116.73) --
        ( 68.37, 98.54) --
        ( 81.45, 88.20) --
        ( 94.54, 82.20) --
        (107.62, 78.59) --
        (120.71, 75.06) --
        (133.80, 72.42) --
        (133.80, 67.50) --
        (120.71, 69.76) --
        (107.62, 73.05) --
        ( 94.54, 76.81) --
        ( 81.45, 82.21) --
        ( 68.37, 91.85) --
        ( 59.64,108.20) --
        cycle;
    
    \path[] ( 59.64,116.73) --
        ( 68.37, 98.54) --
        ( 81.45, 88.20) --
        ( 94.54, 82.20) --
        (107.62, 78.59) --
        (120.71, 75.06) --
        (133.80, 72.42);
    
    \path[] (133.80, 67.50) --
        (120.71, 69.76) --
        (107.62, 73.05) --
        ( 94.54, 76.81) --
        ( 81.45, 82.21) --
        ( 68.37, 91.85) --
        ( 59.64,108.20);
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[fill=fillColor,fill opacity=0.20] ( 59.64,111.90) --
        ( 68.37, 93.54) --
        ( 81.45, 83.93) --
        ( 94.54, 78.67) --
        (107.62, 75.10) --
        (120.71, 72.31) --
        (133.80, 69.82) --
        (133.80, 65.14) --
        (120.71, 67.38) --
        (107.62, 70.08) --
        ( 94.54, 73.35) --
        ( 81.45, 78.23) --
        ( 68.37, 86.89) --
        ( 59.64,102.54) --
        cycle;
    
    \path[] ( 59.64,111.90) --
        ( 68.37, 93.54) --
        ( 81.45, 83.93) --
        ( 94.54, 78.67) --
        (107.62, 75.10) --
        (120.71, 72.31) --
        (133.80, 69.82);
    
    \path[] (133.80, 65.14) --
        (120.71, 67.38) --
        (107.62, 70.08) --
        ( 94.54, 73.35) --
        ( 81.45, 78.23) --
        ( 68.37, 86.89) --
        ( 59.64,102.54);
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[fill=fillColor,fill opacity=0.20] ( 59.64,109.44) --
        ( 68.37, 91.29) --
        ( 81.45, 83.41) --
        ( 94.54, 78.65) --
        (107.62, 75.60) --
        (120.71, 72.97) --
        (133.80, 70.72) --
        (133.80, 67.84) --
        (120.71, 69.97) --
        (107.62, 72.59) --
        ( 94.54, 75.60) --
        ( 81.45, 79.95) --
        ( 68.37, 87.03) --
        ( 59.64,101.12) --
        cycle;
    
    \path[] ( 59.64,109.44) --
        ( 68.37, 91.29) --
        ( 81.45, 83.41) --
        ( 94.54, 78.65) --
        (107.62, 75.60) --
        (120.71, 72.97) --
        (133.80, 70.72);
    
    \path[] (133.80, 67.84) --
        (120.71, 69.97) --
        (107.62, 72.59) --
        ( 94.54, 75.60) --
        ( 81.45, 79.95) --
        ( 68.37, 87.03) --
        ( 59.64,101.12);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round] ( 55.94, 61.64) rectangle (137.50,138.65);
    \end{scope}
    \begin{scope}
    \path[clip] (163.59, 61.64) rectangle (245.16,138.65);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (163.59, 61.64) rectangle (245.16,138.65);
    \definecolor{drawColor}{gray}{0.92}
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (163.59, 80.25) --
        (245.16, 80.25);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (163.59,100.39) --
        (245.16,100.39);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (163.59,120.54) --
        (245.16,120.54);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (173.85, 61.64) --
        (173.85,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (195.65, 61.64) --
        (195.65,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (217.46, 61.64) --
        (217.46,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (239.27, 61.64) --
        (239.27,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (163.59, 69.72) --
        (245.16, 69.72);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (163.59, 90.78) --
        (245.16, 90.78);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (163.59,110.00) --
        (245.16,110.00);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (163.59,131.07) --
        (245.16,131.07);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (184.75, 61.64) --
        (184.75,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (206.56, 61.64) --
        (206.56,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (228.37, 61.64) --
        (228.37,138.65);
    \definecolor{drawColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (167.30,132.61) --
        (176.03,108.75) --
        (189.11, 95.85) --
        (202.20, 88.23) --
        (215.28, 83.09) --
        (228.37, 79.05) --
        (241.46, 75.74);
    \definecolor{drawColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (167.30, 97.93) --
        (176.03, 87.48) --
        (189.11, 83.91) --
        (202.20, 82.50) --
        (215.28, 81.77) --
        (228.37, 81.29) --
        (241.46, 81.00);
    \definecolor{drawColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (167.30,108.16) --
        (176.03, 92.42) --
        (189.11, 83.46) --
        (202.20, 77.82) --
        (215.28, 74.19) --
        (228.37, 71.15) --
        (241.46, 68.89);
    \definecolor{drawColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (167.30, 98.76) --
        (176.03, 85.43) --
        (189.11, 78.77) --
        (202.20, 74.58) --
        (215.28, 71.70) --
        (228.37, 69.46) --
        (241.46, 67.41);
    \definecolor{drawColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (167.30, 98.29) --
        (176.03, 88.16) --
        (189.11, 82.19) --
        (202.20, 77.74) --
        (215.28, 74.55) --
        (228.37, 71.84) --
        (241.46, 69.56);
    \definecolor{drawColor}{RGB}{0,176,246}
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (167.30, 98.76) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (176.03, 85.43) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (189.11, 78.77) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (202.20, 74.58) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (215.28, 71.70) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (228.37, 69.46) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (241.46, 67.41) circle (  1.53);
    \definecolor{drawColor}{RGB}{0,191,125}
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (167.30,108.16) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (176.03, 92.42) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (189.11, 83.46) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (202.20, 77.82) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (215.28, 74.19) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (228.37, 71.15) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (241.46, 68.89) circle (  1.53);
    \definecolor{drawColor}{RGB}{248,118,109}
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (167.30,132.61) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (176.03,108.75) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (189.11, 95.85) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (202.20, 88.23) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (215.28, 83.09) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (228.37, 79.05) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (241.46, 75.74) circle (  1.53);
    \definecolor{drawColor}{RGB}{163,165,0}
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (167.30, 97.93) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (176.03, 87.48) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (189.11, 83.91) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (202.20, 82.50) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (215.28, 81.77) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (228.37, 81.29) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (241.46, 81.00) circle (  1.53);
    \definecolor{drawColor}{RGB}{231,107,243}
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (167.30, 98.29) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (176.03, 88.16) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (189.11, 82.19) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (202.20, 77.74) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (215.28, 74.55) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (228.37, 71.84) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (241.46, 69.56) circle (  1.53);
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[fill=fillColor,fill opacity=0.20] (167.30,135.15) --
        (176.03,110.99) --
        (189.11, 98.16) --
        (202.20, 90.39) --
        (215.28, 85.20) --
        (228.37, 81.27) --
        (241.46, 77.89) --
        (241.46, 73.29) --
        (228.37, 76.50) --
        (215.28, 80.68) --
        (202.20, 85.77) --
        (189.11, 93.20) --
        (176.03,106.18) --
        (167.30,129.63) --
        cycle;
    
    \path[] (167.30,135.15) --
        (176.03,110.99) --
        (189.11, 98.16) --
        (202.20, 90.39) --
        (215.28, 85.20) --
        (228.37, 81.27) --
        (241.46, 77.89);
    
    \path[] (241.46, 73.29) --
        (228.37, 76.50) --
        (215.28, 80.68) --
        (202.20, 85.77) --
        (189.11, 93.20) --
        (176.03,106.18) --
        (167.30,129.63);
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[fill=fillColor,fill opacity=0.20] (167.30, 99.47) --
        (176.03, 88.37) --
        (189.11, 84.50) --
        (202.20, 82.96) --
        (215.28, 82.14) --
        (228.37, 81.62) --
        (241.46, 81.27) --
        (241.46, 80.72) --
        (228.37, 80.95) --
        (215.28, 81.38) --
        (202.20, 82.02) --
        (189.11, 83.31) --
        (176.03, 86.54) --
        (167.30, 96.24) --
        cycle;
    
    \path[] (167.30, 99.47) --
        (176.03, 88.37) --
        (189.11, 84.50) --
        (202.20, 82.96) --
        (215.28, 82.14) --
        (228.37, 81.62) --
        (241.46, 81.27);
    
    \path[] (241.46, 80.72) --
        (228.37, 80.95) --
        (215.28, 81.38) --
        (202.20, 82.02) --
        (189.11, 83.31) --
        (176.03, 86.54) --
        (167.30, 96.24);
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[fill=fillColor,fill opacity=0.20] (167.30,112.43) --
        (176.03, 95.47) --
        (189.11, 86.25) --
        (202.20, 80.30) --
        (215.28, 76.46) --
        (228.37, 73.49) --
        (241.46, 71.12) --
        (241.46, 66.34) --
        (228.37, 68.45) --
        (215.28, 71.58) --
        (202.20, 74.94) --
        (189.11, 80.14) --
        (176.03, 88.72) --
        (167.30,102.49) --
        cycle;
    
    \path[] (167.30,112.43) --
        (176.03, 95.47) --
        (189.11, 86.25) --
        (202.20, 80.30) --
        (215.28, 76.46) --
        (228.37, 73.49) --
        (241.46, 71.12);
    
    \path[] (241.46, 66.34) --
        (228.37, 68.45) --
        (215.28, 71.58) --
        (202.20, 74.94) --
        (189.11, 80.14) --
        (176.03, 88.72) --
        (167.30,102.49);
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[fill=fillColor,fill opacity=0.20] (167.30,103.41) --
        (176.03, 88.34) --
        (189.11, 81.39) --
        (202.20, 76.87) --
        (215.28, 73.78) --
        (228.37, 71.61) --
        (241.46, 69.42) --
        (241.46, 65.14) --
        (228.37, 67.01) --
        (215.28, 69.35) --
        (202.20, 71.94) --
        (189.11, 75.68) --
        (176.03, 81.94) --
        (167.30, 92.43) --
        cycle;
    
    \path[] (167.30,103.41) --
        (176.03, 88.34) --
        (189.11, 81.39) --
        (202.20, 76.87) --
        (215.28, 73.78) --
        (228.37, 71.61) --
        (241.46, 69.42);
    
    \path[] (241.46, 65.14) --
        (228.37, 67.01) --
        (215.28, 69.35) --
        (202.20, 71.94) --
        (189.11, 75.68) --
        (176.03, 81.94) --
        (167.30, 92.43);
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[fill=fillColor,fill opacity=0.20] (167.30,103.70) --
        (176.03, 89.92) --
        (189.11, 83.72) --
        (202.20, 79.09) --
        (215.28, 75.90) --
        (228.37, 73.42) --
        (241.46, 71.09) --
        (241.46, 67.88) --
        (228.37, 70.10) --
        (215.28, 73.09) --
        (202.20, 76.26) --
        (189.11, 80.51) --
        (176.03, 86.21) --
        (167.30, 90.42) --
        cycle;
    
    \path[] (167.30,103.70) --
        (176.03, 89.92) --
        (189.11, 83.72) --
        (202.20, 79.09) --
        (215.28, 75.90) --
        (228.37, 73.42) --
        (241.46, 71.09);
    
    \path[] (241.46, 67.88) --
        (228.37, 70.10) --
        (215.28, 73.09) --
        (202.20, 76.26) --
        (189.11, 80.51) --
        (176.03, 86.21) --
        (167.30, 90.42);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round] (163.59, 61.64) rectangle (245.16,138.65);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (158.64, 66.68) {0.3};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (158.64, 87.75) {1.0};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (158.64,106.97) {3.0};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (158.64,128.04) {10.0};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (160.84, 69.72) --
        (163.59, 69.72);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (160.84, 90.78) --
        (163.59, 90.78);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (160.84,110.00) --
        (163.59,110.00);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (160.84,131.07) --
        (163.59,131.07);
    \end{scope}
    \begin{scope}
    \path[clip] ( 55.94,138.65) rectangle (137.50,155.22);
    \definecolor{drawColor}{gray}{0.20}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] ( 55.94,138.65) rectangle (137.50,155.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 96.72,143.90) {$\gamma = 1$};
    \end{scope}
    \begin{scope}
    \path[clip] (163.59,138.65) rectangle (245.16,155.22);
    \definecolor{drawColor}{gray}{0.20}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] (163.59,138.65) rectangle (245.16,155.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (204.38,143.90) {$\gamma = 5$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 77.09, 58.89) --
        ( 77.09, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 98.90, 58.89) --
        ( 98.90, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (120.71, 58.89) --
        (120.71, 61.64);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 77.09, 50.63) {500};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 98.90, 50.63) {1000};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (120.71, 50.63) {1500};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (184.75, 58.89) --
        (184.75, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (206.56, 58.89) --
        (206.56, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (228.37, 58.89) --
        (228.37, 61.64);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (184.75, 50.63) {500};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (206.56, 50.63) {1000};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (228.37, 50.63) {1500};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 50.99, 73.85) {0.03};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 50.99, 94.07) {0.10};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 50.99,112.52) {0.30};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at ( 50.99,132.74) {1.00};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 53.19, 76.88) --
        ( 55.94, 76.88);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 53.19, 97.10) --
        ( 55.94, 97.10);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 53.19,115.55) --
        ( 55.94,115.55);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] ( 53.19,135.77) --
        ( 55.94,135.77);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  1.10] at (150.55, 38.59) {$m$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,rotate= 90.00,anchor=base,inner sep=0pt, outer sep=0pt, scale=  1.10] at ( 18.58,100.14) {$\mbox{MSE}$};
    
    \node[text=drawColor,rotate= 90.00,anchor=base,inner sep=0pt, outer sep=0pt, scale=  1.10] at ( 30.46,100.14) {};
    \end{scope}
    \begin{scope}
    \path[clip] (250.66,  5.50) rectangle (471.48,160.72);
    \definecolor{drawColor}{RGB}{255,255,255}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] (250.66,  5.50) rectangle (471.48,160.72);
    \end{scope}
    \begin{scope}
    \path[clip] (276.75, 61.64) rectangle (360.52,138.65);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (276.75, 61.64) rectangle (360.52,138.65);
    \definecolor{drawColor}{gray}{0.92}
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (276.75, 73.78) --
        (360.52, 73.78);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (276.75,102.12) --
        (360.52,102.12);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (276.75,130.47) --
        (360.52,130.47);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (304.70, 61.64) --
        (304.70,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (332.29, 61.64) --
        (332.29,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (359.88, 61.64) --
        (359.88,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (276.75, 61.75) --
        (360.52, 61.75);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (276.75, 85.81) --
        (360.52, 85.81);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (276.75,118.44) --
        (360.52,118.44);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (290.28, 61.64) --
        (290.28,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (319.13, 61.64) --
        (319.13,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (345.45, 61.64) --
        (345.45,138.65);
    \definecolor{drawColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (280.56,127.74) --
        (306.89,128.34) --
        (319.13,128.65) --
        (327.19,129.13) --
        (333.21,128.49) --
        (340.11,128.51) --
        (345.45,128.62) --
        (349.82,128.14) --
        (353.51,128.28) --
        (356.71,128.90);
    \definecolor{drawColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (280.56,118.65) --
        (306.89,104.82) --
        (319.13, 96.05) --
        (327.19, 89.48) --
        (333.21, 84.91) --
        (340.11, 81.79) --
        (345.45, 79.40) --
        (349.82, 77.53) --
        (353.51, 76.98) --
        (356.71, 73.84);
    \definecolor{drawColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (280.56,124.30) --
        (306.89,112.92) --
        (319.13,106.32) --
        (327.19,101.38) --
        (333.21, 98.35) --
        (340.11, 96.12) --
        (345.45, 94.81) --
        (349.82, 92.88) --
        (353.51, 92.36) --
        (356.71, 89.90);
    \definecolor{drawColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (280.56,123.52) --
        (306.89,108.91) --
        (319.13, 99.63) --
        (327.19, 93.70) --
        (333.21, 88.89) --
        (340.11, 85.61) --
        (345.45, 82.76) --
        (349.82, 80.16) --
        (353.51, 79.38) --
        (356.71, 74.62);
    \definecolor{drawColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (280.56,117.81) --
        (306.89,102.78) --
        (319.13, 95.05) --
        (327.19, 90.69) --
        (333.21, 86.51) --
        (340.11, 84.35) --
        (345.45, 82.49) --
        (349.82, 80.14) --
        (353.51, 80.23) --
        (356.71, 76.55);
    \definecolor{drawColor}{RGB}{0,176,246}
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (280.56,123.52) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (306.89,108.91) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (319.13, 99.63) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (327.19, 93.70) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (333.21, 88.89) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (340.11, 85.61) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (345.45, 82.76) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (349.82, 80.16) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (353.51, 79.38) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (356.71, 74.62) circle (  1.53);
    \definecolor{drawColor}{RGB}{0,191,125}
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (280.56,124.30) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (306.89,112.92) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (319.13,106.32) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (327.19,101.38) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (333.21, 98.35) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (340.11, 96.12) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (345.45, 94.81) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (349.82, 92.88) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (353.51, 92.36) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (356.71, 89.90) circle (  1.53);
    \definecolor{drawColor}{RGB}{248,118,109}
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (280.56,127.74) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (306.89,128.34) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (319.13,128.65) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (327.19,129.13) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (333.21,128.49) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (340.11,128.51) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (345.45,128.62) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (349.82,128.14) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (353.51,128.28) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (356.71,128.90) circle (  1.53);
    \definecolor{drawColor}{RGB}{163,165,0}
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (280.56,118.65) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (306.89,104.82) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (319.13, 96.05) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (327.19, 89.48) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (333.21, 84.91) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (340.11, 81.79) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (345.45, 79.40) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (349.82, 77.53) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (353.51, 76.98) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (356.71, 73.84) circle (  1.53);
    \definecolor{drawColor}{RGB}{231,107,243}
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (280.56,117.81) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (306.89,102.78) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (319.13, 95.05) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (327.19, 90.69) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (333.21, 86.51) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (340.11, 84.35) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (345.45, 82.49) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (349.82, 80.14) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (353.51, 80.23) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (356.71, 76.55) circle (  1.53);
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[fill=fillColor,fill opacity=0.20] (280.56,133.71) --
        (306.89,134.27) --
        (319.13,134.55) --
        (327.19,135.15) --
        (333.21,134.46) --
        (340.11,134.52) --
        (345.45,134.84) --
        (349.82,134.01) --
        (353.51,134.35) --
        (356.71,134.88) --
        (356.71,122.06) --
        (353.51,121.31) --
        (349.82,121.42) --
        (345.45,121.45) --
        (340.11,121.62) --
        (333.21,121.65) --
        (327.19,122.22) --
        (319.13,121.90) --
        (306.89,121.56) --
        (280.56,120.90) --
        cycle;
    
    \path[] (280.56,133.71) --
        (306.89,134.27) --
        (319.13,134.55) --
        (327.19,135.15) --
        (333.21,134.46) --
        (340.11,134.52) --
        (345.45,134.84) --
        (349.82,134.01) --
        (353.51,134.35) --
        (356.71,134.88);
    
    \path[] (356.71,122.06) --
        (353.51,121.31) --
        (349.82,121.42) --
        (345.45,121.45) --
        (340.11,121.62) --
        (333.21,121.65) --
        (327.19,122.22) --
        (319.13,121.90) --
        (306.89,121.56) --
        (280.56,120.90);
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[fill=fillColor,fill opacity=0.20] (280.56,124.55) --
        (306.89,110.45) --
        (319.13,101.00) --
        (327.19, 93.87) --
        (333.21, 89.25) --
        (340.11, 85.57) --
        (345.45, 83.05) --
        (349.82, 80.67) --
        (353.51, 79.92) --
        (356.71, 76.05) --
        (356.71, 71.52) --
        (353.51, 73.83) --
        (349.82, 74.16) --
        (345.45, 75.44) --
        (340.11, 77.69) --
        (333.21, 80.12) --
        (327.19, 84.65) --
        (319.13, 90.51) --
        (306.89, 98.43) --
        (280.56,111.91) --
        cycle;
    
    \path[] (280.56,124.55) --
        (306.89,110.45) --
        (319.13,101.00) --
        (327.19, 93.87) --
        (333.21, 89.25) --
        (340.11, 85.57) --
        (345.45, 83.05) --
        (349.82, 80.67) --
        (353.51, 79.92) --
        (356.71, 76.05);
    
    \path[] (356.71, 71.52) --
        (353.51, 73.83) --
        (349.82, 74.16) --
        (345.45, 75.44) --
        (340.11, 77.69) --
        (333.21, 80.12) --
        (327.19, 84.65) --
        (319.13, 90.51) --
        (306.89, 98.43) --
        (280.56,111.91);
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[fill=fillColor,fill opacity=0.20] (280.56,130.34) --
        (306.89,119.74) --
        (319.13,113.36) --
        (327.19,108.73) --
        (333.21,105.87) --
        (340.11,103.68) --
        (345.45,102.60) --
        (349.82,100.57) --
        (353.51,100.15) --
        (356.71, 98.01) --
        (356.71, 80.08) --
        (353.51, 83.01) --
        (349.82, 83.68) --
        (345.45, 85.47) --
        (340.11, 87.12) --
        (333.21, 89.40) --
        (327.19, 92.66) --
        (319.13, 98.03) --
        (306.89,104.96) --
        (280.56,117.37) --
        cycle;
    
    \path[] (280.56,130.34) --
        (306.89,119.74) --
        (319.13,113.36) --
        (327.19,108.73) --
        (333.21,105.87) --
        (340.11,103.68) --
        (345.45,102.60) --
        (349.82,100.57) --
        (353.51,100.15) --
        (356.71, 98.01);
    
    \path[] (356.71, 80.08) --
        (353.51, 83.01) --
        (349.82, 83.68) --
        (345.45, 85.47) --
        (340.11, 87.12) --
        (333.21, 89.40) --
        (327.19, 92.66) --
        (319.13, 98.03) --
        (306.89,104.96) --
        (280.56,117.37);
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[fill=fillColor,fill opacity=0.20] (280.56,129.51) --
        (306.89,115.44) --
        (319.13,106.47) --
        (327.19,100.91) --
        (333.21, 96.19) --
        (340.11, 92.87) --
        (345.45, 90.73) --
        (349.82, 87.62) --
        (353.51, 86.92) --
        (356.71, 82.50) --
        (356.71, 65.14) --
        (353.51, 70.39) --
        (349.82, 71.30) --
        (345.45, 73.16) --
        (340.11, 77.04) --
        (333.21, 80.24) --
        (327.19, 85.19) --
        (319.13, 91.64) --
        (306.89,101.33) --
        (280.56,116.67) --
        cycle;
    
    \path[] (280.56,129.51) --
        (306.89,115.44) --
        (319.13,106.47) --
        (327.19,100.91) --
        (333.21, 96.19) --
        (340.11, 92.87) --
        (345.45, 90.73) --
        (349.82, 87.62) --
        (353.51, 86.92) --
        (356.71, 82.50);
    
    \path[] (356.71, 65.14) --
        (353.51, 70.39) --
        (349.82, 71.30) --
        (345.45, 73.16) --
        (340.11, 77.04) --
        (333.21, 80.24) --
        (327.19, 85.19) --
        (319.13, 91.64) --
        (306.89,101.33) --
        (280.56,116.67);
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[fill=fillColor,fill opacity=0.20] (280.56,123.51) --
        (306.89,108.40) --
        (319.13,100.29) --
        (327.19, 95.97) --
        (333.21, 91.68) --
        (340.11, 89.20) --
        (345.45, 87.69) --
        (349.82, 84.90) --
        (353.51, 85.03) --
        (356.71, 81.31) --
        (356.71, 71.26) --
        (353.51, 74.87) --
        (349.82, 74.85) --
        (345.45, 76.65) --
        (340.11, 78.94) --
        (333.21, 80.70) --
        (327.19, 84.75) --
        (319.13, 89.17) --
        (306.89, 96.39) --
        (280.56,111.33) --
        cycle;
    
    \path[] (280.56,123.51) --
        (306.89,108.40) --
        (319.13,100.29) --
        (327.19, 95.97) --
        (333.21, 91.68) --
        (340.11, 89.20) --
        (345.45, 87.69) --
        (349.82, 84.90) --
        (353.51, 85.03) --
        (356.71, 81.31);
    
    \path[] (356.71, 71.26) --
        (353.51, 74.87) --
        (349.82, 74.85) --
        (345.45, 76.65) --
        (340.11, 78.94) --
        (333.21, 80.70) --
        (327.19, 84.75) --
        (319.13, 89.17) --
        (306.89, 96.39) --
        (280.56,111.33);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round] (276.75, 61.64) rectangle (360.52,138.65);
    \end{scope}
    \begin{scope}
    \path[clip] (382.21, 61.64) rectangle (465.98,138.65);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (382.21, 61.64) rectangle (465.98,138.65);
    \definecolor{drawColor}{gray}{0.92}
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (382.21, 78.06) --
        (465.98, 78.06);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (382.21, 95.49) --
        (465.98, 95.49);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (382.21,112.92) --
        (465.98,112.92);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (382.21,121.39) --
        (465.98,121.39);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (410.16, 61.64) --
        (410.16,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (437.75, 61.64) --
        (437.75,138.65);
    
    \path[draw=drawColor,line width= 0.3pt,line join=round] (465.34, 61.64) --
        (465.34,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (382.21, 69.59) --
        (465.98, 69.59);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (382.21, 86.52) --
        (465.98, 86.52);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (382.21,104.46) --
        (465.98,104.46);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (395.74, 61.64) --
        (395.74,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (424.59, 61.64) --
        (424.59,138.65);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (450.91, 61.64) --
        (450.91,138.65);
    \definecolor{drawColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (386.02,127.84) --
        (412.35,128.52) --
        (424.59,128.52) --
        (432.65,128.52) --
        (438.67,128.33) --
        (445.57,128.60) --
        (450.91,128.68) --
        (455.28,128.82) --
        (458.97,127.97) --
        (462.17,128.71);
    \definecolor{drawColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (386.02,117.74) --
        (412.35,103.03) --
        (424.59, 95.14) --
        (432.65, 90.48) --
        (438.67, 87.90) --
        (445.57, 87.19) --
        (450.91, 86.68) --
        (455.28, 86.96) --
        (458.97, 87.14) --
        (462.17, 88.94);
    \definecolor{drawColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (386.02,105.73) --
        (412.35, 94.92) --
        (424.59, 93.20) --
        (432.65, 91.29) --
        (438.67, 90.60) --
        (445.57, 91.27) --
        (450.91, 90.71) --
        (455.28, 90.08) --
        (458.97, 89.58) --
        (462.17, 89.44);
    \definecolor{drawColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (386.02, 99.50) --
        (412.35, 83.36) --
        (424.59, 79.34) --
        (432.65, 77.49) --
        (438.67, 76.18) --
        (445.57, 76.17) --
        (450.91, 74.96) --
        (455.28, 74.98) --
        (458.97, 74.25) --
        (462.17, 73.66);
    \definecolor{drawColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (386.02, 98.73) --
        (412.35, 88.66) --
        (424.59, 86.50) --
        (432.65, 85.76) --
        (438.67, 85.03) --
        (445.57, 85.06) --
        (450.91, 84.75) --
        (455.28, 84.21) --
        (458.97, 84.22) --
        (462.17, 83.88);
    \definecolor{drawColor}{RGB}{0,176,246}
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (386.02, 99.50) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (412.35, 83.36) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (424.59, 79.34) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (432.65, 77.49) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (438.67, 76.18) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (445.57, 76.17) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (450.91, 74.96) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (455.28, 74.98) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (458.97, 74.25) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (462.17, 73.66) circle (  1.53);
    \definecolor{drawColor}{RGB}{0,191,125}
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (386.02,105.73) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (412.35, 94.92) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (424.59, 93.20) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (432.65, 91.29) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (438.67, 90.60) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (445.57, 91.27) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (450.91, 90.71) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (455.28, 90.08) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (458.97, 89.58) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (462.17, 89.44) circle (  1.53);
    \definecolor{drawColor}{RGB}{248,118,109}
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (386.02,127.84) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (412.35,128.52) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (424.59,128.52) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (432.65,128.52) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (438.67,128.33) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (445.57,128.60) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (450.91,128.68) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (455.28,128.82) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (458.97,127.97) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (462.17,128.71) circle (  1.53);
    \definecolor{drawColor}{RGB}{163,165,0}
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (386.02,117.74) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (412.35,103.03) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (424.59, 95.14) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (432.65, 90.48) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (438.67, 87.90) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (445.57, 87.19) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (450.91, 86.68) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (455.28, 86.96) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (458.97, 87.14) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (462.17, 88.94) circle (  1.53);
    \definecolor{drawColor}{RGB}{231,107,243}
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (386.02, 98.73) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (412.35, 88.66) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (424.59, 86.50) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (432.65, 85.76) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (438.67, 85.03) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (445.57, 85.06) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (450.91, 84.75) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (455.28, 84.21) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (458.97, 84.22) circle (  1.53);
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (462.17, 83.88) circle (  1.53);
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[fill=fillColor,fill opacity=0.20] (386.02,134.19) --
        (412.35,134.85) --
        (424.59,134.88) --
        (432.65,134.84) --
        (438.67,134.68) --
        (445.57,135.02) --
        (450.91,135.15) --
        (455.28,135.01) --
        (458.97,134.31) --
        (462.17,134.95) --
        (462.17,121.58) --
        (458.97,120.71) --
        (455.28,121.76) --
        (450.91,121.26) --
        (445.57,121.25) --
        (438.67,121.06) --
        (432.65,121.28) --
        (424.59,121.23) --
        (412.35,121.29) --
        (386.02,120.58) --
        cycle;
    
    \path[] (386.02,134.19) --
        (412.35,134.85) --
        (424.59,134.88) --
        (432.65,134.84) --
        (438.67,134.68) --
        (445.57,135.02) --
        (450.91,135.15) --
        (455.28,135.01) --
        (458.97,134.31) --
        (462.17,134.95);
    
    \path[] (462.17,121.58) --
        (458.97,120.71) --
        (455.28,121.76) --
        (450.91,121.26) --
        (445.57,121.25) --
        (438.67,121.06) --
        (432.65,121.28) --
        (424.59,121.23) --
        (412.35,121.29) --
        (386.02,120.58);
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[fill=fillColor,fill opacity=0.20] (386.02,123.99) --
        (412.35,108.74) --
        (424.59, 99.60) --
        (432.65, 94.26) --
        (438.67, 91.10) --
        (445.57, 89.82) --
        (450.91, 88.83) --
        (455.28, 88.84) --
        (458.97, 88.69) --
        (462.17, 89.95) --
        (462.17, 87.91) --
        (458.97, 85.54) --
        (455.28, 85.01) --
        (450.91, 84.44) --
        (445.57, 84.42) --
        (438.67, 84.48) --
        (432.65, 86.41) --
        (424.59, 90.23) --
        (412.35, 96.58) --
        (386.02,110.59) --
        cycle;
    
    \path[] (386.02,123.99) --
        (412.35,108.74) --
        (424.59, 99.60) --
        (432.65, 94.26) --
        (438.67, 91.10) --
        (445.57, 89.82) --
        (450.91, 88.83) --
        (455.28, 88.84) --
        (458.97, 88.69) --
        (462.17, 89.95);
    
    \path[] (462.17, 87.91) --
        (458.97, 85.54) --
        (455.28, 85.01) --
        (450.91, 84.44) --
        (445.57, 84.42) --
        (438.67, 84.48) --
        (432.65, 86.41) --
        (424.59, 90.23) --
        (412.35, 96.58) --
        (386.02,110.59);
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[fill=fillColor,fill opacity=0.20] (386.02,112.93) --
        (412.35,102.71) --
        (424.59,101.39) --
        (432.65, 99.26) --
        (438.67, 98.54) --
        (445.57, 99.19) --
        (450.91, 98.72) --
        (455.28, 98.09) --
        (458.97, 97.54) --
        (462.17, 97.09) --
        (462.17, 80.41) --
        (458.97, 80.13) --
        (455.28, 80.54) --
        (450.91, 81.18) --
        (445.57, 81.87) --
        (438.67, 81.17) --
        (432.65, 81.82) --
        (424.59, 83.40) --
        (412.35, 85.70) --
        (386.02, 97.33) --
        cycle;
    
    \path[] (386.02,112.93) --
        (412.35,102.71) --
        (424.59,101.39) --
        (432.65, 99.26) --
        (438.67, 98.54) --
        (445.57, 99.19) --
        (450.91, 98.72) --
        (455.28, 98.09) --
        (458.97, 97.54) --
        (462.17, 97.09);
    
    \path[] (462.17, 80.41) --
        (458.97, 80.13) --
        (455.28, 80.54) --
        (450.91, 81.18) --
        (445.57, 81.87) --
        (438.67, 81.17) --
        (432.65, 81.82) --
        (424.59, 83.40) --
        (412.35, 85.70) --
        (386.02, 97.33);
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[fill=fillColor,fill opacity=0.20] (386.02,106.63) --
        (412.35, 90.53) --
        (424.59, 87.01) --
        (432.65, 84.99) --
        (438.67, 83.58) --
        (445.57, 83.88) --
        (450.91, 82.72) --
        (455.28, 82.39) --
        (458.97, 81.87) --
        (462.17, 80.95) --
        (462.17, 65.14) --
        (458.97, 65.26) --
        (455.28, 66.27) --
        (450.91, 65.78) --
        (445.57, 67.05) --
        (438.67, 67.49) --
        (432.65, 68.67) --
        (424.59, 70.29) --
        (412.35, 75.00) --
        (386.02, 91.19) --
        cycle;
    
    \path[] (386.02,106.63) --
        (412.35, 90.53) --
        (424.59, 87.01) --
        (432.65, 84.99) --
        (438.67, 83.58) --
        (445.57, 83.88) --
        (450.91, 82.72) --
        (455.28, 82.39) --
        (458.97, 81.87) --
        (462.17, 80.95);
    
    \path[] (462.17, 65.14) --
        (458.97, 65.26) --
        (455.28, 66.27) --
        (450.91, 65.78) --
        (445.57, 67.05) --
        (438.67, 67.49) --
        (432.65, 68.67) --
        (424.59, 70.29) --
        (412.35, 75.00) --
        (386.02, 91.19);
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[fill=fillColor,fill opacity=0.20] (386.02,103.99) --
        (412.35, 92.97) --
        (424.59, 91.00) --
        (432.65, 90.05) --
        (438.67, 89.16) --
        (445.57, 89.42) --
        (450.91, 89.11) --
        (455.28, 88.41) --
        (458.97, 88.56) --
        (462.17, 87.90) --
        (462.17, 79.51) --
        (458.97, 79.47) --
        (455.28, 79.63) --
        (450.91, 79.97) --
        (445.57, 80.30) --
        (438.67, 80.52) --
        (432.65, 81.06) --
        (424.59, 81.56) --
        (412.35, 83.95) --
        (386.02, 92.85) --
        cycle;
    
    \path[] (386.02,103.99) --
        (412.35, 92.97) --
        (424.59, 91.00) --
        (432.65, 90.05) --
        (438.67, 89.16) --
        (445.57, 89.42) --
        (450.91, 89.11) --
        (455.28, 88.41) --
        (458.97, 88.56) --
        (462.17, 87.90);
    
    \path[] (462.17, 79.51) --
        (458.97, 79.47) --
        (455.28, 79.63) --
        (450.91, 79.97) --
        (445.57, 80.30) --
        (438.67, 80.52) --
        (432.65, 81.06) --
        (424.59, 81.56) --
        (412.35, 83.95) --
        (386.02, 92.85);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round] (382.21, 61.64) rectangle (465.98,138.65);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (377.26, 66.56) {0.5};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (377.26, 83.49) {0.7};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (377.26,101.43) {1.0};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (379.46, 69.59) --
        (382.21, 69.59);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (379.46, 86.52) --
        (382.21, 86.52);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (379.46,104.46) --
        (382.21,104.46);
    \end{scope}
    \begin{scope}
    \path[clip] (276.75,138.65) rectangle (360.52,155.22);
    \definecolor{drawColor}{gray}{0.20}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] (276.75,138.65) rectangle (360.52,155.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (318.64,143.90) {$\gamma = 1$};
    \end{scope}
    \begin{scope}
    \path[clip] (382.21,138.65) rectangle (465.98,155.22);
    \definecolor{drawColor}{gray}{0.20}
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round,line cap=round,fill=fillColor] (382.21,138.65) rectangle (465.98,155.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (424.10,143.90) {$\gamma = 5$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (290.28, 58.89) --
        (290.28, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (319.13, 58.89) --
        (319.13, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (345.45, 58.89) --
        (345.45, 61.64);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (290.28, 50.63) {0.3};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (319.13, 50.63) {1.0};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (345.45, 50.63) {3.0};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (395.74, 58.89) --
        (395.74, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (424.59, 58.89) --
        (424.59, 61.64);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (450.91, 58.89) --
        (450.91, 61.64);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (395.74, 50.63) {0.3};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (424.59, 50.63) {1.0};
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  0.88] at (450.91, 50.63) {3.0};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.30}
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (271.80, 58.72) {0.03};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (271.80, 82.78) {0.05};
    
    \node[text=drawColor,anchor=base east,inner sep=0pt, outer sep=0pt, scale=  0.88] at (271.80,115.41) {0.10};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{gray}{0.20}
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (274.00, 61.75) --
        (276.75, 61.75);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (274.00, 85.81) --
        (276.75, 85.81);
    
    \path[draw=drawColor,line width= 0.6pt,line join=round] (274.00,118.44) --
        (276.75,118.44);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base,inner sep=0pt, outer sep=0pt, scale=  1.10] at (371.37, 38.59) {$\frac{n}{m}$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (153.99, 11.00) rectangle (367.93, 25.45);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (159.49, 11.00) rectangle (173.94, 25.45);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (160.94, 18.23) -- (172.50, 18.23);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{248,118,109}
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (166.72, 18.23) circle (  1.53);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{248,118,109}
    
    \path[fill=fillColor,fill opacity=0.20] (160.20, 11.71) rectangle (173.23, 24.74);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (202.23, 11.00) rectangle (216.69, 25.45);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (203.68, 18.23) -- (215.24, 18.23);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{163,165,0}
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (209.46, 18.23) circle (  1.53);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{163,165,0}
    
    \path[fill=fillColor,fill opacity=0.20] (202.94, 11.71) rectangle (215.97, 24.74);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (244.97, 11.00) rectangle (259.43, 25.45);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (246.42, 18.23) -- (257.98, 18.23);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,191,125}
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (252.20, 18.23) circle (  1.53);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{0,191,125}
    
    \path[fill=fillColor,fill opacity=0.20] (245.68, 11.71) rectangle (258.72, 24.74);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (287.71, 11.00) rectangle (302.17, 25.45);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (289.16, 18.23) -- (300.72, 18.23);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,176,246}
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (294.94, 18.23) circle (  1.53);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{0,176,246}
    
    \path[fill=fillColor,fill opacity=0.20] (288.43, 11.71) rectangle (301.46, 24.74);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{255,255,255}
    
    \path[fill=fillColor] (330.46, 11.00) rectangle (344.91, 25.45);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 1.1pt,line join=round] (331.90, 18.23) -- (343.46, 18.23);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{231,107,243}
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[draw=drawColor,line width= 0.4pt,line join=round,line cap=round,fill=fillColor] (337.68, 18.23) circle (  1.53);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{fillColor}{RGB}{231,107,243}
    
    \path[fill=fillColor,fill opacity=0.20] (331.17, 11.71) rectangle (344.20, 24.74);
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale=  0.88] at (179.44, 15.20) {$\mathbf{W}^m_{\textsc{i}}$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale=  0.88] at (222.19, 15.20) {$\mathbf{W}^m_{\textsc{p}}$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale=  0.88] at (264.93, 15.20) {$\widehat{\mathbf{W}}^m_{\ell^1}$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale=  0.88] at (307.67, 15.20) {$\widehat{\mathbf{W}}^m_{\ell^2}$};
    \end{scope}
    \begin{scope}
    \path[clip] (  0.00,  0.00) rectangle (476.98,166.22);
    \definecolor{drawColor}{RGB}{0,0,0}
    
    \node[text=drawColor,anchor=base west,inner sep=0pt, outer sep=0pt, scale=  0.88] at (350.41, 15.20) {$\widehat{w}^m_{\mbox{rm}}$};
    \end{scope}
    \end{tikzpicture}
    \caption{\textbf{Performance for varying dataset sizes  and ratios.}
    \textit{(Left)} All methods improve as the amount of 
    %
    data is increased. More sophisticated weighting schemes outperform the purely interventional ($\WI$) and plug in estimators ($\Wopthat$) (not depicted due to close performance overlap with pure interventional), whereas data pooling ($\WP$) works well only for small $m$ and $\gamma$.
    \textit{(Right)} When keeping $m$ fixed and adding more observational data, $\Wweak$ clearly works best in strongly confounded ($\gamma=5$) settings. MSE and $\frac{n}{m}$ are plotted on a $\text{log}_{10}$ scale. Shaded areas indicate $\pm 0.5$ standard deviations. 
    }
    %
    \label{fig:plots}
\end{figure*}
We investigate the empirical behavior of our proposed matrix weighted estimators in a finite sample setting and compare them with baselines and existing methods through simulations on synthetic data.\footnote{The source code for all experiments is available at: \href{https://github.com/rudolfwilliam/matrix_weighted_linear_estimators}{https://github.com/rudolfwilliam/matrix\_weighted\_linear\_estimators}}
To this end, we consider different experimental settings in which we vary the strength and sparsity of confounding, as well as the ratio and absolute quantity of observational and interventional data. 
%
%

\paragraph{Compared Methods.}
We report the mean squared error attained by the theoretically optimal weight matrix $\Wopt$ from~\eqref{equ:theor_optimal_matrix} as an oracle, as well as the plug-in estimator~$\Wopthat$ thereof from~\eqref{eq:opt_matrix_estimator}, and the regularized regression-based $\Wweak$ and $\Wsparse$ from~\cref{sec:inductive_biases}. 
For the latter two, we choose the regularization hyperparameters $\lambda_{\ell^2}$ and $\lambda_{\ell^1}$ by cross-validation on the interventional data. 
As baselines, we consider only using interventional data~($\WI=\Ib_p$) and data pooling according to $\WP$ from~\eqref{eq:W_data_pooling}.
We also compare to the~\citet{rosenman2020combining} scalar weighting scheme which was proposed for vectors of binary treatment effects and is given by $\Wb=\wrose\Ib_p$ with
\begin{equation*}
    \widehat{w}^m_{\text{rm}} \; := \; \max \left\{ 1 \; - \; \frac{\text{Tr} \left( \widehat{\textbf{Cov}}\left(\alphaI\right) \right)}{\norm{ \alphaI - \alphaO }_2^2}, \; 0 \right\}.
\end{equation*}
%
We emphasize that other commonly used methods for causal effect estimation from observational data such as propensity score matching~\citep{imai2004treatmentregimes} are not applicable, 
%
because they require the relevant confounders to be observed, which is not the case in our setting.
%



\paragraph{General Setup.} In all experiments, we use $p=30$ treatments, a one-dimensional ($d=1$) confounder~$Z$, and unit/isotropic (co)variances: $\sigma_{N_Y}^2= \sigma_{N_Z}^2= 1$, $\Sigmab_{\Nb_\Xb}=\Ib_p$. We sample $\tilde{\Nb}_{\Xb} \sim \mathcal{N}(\mathbf{0}, \COV(\XO))$,
%
%
$\bm{\alpha} \sim \mathcal{N}(\mathbf{0}, 9\mathbf{I}_p)$, and choose $\bb$ and $\gamma$ depending on the settings described below.
Unless otherwise specified, we then draw $m = 300$ interventional and $n = 600$ observational examples from $\Pint$ and $\Pobs$, respectively, and compute estimates of $\bm{\alpha}$ using the different weighting approaches. We repeat this procedure $1000$ times and report the resulting mean and standard deviation of the mean squared error.

%
%
%
%
%
%
\paragraph{Different Types of Confounding.}
In our main experiment, we investigate how estimators perform under different types of confounding encoded by ~\eqref{equ:SA_X} and~\eqref{eq:outcome}, specifically by the parameters $\bb\in\RR^p$ and $\gamma\in \RR$ (for a scalar confounder $Z$). 
For \textit{spread} confounding, we sample $\bb \sim \mathcal{N}(\mathbf{0}, \mathbf{I}_p)$ such that the confounder affects all treatment variables almost surely.
For \textit{sparse} confounding, we sample $b^{(k)}\sim \Ncal(0,1)$ for $k=1,...,5$, and $b^{(k)}=0$ otherwise,  such that only the first five treatments are confounded.
In both cases, we investigate $\gamma\in\{1,5\}$ which controls the strength of $Z\to~Y$ and thus the extent to which $\bm{\Delta} = 0$ is violated.
%

%
%
%
%

%

%
%

\paragraph{Main Results.} The results are presented in~\cref{tab:sim_experiment_outcomes}.
We find that our regularized estimators generally perform well, particularly when the underlying assumptions are satisfied: under sparse confounding $\Wsparse$ works best, and in the spread confounding case $\Wweak$ is only narrowly outperformed by $\wrose$ and $\WP$ when $\gamma=1$. Data pooling works relatively well when $\gamma=1$ (compared to $\gamma=5$) where the  violation of the identically distributed assumption is weak and the variance from estimating unknown quantities is not compensated by the bias reduction.
In contrast, both the purely interventional approach $\WI$ and the plug-in estimator $\Wopthat$ do not perform very well in this finite sample setting due to high variance, as explained in~\cref{sec:inductive_biases}.
%
%
%
%
%
%
%
%
%
%
%
%

\paragraph{Varying Data Set Sizes and Ratios.}
In~\cref{fig:plots}, we investigate how the different estimators behave across different data set sizes and ratios for the spread confounding setting.
In the left two plots, we vary the amount of interventional data $m$ while fixing the amount of observational data to $n=3m$. 
The results confirm our theoretical results: For small data set sizes, data pooling is a worthwhile alternative to more sophisticated weights, in particular if the violation against the assumption of identical distribution is minor ($\gamma = 1$). However, for large enough data set sizes, the approaches from both previous work and ours achieve a better score. Particularly, we see that $\Wweak$ outperforms all other weights in both scenarios for large enough data sets.

%
In the right two plots, we keep $m=500$ fixed and change~$n$ and thus the ratio of interventional to observational data. Unsurprisingly, we find that the mean squared error of $\WI$ remains constant. For strong confounding ($\gamma = 5$), we see that $\Wweak$ adapts best with a considerable margin: Unlike $\wrose$, it explicitly takes into account (an estimate of) the covariance structure of $\alphaO$  in constructing the weight matrix.
%

%

%

%
\section{Discussion}\label{sec:other_fields}
\paragraph{Connection to Transfer Learning.}\label{sec:transfer_learning}
Our setting bears resemblance to transfer and multi-task learning~\citep{thrun1995learning,caruana1997multitask}, specifically to supervised domain adaptation, which aims to leverage knowledge from a source domain to improve a model in a target domain, for which typically much less data is available.
In our case, we aim to use the source model  $\alphaO$, learned by estimating $\EE[Y|\Xb=\xb]$ in the observational setting, to improve our (high-variance) target model $\alphaI$ of $\EE[Y|\text{do}(\Xb\leftarrow\xb)]$.
Transfer learning 
%
can only work if the domains are sufficiently similar, resulting in numerous approaches leveraging different assumptions about shared components~\citep{quinonero2008dataset}. These assumptions are often phrased in causal terms~\citep{scholkopf2012causal,zhang2013domain,gong2016domain,rojas2018invariant}.
Similarly, our observational (source) and interventional (target) domains share the same causal model and only differ in the treatment assignment mechanisms~\eqref{equ:SA_X} and~\eqref{equ:replaced_SA}. Still, the bias in~\eqref{equ:Delta} can in theory be arbitrary large, and our methods from~\cref{sec:inductive_biases} implicitly rely on it being small or sparse.

\paragraph{Beyond Linear Regression.}
Some of our derivations and theoretical results rely on the fact that the confounding bias in~\eqref{eq:pert_model} is linear in $\xb$. 
For the class of \textit{linear} SCMs~\eqref{equ:multivarSCMconfounder}--\eqref{eq:outcome}, Gaussianity is necessary and sufficient\footnote{Note $\EE[Y|\Xb]=\gammab^\top\EE[\Zb|\Xb]+\alphab^\top\Xb$ and $\EE[\Zb|\Xb]$ is linear in $\Xb$ only in the Gaussian case~\citep[][Thm.~4.2]{peters2017elements}.} for this condition to hold, but it may also hold for more general classes of SCMs. 
For binary treatments $\Xb\in\{0,1\}^p$, in particular, it is always possible to write the difference between the biased and unbiased average treatment effect estimates using a constant offset $\Deltab$ akin to \eqref{eq:delta_plugin}, irrespective of the confounding relationship.\footnote{Specifically, we have $\Deltab= \EE [Y | \Xb = \mathbf{1} ] - \EE [Y | \Xb = \mathbf{0}] - \left( \EE [Y | \text{do}(\Xb \leftarrow \mathbf{1})] - \EE [Y | \text{do}(\Xb \leftarrow \mathbf{0}) ] \right)$.}
%
%
%
Future work may thus investigate nonlinear extensions, e.g., by drawing inspiration from semi-parametrics~\citep{robins1995semiparametric}, doubly robust estimation~\citep{bang2005doubly}, and debiased machine learning~\citep{chernozhukov2018double}.
%
%
%
%
%
%

\paragraph{Incorporating Covariates.}
%
%
%
%
%
Our current formulation does not \textit{explicitly} account for
%
observed confounders, or pre-treatment covariates, which need to be adjusted for in the observational setting to avoid introducing further bias. 
%
In principle, such covariates can simply be included in~$\Xb$, as different treatment components $X_i$ are allowed to be dependent.
However, this may result in high-dimensional treatments and thus render full randomization in~\eqref{equ:replaced_SA} unrealistic.
Other covariates, while unproblematic with regard to bias, may help further reduce variance~\citep{henckel2022graphical}. 
Extending our framework to incorporate different types of covariates is thus a worthwhile future direction.
%
%

%

%
%
%
%
%
%
%
%
%
%
%

%

%


\section{Conclusion}\label{sec:conclusion}
%
In the present work, we have introduced a new class of matrix weighted linear estimators for learning causal effects of continuous treatments from finite observational and interventional data. 
Here, our focus has been on optimizing statistical efficiency, which complements the vast causal inference literature on identification from heterogeneous data. 
Our estimators are connected to classical ideas from shrinkage estimation applied to causal learning and provide a unifying account of data pooling and ridge regression, which emerge as special cases.
We show that our estimators are theoretically grounded and compare favorably to baselines and prior work in simulations.
While we restricted our analysis to linear models for now, we hope that the insights and methods developed here will also be useful for a broader class of causal models and transfer learning problems.

\begin{acknowledgements}
    We thank the anonymous reviewers for useful comments and suggestions that helped improve the manuscript.

    We thank the Branco Weiss Fellowship, administered by ETH Zurich, for the support.
    This work was further supported by the T\"ubingen AI Center and by the German Research Foundation (DFG) under Germany’s excellence strategy – EXC number 2064/1 – project number 390727645.
\end{acknowledgements}

%
%

%

%
%
%

%

%
%
{%
\bibliography{kladny_306}
}
\end{document}