\section{Additional Experiment Details}\label{sec:dataset}
In the benchmarks, we applied the deep Gaussian process \citep{wilson2016deep}. Here, we offer a more detailed discussion of the construction of the six CBO tasks studied in \secref{sec: exps}.

\subsection{Synthetic Tasks}
We study two synthetic CBO tasks constructed from conventional BO benchmark tasks. Here, we rely on the implementation contained in BoTorch's \citep{balandat2020botorch} test function module.

\paragraph*{Rastrigin-1D-1C}
The Rastrigin function is a non-convex function used as a performance test problem for optimization algorithms. It was first proposed by \citet{10018403158} and used as a popular benchmark dataset \citep{pohlheimgeatbx}. It is constructed to be highly multimodal, with local optima being regularly distributed to trap optimization algorithms. Concretely, we negate the 1D Rastrigin function and try to find its maximum:
$f(\instance) = -10{d} -\sum^{d}_{i=1}{(x_i^2 - 10\cos(2\pi{x_i}))},\ d=1$. The range of $\instance$ is $[-5, 5]$, and we construct the constraint to be $c(\instance) = |\instance+0.7|^{1/2}$. When setting the threshold as $\sqrt{2}$, we essentially exclude the global optimum from the feasible area. The constraint enforces the optimization algorithm to explore feasibility rather than allowing algorithms to improve the reward by merely optimizing the objective. Then, the feasible region takes up approximately 60\% of the search space. This one-dimensional task is designed to illustrate the necessity of adaptively trade-off learning of constraints and optimization of the objective. 

We also vary the threshold to control the portion of the feasible region to study the robustness of \algname. \Figref{fig:exps:scan_res} shows the distribution of the objective function and feasible regions on the samples.

\paragraph*{Ackley-5D-2C}
The Ackley function is also a popular benchmark for optimization algorithms. Compared with the Rastrigin function, it is similarly highly multimodal, while the region near the center is growingly steep. Same as what is done for Rastrigin, we negate the 5D Ackley function and try to find its maximum:
$f(\instance) = 20\exp{(-0.2\sqrt{1/d\sum_i^d{x_i^2}})} + \exp{(1/d\sum_i^d{\cos(2\pi x_i)})} + 20 + \exp(1),\ d=5$. The search space is restricted to $[-5, 3]^5$. We construct two constraints to enforce a feasible area approximately taking up 14\% of the search space. The first constraint $(\Vert x - \mathbf{1} \Vert_2 - 5.5)^2 - 1$ constructs two feasible regions. One of them lies in the center, and the other is close to the boundary of the search space. The second constraint $-\Vert x \Vert_{\infty}^2+9$ allows one hypercube feasible region in the center. 


\subsection{Real-world Tasks}
We study four real-world CBO tasks. The first three are extracted from \cite{tanabe2020easy}, which offers a broad selection of real-world multi-objective multi-constraints optimization tasks. The fourth one is a 32-dimensional optimization task extracted from the UCI Machine Learning repository \citep{misc_wave_energy_converters_534}.

\paragraph{Vessel-4D-3C}
The pressure vessel design problem aims to optimize the total cost of a cylindrical pressure vessel. The four variables represent the thicknesses of the shell, the head of a pressure vessel, the inner radius, and the length of the cylindrical section. The problem is originally studied in \cite{kannan1994augmented}, and we follow the formulation in RE2-4-3 in \cite{tanabe2020easy}. The feasible regions take up approximately 78\% of the whole search space.

\paragraph{Spring-3D-6C}
The coil compression spring design problem aims to optimize the volume of spring steel wire, which is used to manufacture the spring \citep{lampinen1999mixed} under static loading. The three input variables denote the number of spring coils, the outside diameter of the spring, and the spring wire diameter, respectively. The constraints incorporate the mechanical characteristics of the spring in real-world applications. We follow the formulation in RE2-3-5 in \cite{tanabe2020easy}. The feasible regions take up approximately 0.38\% of the whole search space.

\paragraph{Car-7D-8C}
The car cab design problem includes seven input variables and eight constraints. The problem is originally studied in \cite{deb2013evolutionary}. We follow the problem formulation in RE9-7-1 in \cite{tanabe2020easy} and focus on the objective of minimizing the weight of the car while meeting the European enhanced Vehicle-Safety Committee (EEVC) safety performance constraints. The seven variables indicate the thickness of different parts of the car. The feasible region takes up approximately 13\% of the whole search space.

\paragraph{Converter-32D-3C}
This UCI dataset we use consists of positions and absorbed power outputs of wave energy converters (WECs) from the southern coast of Sydney. The applied converter model is a fully submerged three-tether converter called CETO. 16 WECs 2D-coordinates are placed and optimized in a size-constrained environment \citep{misc_wave_energy_converters_534}. The input is, therefore, 32 dimensional. We place three constraints on the tasks, including the absorbed power of the first two converters being above a certain threshold of 96000 and the general position being not too distant with the two-norm below 2000. The feasible region takes up approximately 27\% of the whole search space.

\input{C-Appendix-exps.tex}
