\section{Impossibility of Boosting in LLP} \label{sec:impossibility_llp}


The {\sf Max-Cut} problem is: given an undirected graph $G(V, E)$ find a cut given by the assignment $g : V \to \{0,1\}$ which separates the maximum number of edges in $E$ i.e., maximizes $\left|\{e = \{u, v\} \in E| g(u) \neq g(v)\}\right|$. We shall use the following construction of graph $G_{\tn{FS}}(V_{\tn{FS}}, E_{\tn{FS}})$ given in Sec. 3.1 of \cite{Feige2002}: 

{\bf Construction.} Let $\alpha\pi = \theta \in [\pi/2, \pi)$ and $\eps > 0$ be an arbitrarily small parameter such that $\theta + \eps\pi < \pi$. Let $d = O(1/\eps\log(1/\eps))$ and $\gamma = \eps^2/(2d)$. Divide the $(d-1)$-dimensional unit sphere $\mathbb{S}^{d-1}$ into $\left(\frac{O(1)}{\gamma}\right)^d$ equal sized cells of diameter at most $\gamma$ each (this is shown to be possible in Lemma 21 of \cite{Feige2002}). From each cell pick an arbitrary point $\bv$ and add it to $V_{\tn{FS}}$. Add an edge $\{\bu, \bv\}$ to $E_{\tn{FS}}$ for each pair of points $\bu, \bv \in V_{\tn{FS}}$ whose angle is between $\theta$ and $\theta + \eps$. 




Section 3.1 of \cite{Feige2002} shows\footnote{While \cite{Feige2002} state the proof of \eqref{eqn:FSoptd} for a specific value of $\theta$, the proof applies to all values of $\theta \in [\pi/2,\pi)$.} that
\begin{equation}
    \Pr_{\{\bu, \bv\} \in E_{\tn{FS}}}[g(\bu) \neq g(\bv)] \leq \theta/\pi + O(\eps^2)  = \alpha + O(\eps^2) \label{eqn:FSoptd}
\end{equation}
for any $g : V_{\tn{FS}} \to \{0,1\}$. 

\subsection{Proof of Theorem \ref{thm:LLP-imposssibility}}
Let $G_{\tn{FS}}(V_{\tn{FS}}, E_{\tn{FS}})$ be the graph constructed above using $\theta = \alpha\pi \in [\pi/2, \pi)$ and let $\eps$ taken to be the same as that in the statement of Theorem \ref{thm:LLP-imposssibility}. Taking $V_{\tn{FS}}$ to be the underlying set feature-vectors, let the set of bags $\mc{B}$ be $E_{\tn{FS}}$ i.e., each edge $\{\by, \bv\}$ is a bag. All aggregate labels are $1$, so that any bag is satisfied by $g : V_{\tn{FS}} \to \{0,1\}$ iff the corresponding edge is separated by $g$. 

Now, for any bag $\{\bu, \bv\}$ in $\mc{B}$, from the construction of $G_{\tn{FS}}(V_{\tn{FS}}, E_{\tn{FS}})$, the angle between $\bu$ and $\bv$ is at least $\theta$. Thus, a random homogeneous halfspace (given by ${\sf pos}\left(\br^{\sf T}\bx\right)$ for $\br$ chosen uniformly at random from $\mathbb{S}^{d-1}$) satisfies the bag with probability at least $\theta/\pi = \alpha$. 

Thus for any assignment of weights $w_B$ for bags $B \in \mc{B}$, the expected weight of bags satisfied by a random  homogeneous halfspace is $\sum_{B}w_B\Pr_{\br \leftarrow \mathbb{S}^{d-1}}\left[B\tn{ is satisfied by } {\sf pos}\left(\br^{\sf T}\bx\right)\right] = \alpha\sum_Bw_B$ by linearity of expectation. Therefore, there is one classifier with weighted accuracy $\alpha$.

The upper bound on the accuracy of \emph{any} classifier on $\mc{B}$ follows directly from \eqref{eqn:FSoptd} and small enough $\eps > 0$.





