% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Discussion}
\label{sec:discussion}
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
In this work we propose \EmbedSeg, an embedding-based instance segmentation method for 2D and 3D microscopy data, inspired by the work of~\cite{newell2017,brabandere2017,neven2019} and others.
The unmodified\footnote{Small adaptions of the code by Neven~\etal~\cite{neven2019} are required in order to deal with non-RGB images, one hot encoded instance masks etc. 
} embedding-based method by Neven~\etal shows promising results on 2D microscopy data, but the modifications we propose (medoid embedding, test-time augmentation, extension to 3D, hyper-params deduced from training data, one-hot encoded masks, \etc) secure \EmbedSeg's state-of-the-art results on many practically relevant biomedical microscopy datasets in two and three dimensions.

When comparing the results of \EmbedSeg to all obtained baseline predictions we noticed that Cellpose often runs into issues in dense 3D regions.
For example, Cellpose results on the \Organoid dataset produce a lot of spurious over-segmentations, which we believe are a side-effect of Cellpose's interpolation of individual 2D predictions of the sliced 3D input.

\sloppy %%% this is a trick to avoid lines sticking out

StarDist-3D does not have this problem, but is naturally challenged when the objects to be segmented are not star-convex (\eg for the \MouseSkull and \PlatynereisFixed datasets).
On datasets that contain only star-convex objects, \eg labeled cell nuclei, StarDist-3D typically performs on-par or even better than \EmbedSeg (see Appendix~\ref{sec:paryhale} for an example).

\fussy %%% this switches back to more beautiful typesetting

Additionally, we noticed that StarDist-3D performance generally drops at higher IoU-thresholds ($\text{AP}_{\ge 0.7}$ in Tables~\ref{tab:results2d} and~\ref{tab:results3d}).
When we analyzed the reason for this, we found that this is caused by the planarity of faces defined by the predicted vectors that span the star-convex object instances\footnote{While object instances typically have smooth, rounded surfaces, StarDist-3D instances are defined by the convex hull of a given number of vectors radiating out of a source pixel. Such a linear shape approximation causes lower IoU-values and are therefore leading to weaker AP scores at high IoU-thresholds}.

An additional and practically very relevant advantage of \EmbedSeg is its small memory footprint on the GPU, even during training, see Tables~\ref{tab:results2d} and~~\ref{tab:results3d}. 
This can enable users to benefit from our method even on cheap laptop hardware. 
Hence, we strongly feel that the method we propose will lead to improved instance segmentations in in many biomedical projects that require the analysis of microscopy data in two or three dimensions.