\section{Related Work}
\paragraph{Differentially private dataset release.} 
Many recent works \citep{sheffet2017differentially, gondara2020differentially, xie2018differentially, jordon2018pate,lee2019synthesizing, xu2017dppro, kenthapadi2012privacy} study the differentially private data release algorithms.
However, those algorithms either only serve for data release from a \emph{single-party} \citep{sheffet2017differentially, gondara2020differentially}, or focus on the feature dimension reduction or empirical improvement \citep{lee2019synthesizing, xu2017dppro, kenthapadi2012privacy}, which is orthogonal to the study of asymptotical optimality w.r.t. dataset size.
In \cite{sheffet2017differentially} and \cite{gondara2020differentially}, the random Gaussian projection matrices in their method contribute to the differential privacy guarantee,
hence the sharing of projection matrix would violate the privacy guarantee between parties. 
Nevertheless, without sharing this projection matrix, the utility cannot be guaranteed anymore.
In \cite{xie2018differentially} and \cite{jordon2018pate}, they train a  differentially private GAN.
However, it is not obvious to rigorously privately share data information during their training when each party holds different attributes but same instances.
\cite{lee2019synthesizing} proposes a random mixing method and also analyzes the linear model. 
However, the way they mix only works for realizable linear data.
It is not able to be extended to the general linear regression and the asymptotic optimality guarantee.
\cite{xu2017dppro} and \cite{kenthapadi2012privacy} focus on the feature dimension reduction, which is orthogonal to the study of asymptotical optimality w.r.t. dataset size.

\paragraph{Asymptotically optimal differentially private convex optimization.} A large amount of work study differentially private optimization for convex problems \citep{bassily2014private, bassily2019private, feldman2020private} or particularly for linear regression \citep{sheffet2017differentially, kasiviswanathan2011can, chaudhuri2012convergence}.
They mainly differ from our work in the sense that their goal is to release the final model while ours is to release the dataset.

\paragraph{Linear regression in vertical federated learning.} Linear regression is a fundamental machine learning task. \cite{hfn11,nwi+13,gsb+17} studying linear regression over vertically partitioned datasets based on secure multi-party computation. However, cryptographic protocols such as Homomorphic Encryption~\citep{hfn11,nwi+13} and garbled circuits~\citep{nwi+13,gsb+17} lead to heavy overhead on computation and communication. From this aspect, DP-based techniques are more practical.
