\section{Conclusion}
This paper studies the OPT-in-Pareto, a problem that has been studied in operation research with restrictive linear or convexity assumption but largely under-explored in deep learning literature, in which the objectives are non-linear and non-convex. Applying algorithms such as manifold gradient descent requires eigen-computation of the Hessian matrix at each iteration and thus can be expensive. We propose a first-order approximation algorithm called Pareto Navigation Gradient Descent (PNG) with theoretically guaranteed descent and convergence property to solve OPT-in-Pareto. %Empirically, we show that PNG works well on solving various of OPT-in-Pareto problem and has the potential to be applied to different multi-task learning applications. 
%One limitation of PNG is that an extra small optimization system is required in order to obtain the desired gradient at each step. This can be improved by applying similar methods in \citet{NEURIPS2018_432aca3a}. %Achieving fairness can be viewed as multitask optimization problem, which applying our algorithm, malicious users can obtain an unfair model easily, which can be a potential negative social impact of this paper.
