e-SimFT: Pareto-Optimal Sampling of Generative Design Models Fine-tuned with Simulation Feedback

Track: Main Track

Keywords: Reinforcement learning, Pareto front, Sampling, Fine-tuning, Alignment, Generative Design

Abstract: Deep generative models have recently shown success in solving complex engineering design problems where models predict solutions that address the design requirements specified as input. However, there remains a challenge in aligning such models for effective design exploration. For many design problems, finding a solution that meets all requirements is infeasible. In such a case, engineers prefer to obtain a set of Pareto-optimal solutions with respect to those requirements, but uniform sampling of generative models may not yield a useful Pareto front. To address this gap, we first fine-tune generative design models with simulation feedback, and then apply epsilon-sampling, inspired by the epsilon-constraint method used for Pareto front generation with classical optimization algorithms, to construct a high-quality Pareto front with the fine-tuned models. Our framework, named e-SimFT, is shown to produce better-quality Pareto fronts than existing multi-objective alignment methods developed for large language models.

Submission Number: 41