Abstract: Highlights•LLMs generate correct code for synthesis tasks but often miss certain grammar rules.•Five LLMs were used within the Similarity-Based Many-Objective G3P (SBMaOG3P) system.•Results show SBMaOG3P outperforms LLMs and G3P in finding grammar-compliant programs.