Evaluating the Performance of Large Language Models for Geometry and Simulation File Generation in Physics-Based Simulations

Finite-element simulations and computer-aided design workflows require complex preprocessing, with geometry creation and simulation setup traditionally demanding significant manual expertise. The question emerges: can machine learning, namely large language models, help automate these processes? This study evaluates how well nine large language models can automate finite-element simulations starting from natural language prompts, generating both the geometry files for meshing (using Gmsh, an open-source geometry and mesh generator) and the input files needed for the solver (using Elmer, an open-source multiphysics simulation tool). Two standard test cases, a simple bar and a wheel and axle assembly, are used to evaluate and compare their performance. A set of criteria and a scoring system is introduced to assess performance across geometry and simulation setup, covering aspects such as file completeness, Boolean operations, shape fidelity, and displacement error. Results show that most LLMs excel at generating solver input files, achieving 78–88% success rate with