Automated Algorithm Design
Date:
Invited talk for the AutoML Seminar Series.
Title: LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms
Speakers:
- Elena Raponi: https://www.universiteitleiden.nl/en/
- Niki van Stein: https://www.universiteitleiden.nl/en/
Papers:
- LLaMEA-BO: https://arxiv.org/abs/2505.21034
- LLaMEA: https://ieeexplore.ieee.org/abstract/
Abstract: Large Language Models (LLMs) are rapidly becoming creative co-pilots for AI, Optimization and AutoML, moving beyond tuning to generating and evolving entire algorithms on demand. This talk introduces LLaMEA, an evolutionary computation framework that uses LLMs to iteratively mutate and improve full codebases, generating competitive optimizers across continuous, combinatorial, and hyperparameter spaces. We specifically focus on its application to Bayesian Optimization (LLaMEA-BO), showing how LLM-generated code, guided by adapted prompt engineering and mutation schedules, consistently discovers high-performing BO algorithms and scales well across dimensions and tasks. The talk also covers integration with AutoML toolchains, insights from the BLADE benchmarking suite, and offers practical takeaways with open-source code.