LLM Bias in Algorithm Evolution
Researching LLM biases in heuristic algorithm generation using frameworks like ShinkaEvolve and OpenEvolve. Investigating why LLMs tend toward local optimization and avoid bold algorithmic modifications.
Overview
Our research investigates the automation of heuristic algorithm generation and improvement using frameworks based on open-source systems like ShinkaEvolve and OpenEvolve, with particular emphasis on understanding LLM biases. A significant challenge we have observed is that LLMs, when operating within these frameworks, exhibit a tendency to avoid bold, potentially high-impact modifications to algorithms. Instead, they often become trapped in local optimization patterns, making incremental changes that yield minimal score improvements.
Key Features
LLM Bias Analysis
Investigating the inherent biases in LLMs that cause conservative, risk-averse behavior in algorithm modification.
Local Optima Problem
Studying why LLMs get trapped in local optimization patterns, making only incremental changes with minimal improvements.
Evolutionary Frameworks
Working with ShinkaEvolve and OpenEvolve to automate heuristic algorithm generation and improvement.
Hypothesis Validation
Conducting experiments to validate that fixed biases in LLM operation within frameworks cause conservative behavior.
Technologies Used
Challenges Overcome
- LLMs avoid bold, potentially high-impact algorithm modifications
- Tendency to get trapped in local optimization patterns
- Fixed biases inherent in LLM operation within frameworks
- Incremental changes yielding minimal score improvements
Outcomes & Impact
- Identified key LLM bias patterns in algorithm evolution
- Developing working hypothesis on fixed bias behavior
- Conducting experiments to validate hypothesis and explore solutions