AutoResearch tops Optuna in code-space search

Agentic code-space search leverages an LLM's structural understanding of code to outperform traditional Bayesian hyperparameter optimization. The agent uses embedded knowledge of machine learning best practices to prune the search space and can perform macro-optimizations like changing loss functions or layer types. This approach significantly reduces GPU training time, making it a more economical choice for large-scale research and development despite higher token costs.