Imagine automating the entire process of building a machine learning model, from preparing the data to picking the best algorithm. That's the promise of Automated Machine Learning (AutoML). Traditional AutoML systems have been good at optimizing parts of this process, but they often struggle with the bigger picture. They’re like a chef who’s great at grilling but can't plan a full menu. Large Language Model (LLM) agents have shown potential, but they tend to generate solutions that aren't very diverse and can be suboptimal – like a chef randomly throwing ingredients together. Researchers have now developed a new approach called Tree-Search Enhanced LLM Agents (SELA), which combines the strengths of LLMs with a powerful search strategy. Imagine a chef (the LLM) who can create different dishes (machine learning pipelines) but needs help picking the best menu (optimal model). SELA provides this guidance by using Monte Carlo Tree Search (MCTS). MCTS acts like a restaurant manager who systematically tests different menu combinations based on customer feedback (results from running the models). With each iteration, the manager (MCTS) learns which dishes work well together and refines the menu until it’s perfect. SELA represents different pipeline configurations as a tree. This allows the system to explore different approaches intelligently. Instead of randomly trying different models or tweaking parameters, SELA uses MCTS to plan and execute experiments, learning from each attempt. The LLM agent acts as the chef, executing the steps of the chosen pipeline configuration – preparing the data, selecting the model, and training it. Then, the results are fed back into the tree search, guiding the next iteration. Essentially, SELA is an AI system that learns how to build the best machine learning models for a given problem. In tests on 20 different datasets, SELA outperformed several other AutoML systems, achieving a win rate of 65% to 80%. This tree-search approach not only boosts performance but also makes the entire process more efficient. By reusing parts of previously tested pipelines, SELA avoids redundant computations and speeds up the search for the optimal solution. While SELA has shown great promise in AutoML, its potential extends far beyond. The same principles could be applied to other complex problems like software engineering, scientific discovery, or even robotics. Imagine an AI system that can design experiments, analyze the results, and learn from its mistakes, constantly improving its strategies. That's the power of SELA and the exciting future of tree-search enhanced LLM agents.