CoFE: Collaborative Feature Engineering via Semantically-Guided Exploration and Diagnostic-Driven Refinement

Abstract

In domains such as finance, healthcare, and industry, feature engineering remains the key bottleneck limiting the performance of machine learning models on tabular data. While Automated Feature Engineering (AutoFE) aims to reduce this manual effort, existing approaches still suffer from distinct limitations: data-driven exploration can waste substantial computation on semantically meaningless feature combinations, whereas knowledge-driven approaches using Large Language Models (LLMs) struggle to construct high-order interactions without rich structural context and are typically guided only by coarse global metrics. We propose CoFE (Collaborative Feature Engineering), a two-phase framework that tightly couples search-based exploration with LLM-driven reasoning. In the exploration phase, CoFE leverages an LLM-constructed semantic feature schema to guide a Monte Carlo Tree Search (MCTS), enforcing semantic constraints while encouraging a diverse pool of complex candidate features that provides the missing structural context for LLMs. In the refinement phase, CoFE introduces feature health reports, a diagnostic artifact that supplies the LLM with actionable sample-level and structural feedback for targeted corrections. Experiments on 16 public tabular benchmarks show that CoFE consistently outperforms state-of-the-art data-driven and LLM-based AutoFE methods on the majority of datasets, while offering favorable computational efficiency.