{"id":116054,"date":"2026-04-23T13:15:02","date_gmt":"2026-04-23T20:15:02","guid":{"rendered":"https:\/\/developer.nvidia.com\/blog\/?p=116054"},"modified":"2026-04-23T13:15:30","modified_gmt":"2026-04-23T20:15:30","slug":"winning-a-kaggle-competition-with-generative-ai-assisted-coding","status":"publish","type":"post","link":"https:\/\/developer.nvidia.com\/blog\/winning-a-kaggle-competition-with-generative-ai-assisted-coding\/","title":{"rendered":"Winning a Kaggle Competition with Generative AI\u2013Assisted Coding"},"content":{"rendered":"\n<p>In March 2026, three LLM agents generated over 600,000 lines of code, ran 850 experiments, and helped secure a first-place finish in a Kaggle playground competition.<\/p>\n\n\n\n<p>Success in modern machine learning competitions is increasingly defined by how quickly you can generate, test, and iterate on ideas. LLM agents, combined with GPU acceleration, dramatically compress this loop.&nbsp;<\/p>\n\n\n\n<p>Historically, two bottlenecks have limited this experimentation:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How quickly you can write code for new experiments.<\/li>\n\n\n\n<li>How quickly you can execute those experiments.<\/li>\n<\/ul>\n\n\n\n<p>GPUs and libraries like NVIDIA cuDF, NVIDIA cuML, XGBoost, and PyTorch have largely solved the second problem. LLM agents now address the first problem\u2014unlocking a new scale of rapid, iterative experimentation.<\/p>\n\n\n\n<p>This blog post describes how I used LLM agents to accelerate the discovery of the most performant tabular data prediction solutions.<\/p>\n\n\n\n<h2 id=\"case_study_kaggle_playground_churn_prediction\"  class=\"wp-block-heading\">Case study: Kaggle Playground churn prediction<a href=\"#case_study_kaggle_playground_churn_prediction\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h2>\n\n\n\n<p>The March 2026 <a href=\"https:\/\/www.kaggle.com\/competitions\/playground-series-s6e3\">Kaggle Playground competition<\/a> challenged participants to predict telecom customer churn with performance measured by area under the curve (AUC)\u2014where the most accurate solution wins.\u00a0<\/p>\n\n\n\n<p>The <a href=\"https:\/\/www.kaggle.com\/competitions\/playground-series-s6e3\/writeups\/1st-place-gpt5-4-gemini3-1-claudeopus4-6-kgm\">first-place solution<\/a> is a four-level stack of 150 models, selected from 850.&nbsp;<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure data-wp-context=\"{&quot;imageId&quot;:&quot;69eff1fe85cc9&quot;}\" data-wp-interactive=\"core\/image\" class=\"aligncenter size-full wp-lightbox-container\"><img loading=\"lazy\" decoding=\"async\" width=\"1999\" height=\"1125\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on-async--click=\"actions.showLightbox\" data-wp-on-async--load=\"callbacks.setButtonStyles\" data-wp-on-async-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack.webp\" alt=\"Diagram of the winning four-level stack. The first level extracts features. The next two levels extract patterns. And the final level predicts the target.\u00a0\" class=\"wp-image-116057\" srcset=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack.webp 1999w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-179x101.png 179w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-300x169.png 300w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-768x432.png 768w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-625x352.png 625w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-1536x864.png 1536w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-645x363.png 645w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-660x370.png 660w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-500x281.png 500w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-160x90.png 160w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-362x204.png 362w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-195x110.png 195w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-1024x576.png 1024w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Four-Level-Stack-960x540.png 960w\" sizes=\"auto, (max-width: 1999px) 100vw, 1999px\" \/><button\n\t\t\tclass=\"lightbox-trigger\"\n\t\t\ttype=\"button\"\n\t\t\taria-haspopup=\"dialog\"\n\t\t\taria-label=\"Enlarge\"\n\t\t\tdata-wp-init=\"callbacks.initTriggerButton\"\n\t\t\tdata-wp-on-async--click=\"actions.showLightbox\"\n\t\t\tdata-wp-style--right=\"state.imageButtonRight\"\n\t\t\tdata-wp-style--top=\"state.imageButtonTop\"\n\t\t>\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewBox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\" \/>\n\t\t\t<\/svg>\n\t\t<\/button><figcaption class=\"wp-element-caption\"><em>Figure 1. Diagram showing how 150 models are combined into a four-level stack winning solution<\/em><\/figcaption><\/figure><\/div>\n\n\n<h2 id=\"guided_llm_agent_workflow\"  class=\"wp-block-heading\">Guided LLM agent workflow<a href=\"#guided_llm_agent_workflow\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h2>\n\n\n\n<p>In this tabular data competition, I guided LLM agents to follow the Kaggle Grandmaster playbook described in a <a href=\"https:\/\/developer.nvidia.com\/blog\/the-kaggle-grandmasters-playbook-7-battle-tested-modeling-techniques-for-tabular-data\/\">previous blog post<\/a>.<\/p>\n\n\n\n<p>Specifically, LLM agents follow a workflow: starting with exploratory data analysis (EDA), then building baselines, followed by feature engineering, and finally combining models through hill climbing and stacking.<\/p>\n\n\n\n<p>The solution used multiple LLM agents\u2014GPT-5.4 Pro, Gemini 3.1 Pro, Claude Opus 4.6\u2014 in a human-in-the-loop workflow.&nbsp;<\/p>\n\n\n\n<h3 id=\"step_1_llm_agents_perform_eda\"  class=\"wp-block-heading\">Step 1: LLM agents perform EDA<a href=\"#step_1_llm_agents_perform_eda\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h3>\n\n\n\n<p>An LLM agent must understand data structure before generating a full pipeline.<\/p>\n\n\n\n<p>Key questions include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How many rows and columns are in the training and test sets?&nbsp;<\/li>\n\n\n\n<li>What&#8217;s the target column, and how is it formatted?&nbsp;<\/li>\n\n\n\n<li>Is the task classification or regression?&nbsp;<\/li>\n\n\n\n<li>What features are available and how are they formatted?&nbsp;<\/li>\n\n\n\n<li>Which features are categorical or numeric?&nbsp;<\/li>\n\n\n\n<li>Is there missing data?<\/li>\n<\/ul>\n\n\n\n<p>This information can be provided upfront or inferred automatically through EDA.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure data-wp-context=\"{&quot;imageId&quot;:&quot;69eff1fe86b7c&quot;}\" data-wp-interactive=\"core\/image\" class=\"aligncenter size-full wp-lightbox-container\"><img loading=\"lazy\" decoding=\"async\" width=\"1670\" height=\"538\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on-async--click=\"actions.showLightbox\" data-wp-on-async--load=\"callbacks.setButtonStyles\" data-wp-on-async-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code.webp\" alt=\"Diagram showing an LLM Agent writing and running EDA code to explore data.\" class=\"wp-image-116058\" srcset=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code.webp 1670w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-179x58.png 179w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-300x97.png 300w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-768x247.png 768w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-625x201.png 625w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-1536x495.png 1536w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-645x208.png 645w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-500x161.png 500w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-160x52.png 160w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-362x117.png 362w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-341x110.png 341w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-1024x330.png 1024w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/EDA-Code-960x309.png 960w\" sizes=\"auto, (max-width: 1670px) 100vw, 1670px\" \/><button\n\t\t\tclass=\"lightbox-trigger\"\n\t\t\ttype=\"button\"\n\t\t\taria-haspopup=\"dialog\"\n\t\t\taria-label=\"Enlarge\"\n\t\t\tdata-wp-init=\"callbacks.initTriggerButton\"\n\t\t\tdata-wp-on-async--click=\"actions.showLightbox\"\n\t\t\tdata-wp-style--right=\"state.imageButtonRight\"\n\t\t\tdata-wp-style--top=\"state.imageButtonTop\"\n\t\t>\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewBox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\" \/>\n\t\t\t<\/svg>\n\t\t<\/button><figcaption class=\"wp-element-caption\"><em>Figure 2. LLM Agents explore the data by repeatedly writing and running EDA code<\/em><\/figcaption><\/figure><\/div>\n\n\n<p>If using LLM in a chat window, prompt with:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cPlease write EDA code to explore the CSV file train.csv and test.csv. I will run the code and share the plots and text back with you.\u201d \n<\/pre><\/div>\n\n\n<p>If using LLM with code execution like Claude Code, then you can ask the LLM to write and run its own code to understand the data.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cPlease write and run EDA code to understand the CSV files train.csv and test.csv\u201d\n<\/pre><\/div>\n\n\n<h3 id=\"step_2_llm_agents_build_baselines\"  class=\"wp-block-heading\">Step 2: LLM agents build baselines<a href=\"#step_2_llm_agents_build_baselines\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h3>\n\n\n\n<p>Once the LLM understands the data, specifically the feature columns and target column, it&#8217;s time to write the first full pipeline to train a kfold model by asking the LLM for a specific model.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cPlease write full code pipeline to read train.csv and test.csv and train a kfold XGBoost model. Save the OOF (out of fold predictions) and the Test PREDS to disk as Numpy files. Display the metric score each fold and overall.\u201d\n<\/pre><\/div>\n\n\n<p>Copy and paste the output code into your codebase. When using a command-line or IDE agent, have it create a Python or Jupyter Notebook directly.&nbsp;<\/p>\n\n\n\n<p>Run the code to obtain your first CV metric score, OOF, and Test PRED files.<\/p>\n\n\n\n<p>You can ask the LLM to build a variety of baselines, including GBDT, NN, and ML models. Each experiment reports a CV score and saves predictions to disk as: \u201c<code>train_oof_[MODEL]_[VERSION].npy<\/code>\u201d and \u201c<code>test_preds_[MODEL]_[VERSION].npy<\/code>\u201d.&nbsp;<\/p>\n\n\n\n<p>These files are important, and we&#8217;ll use them later.<\/p>\n\n\n\n<h3 id=\"step_3_llm_agents_perform_feature_engineering\"  class=\"wp-block-heading\">Step 3: LLM agents perform feature engineering<a href=\"#step_3_llm_agents_perform_feature_engineering\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h3>\n\n\n\n<p>We now have a collection of diverse models and know their baseline CV metric scores. We can improve each model with feature engineering and\/or model tuning\/improvements. Feature engineering focuses on transforming the data so our models can extract more signal. And model tuning\/improvements focus on modifying the model to extract more signal. LLM agents excel at both of these tasks.&nbsp;<\/p>\n\n\n\n<p>Iteratively running experiments and keeping all ideas that improve the models leads to better and better models. For each experiment, good or bad, always save the OOF and test prediction to disk.<\/p>\n\n\n\n<figure data-wp-context=\"{&quot;imageId&quot;:&quot;69eff1fe87f0f&quot;}\" data-wp-interactive=\"core\/image\" class=\"wp-block-image size-large wp-lightbox-container\"><img loading=\"lazy\" decoding=\"async\" width=\"625\" height=\"222\" data-wp-class--hide=\"state.isContentHidden\" data-wp-class--show=\"state.isContentVisible\" data-wp-init=\"callbacks.setButtonStyles\" data-wp-on-async--click=\"actions.showLightbox\" data-wp-on-async--load=\"callbacks.setButtonStyles\" data-wp-on-async-window--resize=\"callbacks.setButtonStyles\" src=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-625x222.png\" alt=\"Diagram showing an LLM Agent writing and running new feature engineering to improve model performance.\" class=\"wp-image-116059\" srcset=\"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-625x222.png 625w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-179x64.png 179w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-300x107.png 300w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-768x273.png 768w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-1536x546.png 1536w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-645x229.png 645w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-500x178.png 500w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-160x57.png 160w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-362x129.png 362w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-310x110.png 310w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-1024x364.png 1024w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration-960x341.png 960w, https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/Agent-Iteration.webp 1694w\" sizes=\"auto, (max-width: 625px) 100vw, 625px\" \/><button\n\t\t\tclass=\"lightbox-trigger\"\n\t\t\ttype=\"button\"\n\t\t\taria-haspopup=\"dialog\"\n\t\t\taria-label=\"Enlarge\"\n\t\t\tdata-wp-init=\"callbacks.initTriggerButton\"\n\t\t\tdata-wp-on-async--click=\"actions.showLightbox\"\n\t\t\tdata-wp-style--right=\"state.imageButtonRight\"\n\t\t\tdata-wp-style--top=\"state.imageButtonTop\"\n\t\t>\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"12\" height=\"12\" fill=\"none\" viewBox=\"0 0 12 12\">\n\t\t\t\t<path fill=\"#fff\" d=\"M2 0a2 2 0 0 0-2 2v2h1.5V2a.5.5 0 0 1 .5-.5h2V0H2Zm2 10.5H2a.5.5 0 0 1-.5-.5V8H0v2a2 2 0 0 0 2 2h2v-1.5ZM8 12v-1.5h2a.5.5 0 0 0 .5-.5V8H12v2a2 2 0 0 1-2 2H8Zm2-12a2 2 0 0 1 2 2v2h-1.5V2a.5.5 0 0 0-.5-.5H8V0h2Z\" \/>\n\t\t\t<\/svg>\n\t\t<\/button><figcaption class=\"wp-element-caption\"><em><em>Figure 3. LLM Agents improve models by repeatedly exploring feature engineering and evaluating the new model<\/em><\/em><\/figcaption><\/figure>\n\n\n\n<p>LLM agents can write code as fast as we want. To accelerate the cycle, we run each experiment as fast as possible by always using GPUs and GPU libraries such as cuDF, cuML, gradient boosting decision tree GPUs, and PyTorch GPUs.&nbsp;<\/p>\n\n\n\n<p>To generate new ideas, we can either propose them or have LLMs generate them. There are many effective ways to encourage LLMs to generate ideas, such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Ask LLMs to find and read research papers on the topic.<\/li>\n\n\n\n<li>Ask LLMs to read forums and publicly shared code on the topic.<\/li>\n\n\n\n<li>Have LLMs perform EDA to determine relationships between features and targets for feature engineering.<\/li>\n\n\n\n<li>Ask LLMs for ideas based on the current knowledge base.<\/li>\n\n\n\n<li>Have humans brainstorm with LLMs and create ideas together.<\/li>\n<\/ul>\n\n\n\n<p>Using one of our ideas, we can ask an LLM agent to create new code from our old code:<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cPlease write me a complete replacement code for the code below that uses XYZ instead of ABC\u201d.\n<\/pre><\/div>\n\n\n<p>We now have a new experiment to run!<\/p>\n\n\n\n<h3 id=\"step_4_llm_agents_combine_models\"  class=\"wp-block-heading\">Step 4: LLM agents combine models<a href=\"#step_4_llm_agents_combine_models\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h3>\n\n\n\n<p>At this point, we have lots of experiment results, each with its own model and different feature engineering saved in a Python script or Jupyter Notebook. LLM agents excel at combining all these models and ideas and can help us use and manage all our models in various ways, like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Summarize all model types and feature engineering.<\/li>\n\n\n\n<li>Combine ideas from different models and feature engineering to build new, stronger single models.<\/li>\n\n\n\n<li>Build ensembles from different models.<\/li>\n\n\n\n<li>Stack models over other models.<\/li>\n\n\n\n<li>Use some models to pseudo-label \/ knowledge distill into new, stronger single models.<\/li>\n<\/ul>\n\n\n\n<p>One of the most useful first things I like to do is ask an LLM agent to summarize all our experiments. We can drag and drop files into a chat window, or use an LLM command-line agent (like Claude Code) to read and aggregate results across multiple files. This helps us better understand the data and problem, showing us what is working.<\/p>\n\n\n\n<p>One powerful technique is to ask the LLM agent to combine multiple ideas\/models into a single model.<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cCan you read all these IPYNB files and use all these ideas to write full code to train a new single XGBoost model which is stronger than all of these models?\u201d\u00a0\n<\/pre><\/div>\n\n\n<p>Another technique is to transfer the knowledge from some or all of our models into a single model. We use our OOF and test predictions (which are essentially pseudo-labels) to transfer knowledge into a new, stronger single model.&nbsp;<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cCan you please train a new single NN or GBDT using knowledge distillation from all our OOF and Test PREDs and make a new high performing single model?\u201d\n<\/pre><\/div>\n\n\n<p>Both techniques above produce new experiments and new OOF and test prediction files. Each baseline model and experiment with new feature engineering and\/or model improvements has an associated OOF and test prediction file. It&#8217;s common to have hundreds of files. We can now ask LLMs to combine using hill climbing and\/or stacking.&nbsp;<\/p>\n\n\n<div class=\"wp-block-syntaxhighlighter-code \"><pre class=\"brush: bash; title: ; notranslate\" title=\"\">\n\u201cCan you please try combining all our OOF and Test PREDs using various meta models? Please try Hill Climbing, Ridge\/Logistic regression, NN, and GBDT stackers. Thanks\u201d\n<\/pre><\/div>\n\n\n<h2 id=\"results\"  class=\"wp-block-heading\">Results<a href=\"#results\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h2>\n\n\n\n<p>Following the four steps above, we create a diverse set of models. Then we improve each model\u2019s performance. Finally, we combine everything into a powerful solution. The advantage lies in exploring many ideas quickly with GPU-accelerated model execution and LLM agents to write code faster. Everyone can employ these techniques when searching for the most performant solution to their tabular data prediction tasks.<\/p>\n\n\n\n<h2 id=\"get_started\"  class=\"wp-block-heading\">Get started<a href=\"#get_started\" class=\"heading-anchor-link\"><i class=\"fas fa-link\"><\/i><\/a><\/h2>\n\n\n\n<p>Ready to accelerate your results? Get started by exploring the <a href=\"https:\/\/developer.nvidia.com\/topics\/ai\/data-science\/cuda-x-data-science-libraries\/cudf\">cuDF<\/a> and <a href=\"https:\/\/developer.nvidia.com\/topics\/ai\/data-science\/cuda-x-data-science-libraries\/cuml\">cuML<\/a> libraries and <a href=\"https:\/\/developer.nvidia.com\/topics\/ai\/data-science\/cuda-x-data-science-libraries\">CUDA-X for data science<\/a>.<\/p>\n\n\n\n<p>For a deeper dive, sharpen your skills with a <a href=\"https:\/\/learn.nvidia.com\/courses\/course-detail?course_id=course-v1:DLI+S-DS-06+V1\">DLI workshop on feature engineering<\/a>. Pick up professional strategies from the post <a href=\"https:\/\/developer.nvidia.com\/blog\/the-kaggle-grandmasters-playbook-7-battle-tested-modeling-techniques-for-tabular-data\/\">The Kaggle Grandmasters Playbook: 7 Battle-Tested Modeling Techniques for Tabular Data<\/a>.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In March 2026, three LLM agents generated over 600,000 lines of code, ran 850 experiments, and helped secure a first-place finish in a Kaggle playground competition. Success in modern machine learning competitions is increasingly defined by how quickly you can generate, test, and iterate on ideas. LLM agents, combined with GPU acceleration, dramatically compress this &hellip; <a href=\"https:\/\/developer.nvidia.com\/blog\/winning-a-kaggle-competition-with-generative-ai-assisted-coding\/\">Continued<\/a><\/p>\n","protected":false},"author":923,"featured_media":115484,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"publish_to_discourse":"","publish_post_category":"","wpdc_auto_publish_overridden":"1","wpdc_topic_tags":"","wpdc_pin_topic":"","wpdc_pin_until":"","discourse_post_id":"","discourse_permalink":"","wpdc_publishing_response":"success","wpdc_publishing_error":"You\u2019ve performed this action too many times. Please wait a few seconds before trying again.","nv_subtitle":"","ai_post_summary":"<ul><li>In March 2026, three LLM agents worked together to generate over 600,000 lines of code, run 850 experiments, and secure first place in a Kaggle competition focused on predicting telecom customer churn.<\/li><li>Combining LLM agents with GPU acceleration from tools like NVIDIA cuDF and NVIDIA cuML allowed for much faster creation and testing of new ideas, helping build and improve many machine learning models quickly.<\/li><li>A step-by-step workflow guided LLM agents through understanding the data, building simple baseline models, improving those models through feature engineering, and finally combining the best models together to build the winning solution.<\/li><\/ul>","footnotes":"","_links_to":"","_links_to_target":""},"categories":[3110,696,4146],"tags":[813,3650],"coauthors":[1620],"class_list":["post-116054","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-generative-ai","category-data-science","category-development","tag-kaggle","tag-llm-techniques","tagify_workload-generative-ai","tagify_workload-data-science"],"acf":{"post_industry":["General"],"post_products":["cuDF","cuML"],"post_learning_levels":["Intermediate Technical"],"post_content_types":["Tutorial"],"post_collections":""},"jetpack_featured_media_url":"https:\/\/developer-blogs.nvidia.com\/wp-content\/uploads\/2026\/04\/agentic-ai.webp","primary_category":{"category":"Data Science","link":"https:\/\/developer.nvidia.com\/blog\/category\/data-science\/","id":696,"data_source":""},"nv_translations":[],"jetpack_shortlink":"https:\/\/wp.me\/pcCQAL-ubQ","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/posts\/116054","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/users\/923"}],"replies":[{"embeddable":true,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/comments?post=116054"}],"version-history":[{"count":4,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/posts\/116054\/revisions"}],"predecessor-version":[{"id":116072,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/posts\/116054\/revisions\/116072"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/media\/115484"}],"wp:attachment":[{"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/media?parent=116054"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/categories?post=116054"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/tags?post=116054"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/developer-blogs.nvidia.com\/wp-json\/wp\/v2\/coauthors?post=116054"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}