agentic-kaggle-skill

Treat every competition as a validation problem first and a modeling problem second. The default target platform is Kaggle, so prefer Kaggle-native notebooks/scripts, datasets, model artifacts, competition submissions, and score receipts. For code competitions, assume the final notebook/kernel will be rerun by Kaggle against hidden data unless the competition docs prove otherwise. • Read the competition page, classify the submission mode as classic file submission or code/notebook scoring, then inspect the rules, data-use terms, sharing policy, data dictionary, metric, submission format, train/test construction hints, and leakage warnings. • If live competition intelligence tools are available, inspect top public open notebook solutions and relevant discussion activity before major architecture choices; treat them as clues, not authority. • Identify the task type: binary, multiclass, multilabel, regression, ranking, image, segmentation, text, time series, grouped entities, or a hybrid. • Design folds before feature engineering or modeling. Prefer a fold column saved into the training data so every experiment uses the same comparison surface. • Build the simplest metric-correct baseline and produce out-of-fold (OOF) predictions plus a valid submission. • Proactively plan a stronger architecture once the baseline is trustworthy: diverse model families, feature/embedding producers, augmentation/pseudo-label/distillation stages, calibration/postprocessing, and an ensemble or stacker. • Iterate with validation gates: test one meaningful change at a time when possible, but launch several independent producer notebooks in parallel when they create diverse artifacts that can be compared by OOF score or ensemble diversity. • Offload heavy training, inference, embedding generation, image/text experiments, or memory-risky jobs to Kaggle notebooks/scripts when local compute may OOM or take too long. • For sophisticated architectures, split work into a small Kaggle pipeline: run several independent producer notebooks/scripts first, save each useful producer output as a private Kaggle dataset, then run one consumer notebook/script that attaches those datasets and creates the final OOF/test/submission outputs. When a producer trains a model, its checkpoint, tokenizer/config, fold metadata, OOF/test predictions, and manifest should be exported as a Kaggle dataset; downstream notebooks load the model from /kaggle/input/.... • Submit the final submission-producing artifact to Kaggle for scoring and retrieve the resulting submission status/score. • If Kaggle returns a code-competition error or vague scoring failure, enter the debugging loop: retrieve available logs, classify likely failure mode, patch defensively, rerun the final kernel, resubmit, and repeat until scored or concretely blocked. • Track local CV, remote Kaggle run status, Kaggle submission score, public LB, private-risk notes, seed, code version, data version, and artifact paths for every run. • Ensemble only with OOF predictions generated without in-fold leakage.

• Read references/method-map.md for the neutral workflow map behind the skill. • Read references/information-sharing-policy.md before publishing competition code, notebooks, datasets, models, artifacts, or reports outside the active team or workspace. • Read references/competition-intel.md when a Kaggle competition slug, URL, title, public leaderboard context, open solutions, or discussion activity may inform the approach. • Read references/cross-validation-and-metrics.md when choosing folds, metrics, thresholds, or leakage checks. • Read references/tabular-workflow.md for categorical variables, feature engineering, selection, and hyperparameter tuning. • Read references/image-text-workflow.md for image, segmentation, and NLP competition approaches. • Read references/kaggle-code-competition-pipeline.md when the target is a Kaggle code competition, hidden rerun, final notebook scoring flow, or model artifact handoff between producer and consumer notebooks. • Read references/advanced-notebook-architecture.md when a stronger solution may need multiple model families, staged feature/embedding/model producers, pseudo-labeling, distillation, postprocessing, blending, stacking, or parallel Kaggle GPU notebooks. • Read references/kaggle-offload.md when local runs are heavy, GPU is useful, Kaggle data access is needed, or remote notebook results must be collected. • Read references/kaggle-pipeline-datasets.md when using multiple Kaggle notebooks, intermediate datasets, notebook output sources, or kagglehub. • Read references/submission-endgame.md before stopping work; this skill is not done until Kaggle scoring has been attempted and the result has been collected or a concrete blocker is documented. • Read references/code-competition-debugging.md when a Kaggle code competition submission fails, times out, OOMs, produces no score, or reports a vague hidden-run/scoring error. • Read references/ensembling-and-reproducibility.md for project layout, OOF artifacts, stacking, blending, and repeatability. • Read references/research/ or examples/ only when the user asks for historical case studies, Hermes-era patterns, or concrete competition lessons from this repository. • Run scripts/scaffold_competition.py to create a competition workspace. • Run scripts/make_folds.py to add a fold column to a training CSV. • Run scripts/prepare_kaggle_kernel.py to create a Kaggle kernel folder with metadata and retrievable experiment logs. • Run scripts/prepare_kaggle_dataset.py to create metadata and commands for versioned intermediate artifact datasets.

Start with the metric and validation: • Reproduce the competition metric locally, including clipping, transforms, averaging mode, thresholds, and sample weights. • Check whether the competition allows public code, external data, pretrained weights, generated artifacts, and public repositories before sharing anything outside the team. • Classify the Kaggle submission path early. If it is a code competition or notebook-scored workflow, design the final consumer notebook as the scoring artifact from the start. • Use kaggle-competition-intel MCP tools when available: top_open_solutions for score-ranked plus latest high-vote notebooks, and competition_discussions for top-voted plus recently active discussion topics before choosing expensive baselines. • Choose folds to match the hidden test distribution. Use stratification for classification, binned stratification for regression, group splits for repeated entities, time-aware splits for temporal data, and multilabel-aware splits when label co-occurrence matters. • Save OOF predictions for every model. OOF files are the currency for error analysis, threshold tuning, blending, and stacking. • Compare mean fold score and fold variance, not only the best fold. • Treat a public LB jump that disagrees with CV as a validation question before treating it as a modeling victory. Then build baselines in this order: • Metric-only or constant baseline to verify scoring and submission format. • Fast classical baseline: LightGBM/CatBoost/XGBoost for tabular, TF-IDF plus linear model for text, pretrained backbone for images. • Strong baseline with stable folds, clean artifacts, and OOF predictions. • Feature/model experiments with run logs and one primary metric. • Ensembling or stacking after several diverse, individually validated models exist. Do not wait for the user to explicitly ask for sophistication when the competition warrants it. After a stable baseline, propose and execute a stronger staged architecture if public solutions, discussion intel, data modality, metric pressure, or CV plateau suggests that a single notebook/model will underperform. Keep the architecture gated by OOF evidence, artifact manifests, and final Kaggle scoring. When a run is likely to exceed local RAM/VRAM, require a GPU/TPU, or needs Kaggle-only data mounts, prepare a Kaggle kernel run instead of forcing it locally. The remote run must emit experiment_log.json, metrics.jsonl, and an artifact manifest so results can be retrieved and interpreted after kaggle kernels output. When the solution has multiple heavy stages, keep the remote graph shallow and inspectable: • Use fewer than five independent producer notebooks/scripts per wave. • Give each producer a single responsibility such as feature generation, embedding extraction, fold training, checkpoint training, model distillation, or inference. • For code competitions, prefer producer notebooks that train models/checkpoints and export them as private Kaggle datasets; the final consumer should attach those datasets, load models from /kaggle/input/..., and perform hidden-test-safe inference. • Save producer notebook outputs as versioned private Kaggle datasets for durable reuse and as stable inputs to downstream notebooks; use direct kernel_sources only for short-lived chains where durability/versioning is unnecessary. • Run one final consumer notebook/script that attaches the produced datasets, reads their artifacts, writes the final OOF, test predictions, blend/stack report, and submission, then submit that final artifact/kernel to Kaggle for scoring. • Use