AI SummaryA practical guide to Supervised Fine-Tuning using SFTTrainer and Unsloth optimizations for faster instruction tuning of language models. Developers building custom instruction-following models will find actionable patterns for dataset preparation, training configuration, and thinking model implementations.
Install
# Add to your project root as SKILL.md curl -o SKILL.md "https://raw.githubusercontent.com/majiayu000/claude-skill-registry/main/skills/data/sft/SKILL.md"
Description
Supervised Fine-Tuning with SFTTrainer and Unsloth. Covers dataset preparation, chat template formatting, training configuration, and Unsloth optimizations for 2x faster instruction tuning. Includes thinking model patterns.
Overview
SFT adapts a pre-trained LLM to follow instructions by training on instruction-response pairs. Unsloth provides an optimized SFTTrainer for 2x faster training with reduced memory usage. This skill includes patterns for training thinking/reasoning models.
Usage
FastLanguageModel.for_inference(model) thinking, response = generate_with_thinking(model, tokenizer, "What is 15 + 27?") print(f"Thinking: {thinking}") print(f"Response: {response}") `
Quick Reference
| Component | Purpose | |-----------|---------| | FastLanguageModel | Load model with Unsloth optimizations | | SFTTrainer | Trainer for instruction tuning | | SFTConfig | Training hyperparameters | | dataset_text_field | Column containing formatted text | | Token ID 151668 | </think> boundary for Qwen3-Thinking models |
Critical Environment Setup
`python import os from dotenv import load_dotenv load_dotenv()
Quality Score
Acceptable
74/100
Trust & Transparency
Open Source — MIT
Source code publicly auditable
Verified Open Source
Hosted on GitHub — publicly auditable
Actively Maintained
Last commit Yesterday
100 stars — Growing Community
15 forks