TRL

Reading on training

• Prompt, or fine-tune?Almost every 'the model does not know this' problem is a retrieval problem wearing a hat. Here is how to tell the difference before you spend a week on a training run.

• PEFTParameter-efficient fine-tuning methods such as LoRA and QLoRA.
• LLaMA-FactoryUnified interface for SFT, DPO and RLHF on open models.
• torchtunePyTorch-native recipes for fine-tuning and aligning open models.

Listed as an alternative to

• LLaMA-FactoryLLaMA-FactoryUnified interface for SFT, DPO and RLHF on open models.Use when One interface for SFT, DPO and RLHF across open models.Skip when You want a minimal, readable training script.Apache-2.0 · self-hosted · free
• PEFTPEFTParameter-efficient fine-tuning methods such as LoRA and QLoRA.Use when Parameter-efficient fine-tuning — LoRA and variants — on limited hardware.Skip when You need to change the model's capabilities, not bolt on an adapter.BSD-3-Clause · self-hosted · free
• torchtunetorchtunePyTorch-native recipes for fine-tuning and aligning open models.Use when PyTorch-native recipes that you can read and modify.Skip when You want the breadth of a config-driven stack.BSD-3-Clause · self-hosted · free
• Hugging Face TRLHugging Face TRLPreference and reward optimisation on top of any PEFT setup.Use when Preference and reward optimisation hosted, without a cluster.Skip when Training data cannot be uploaded to a third party.proprietary · saas · usage-based

Also in training

• UnslothUnslothHand-written kernels that cut LoRA memory and time sharply.Use when LoRA fine-tuning on a single GPU, with memory and time roughly halved.Skip when You need a training stack you can audit line by line.Apache-2.0 · self-hosted · free
• AxolotlAxolotlConfiguration-driven fine-tuning across common architectures.Use when Configuration-driven fine-tuning across many architectures.Skip when You need to hand-tune the training loop itself.Apache-2.0 · self-hosted · free
• LLaMA-FactoryLLaMA-FactoryUnified interface for SFT, DPO and RLHF on open models.Use when One interface for SFT, DPO and RLHF across open models.Skip when You want a minimal, readable training script.Apache-2.0 · self-hosted · free
• PEFTPEFTParameter-efficient fine-tuning methods such as LoRA and QLoRA.Use when Parameter-efficient fine-tuning — LoRA and variants — on limited hardware.Skip when You need to change the model's capabilities, not bolt on an adapter.BSD-3-Clause · self-hosted · free
• DeepSpeedDeepSpeedZeRO sharding and pipeline parallelism for large-scale training.Use when ZeRO sharding and pipeline parallelism across many GPUs.Skip when The model fits on one device — the complexity buys nothing.MIT · self-hosted · free
• Megatron-LMMegatron-LMTensor and pipeline parallel training for multi-GPU clusters.Use when Tensor and pipeline parallel pretraining at cluster scale.Skip when You are fine-tuning rather than pretraining.Apache-2.0 · self-hosted · free
• torchtunetorchtunePyTorch-native recipes for fine-tuning and aligning open models.Use when PyTorch-native recipes that you can read and modify.Skip when You want the breadth of a config-driven stack.BSD-3-Clause · self-hosted · free
• Hugging Face TRLHugging Face TRLPreference and reward optimisation on top of any PEFT setup.Use when Preference and reward optimisation hosted, without a cluster.Skip when Training data cannot be uploaded to a third party.proprietary · saas · usage-based
• ColabColabHosted GPUs for small runs and quick experiments.Use when A disposable GPU for a quick experiment, with no setup.Skip when Anything reproducible — a notebook is not a training pipeline.proprietary · saas · free-tier
• ReplicateReplicateHosted fine-tuning and deployment for open models.Use when Hosted fine-tuning and deployment of an open model, billed per run.Skip when You need custom training code or strict residency.proprietary · saas · usage-based
• Weights & Biases LaunchWeights & Biases LaunchManaged training runs with sweeps and artifact tracking.Use when Managed training runs with sweeps and artifact tracking wired up.Skip when You want training infrastructure you run yourself.proprietary · saas · subscription