In simple English
Customizing an artificial intelligence model usually requires writing complex code and managing many moving parts. Axolotl puts all those settings into one simple text file. You can choose how the model learns, feed it your own data, and start training with a single command. It works with many popular open models and runs on your own hardware or in the cloud. The software is free to use under an open-source licence.
What it is
Axolotl is a Python-based framework designed to streamline the post-training and fine-tuning of large language models and vision models. It provides a unified command-line tool and configuration system to handle dataset preprocessing, training, quantization, and evaluation.
What it is good for
Solo builders and researchers who want to adapt open-source artificial intelligence models to their own specific tasks without writing custom training loops from scratch.
- Supports many popular model families including LLaMA, Mistral, and Qwen.
- Uses a single configuration file for the entire training pipeline.
- Includes memory-saving features like Flash Attention and quantization to run on smaller graphics cards.
- Requires an expensive graphics card with plenty of memory to train models.
- Setting up the correct Python and PyTorch dependencies can be difficult for non-programmers.
- Telemetry is enabled by default and requires an environment variable to turn off.
Writing custom PyTorch training scripts and manual Hugging Face data loaders.
Hugging Face transformers and a cloud GPU provider like RunPod or Vast.ai.
- unslothai/unsloth78k stars
Another popular fine-tuning tool that focuses on fast training speeds and low memory use on single graphics cards.
- hiyouga/LlamaFactory76k stars
A web-based and command-line framework for fine-tuning many different language models with minimal code.
- ggml-org/llama.cpp132k stars
A C and C++ library and command-line tool for running language models locally.
- EleutherAI/lm-evaluation-harness14k stars
Runs standard question sets against a language model and reports the scores.
- mudler/LocalAI50k stars
Serves text, vision, speech, image and video models from one self-hosted service that imitates the paid interfaces.
curl -LsSf https://astral.sh/uv/install.sh | sh export UV_TORCH_BACKEND=cu130 uv venv --python 3.12 source .venv/bin/activate uv pip install torch==2.14.0 torchvision uv pip install --no-build-isolation axolotl[deepspeed]