47.3k stars · Apache-2.0 · 3.3.2 (2026-09-17) · Track this in Scout
Runs chains of scheduled jobs written in Python, and keeps a record of every run.
▶Repo detailsthe review · specs · pros & cons · install
What it does
You write the chain in Python as a set of tasks and the order between them. It schedules those chains, runs each task, and stores the outcome. A web interface shows every run as a grid, so a failed step is one click from its log. Failed tasks retry on a policy you set, and a chain can be re-run from the step that broke rather than from the beginning. It ships with connectors for a long list of outside systems, including the major clouds, common databases and message queues, published as separate provider packages so they update on their own schedules. Version 3 moved task execution behind an API, which means tasks no longer need direct database access, and that was the biggest change in the project's history. It is not a tool for reacting to single events in milliseconds, and it is not a data processing engine: it decides what runs and when, while the actual work happens in the code or the systems it calls.
Why it matters
Who it suits. Anyone whose nightly work has outgrown a list of cron entries, and who needs to know which step failed and be able to re-run just that step. The large connector library makes it strongest where data has to move between several outside systems. Skip it if there are fewer than about five jobs, because a scheduler of this size costs more to run and watch than the problem it solves.
Verdict. Worth it only above a certain size, and the honest advice is to be sure you have reached that size. Code was pushed this morning and version 3.3.2 landed on 17 September 2026, so there is no doubt it is alive and well staffed. The cost is real: a database, a scheduler process, workers, and a few gigabytes of memory before any of your own work runs. For a smaller setup, PrefectHQ/prefect from Edition 55 adds scheduling to existing Python functions with two decorators, and kestra-io/kestra from Edition 45 describes jobs in YAML instead of Python, which some people prefer and others find limiting. dagster-io/dagster is the main rival at the same weight and organises work around the data produced rather than the tasks performed.
- The run history and the grid view make a failure easy to find, which is the whole point of a scheduler.
- A very large library of connectors to outside systems, maintained separately from the core.
- Enormous community, so almost every problem has already been asked about in public.
- Heavy. A database, a scheduler and workers, and a few gigabytes of memory, before your own jobs.
- The jump from version 2 to version 3 changed how tasks run, so older guides can mislead.
- Installing it needs a constraints file to pin versions, and ignoring that causes dependency conflicts.
PrefectHQ/prefectApache-2.0, covered in Edition 55; adds scheduling, retries and run history to existing Python functions with two decorators, and is far lighter to run.
Track this in Scout- dagster-io/dagster
Apache-2.0, the same weight class, but organised around the data each step produces rather than the steps themselves.
Track this in Scout 
kestra-io/kestraApache-2.0, covered in Edition 45; the same job with workflows written in YAML rather than Python.
Track this in Scout
python3 -m venv venv source venv/bin/activate pip install "apache-airflow==3.3.2" \ --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.3.2/constraints-3.12.txt" airflow standalone
