In simple English
This computer program turns text descriptions or still pictures into ten second videos with matching sound. It needs eight professional graphics cards with massive memory to run. You download huge model files from the internet before you can start. The software is free to use under an open source licence. Most solo builders cannot run this on normal computers.
What it is
Magi-2 Preview is an open weights video generation model built on a mixture of experts architecture. It generates low resolution video in a preview stage and scales it up to high definition in a refiner stage.
What it is good for
Researchers and developers with access to high end server hardware who want to test state of the art video generation models.
- Generates synchronized sound alongside the video clip.
- Supports both text-to-video and image-to-video generation.
- Uses an open source Apache 2.0 licence.
- Requires eight NVIDIA Hopper graphics cards with 80 GB of memory each.
- Needs about 307 GB of disk space to store the model weights.
- Setup is difficult and requires advanced knowledge of container tools and Python.
- QwenLM/Qwen328k stars
A large language model family that provides text encoding capabilities similar to the text encoder used here.
- mpv-player/mpv38k stars
A command-line media player that handles most audio, video and subtitle formats and can be driven entirely from scripts.
- snapcast/snapcast8.0k stars
A server and clients that play one audio stream in several rooms, kept in time with each other.
- MusicPlayerDaemon/MPD2.8k stars
A background music player with no interface of its own, controlled over a network by separate client programs.
docker pull sandai/magi-2-preview:latest docker run --gpus all -it -v /path/to/ckpt:/workspace/ckpt sandai/magi-2-preview:latest