YuE2
An open model that turns lyrics and a style prompt into a full song, with an editable melody-and-chord plan first. YuE2 is the current version of the official YuE repository. The original YuE is kept on a separate branch.
- Type AI generation
- Difficulty Hard
- Hardware Official starting point is Linux, Python 3.12, and an NVIDIA GPU with BF16 support and 24 GB of VRAM. Output is 48 kHz stereo. Windows, AMD, Apple Silicon, Intel, and low-memory paths exist but are documented as experimental. Model files download from Hugging Face on first use.
- License Apache 2.0 (code); CC BY-NC 4.0 plus creator permission (weights)
What it does
Give YuE2 lyrics (with section tags such as [Verse] and [Chorus]) and a style prompt that can include genre, instruments, vocal character, language, and tempo. In the default mode it first writes a melody-and-chord plan, then renders a complete song with vocals and accompaniment. You can inspect or edit that plan and generate again. Other modes plan only the melody, skip the symbolic plan, or accept a score you supply.
The same model is also used for zero-shot covers and for back-and-forth edits to arrangement or lyrics. This is a research-grade local pipeline, not a polished consumer app. The project links a free online demo, but this page is about running the official repository yourself.
Why it matters for hip-hop
Lyrics-to-song is the useful part. You can draft a verse and hook, describe a pocket, and hear a sketch with a beat under it. The editable plan is the difference from a black-box clip: if the melody is close but the chords are wrong, you can change the plan instead of rolling the dice again. It is still a sketch machine. Vocal character is guided by the style prompt, not a guaranteed singer identity. The official docs say pronunciation tricks in the lyrics are not a real control channel.
Getting it running
The official quick start is: clone the repo, make a Python 3.12 virtual environment, install the package, then run the example generator. That is a serious local install. The 24 GB VRAM figure is why this is Hard. If your card is smaller, read the project's hardware doc before you spend an evening on it. The maintainers do not treat those other paths as fully validated.
License notes
Two different licenses apply. The code is Apache 2.0, a permissive software license. The weights are CC BY-NC 4.0, with an extra written permission in the project's model-license file. Under that permission, personal users, content creators, and musicians acting in an individual capacity may use the weights free of charge and may publish, sell, or otherwise monetize their outputs, with no fees or royalties owed to the model licensors. Academic use is free for non-commercial research and education. Companies that want to use the weights commercially need to contact the project about a commercial license. Redistributing or selling the weights themselves is not part of the creator permission. Credit is encouraged and optional for outputs. The permission requires responsible use: no illegal, harmful, deceptive, or unethical use. This is not legal advice, and it does not clear lyrics, samples, or reference recordings you did not create.