gabimtme
Joined 3 years ago
Joined 3 years ago
There's a model version that is able to generate music conditioned not only on natural language prompts, but also on other pieces of music, so it's possible to generate chunks of 10s where each chunk is generated based on the previous one.
The challenge with that model is that it's hard to export it in ONNX format so that it can be run outside of a machine learning framework in Python.
This is a terminal application that runs the latest AI models for music generation locally, using the CPU or GPU of the device, and without the need of heavy dependencies like Python or machine learning frameworks. It works on Linux, Mac and Windows seamlessly, with a binary size of just ~30 Mb for the non-GPU versions.
The app works like this:
- It accepts a natural language prompt from the user
- Generates a music sample conditioned by the prompt
- Encodes the generated sample into .wav format and plays it on the device
Additionally, it ships a UI that allows interacting with the AI models in a chat-like web application, storing chat history and generated music on the device.
The vision of the project is that it can eventually generate infinite music streams in real time, for example, an infinite stream of always new LoFi songs for listening while coding, but not quite there yet...
Hope you like it!
If a node is depended upon a lot, all the resulting edges induce weaker forces to adjacent nodes, so this accounts for the fact that some files will be depended upon a lot, and that's fine.
There's also the option to just exclude that kind of files from the analysis with the --exclude flag. I've found that to be useful for massive auto-generated files.
You can read more about the limitations here https://huggingface.co/facebook/musicgen-small