README.md

metadata

title: VR Game Music Generator
emoji: 🎵
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 3.50.2
app_file: app.py
python_version: 3.11
pinned: false

VR Game Music Generator

Generate music from text descriptions using the text2midi AI model. Designed for integration with Unity and other game engines via the Gradio API.

Features

• Text-to-music generation using AI
• Real-time audio streaming (no file persistence)
• RESTful API for game engine integration
• Supports various music styles and instruments

API Usage

Endpoint

POST https://YOUR-SPACE.hf.space/api/generate

Request

{
    "data": ["A cheerful pop song with piano and drums", 512, 0.9]
}

Parameters:

• data[0]: Music prompt (string)
• data[1]: Max length in tokens (256-2048, default: 512)
• data[2]: Temperature (0.1-1.5, default: 0.9)

Response

{
    "data": [
        {"path": "/file=...", "url": "https://...", "orig_name": "audio.wav"},
        "AI-generated audio for: 'A cheerful pop song...'"
    ]
}

Unity Integration

using UnityEngine;
using UnityEngine.Networking;
using System.Collections;

public class MusicGenerator : MonoBehaviour
{
    private const string API_URL = "https://YOUR-SPACE.hf.space/api/generate";

    public IEnumerator GenerateMusic(string prompt, System.Action<AudioClip> callback)
    {
        string json = $"{{\"data\": [\"{prompt}\", 512, 0.9]}}";

        using (UnityWebRequest request = new UnityWebRequest(API_URL, "POST"))
        {
            byte[] bodyRaw = System.Text.Encoding.UTF8.GetBytes(json);
            request.uploadHandler = new UploadHandlerRaw(bodyRaw);
            request.downloadHandler = new DownloadHandlerBuffer();
            request.SetRequestHeader("Content-Type", "application/json");

            yield return request.SendWebRequest();

            if (request.result == UnityWebRequest.Result.Success)
            {
                // Parse response and download audio from returned URL
                var response = JsonUtility.FromJson<GradioResponse>(request.downloadHandler.text);
                yield return DownloadAudio(response.data[0].url, callback);
            }
        }
    }

    private IEnumerator DownloadAudio(string url, System.Action<AudioClip> callback)
    {
        using (UnityWebRequest www = UnityWebRequestMultimedia.GetAudioClip(url, AudioType.WAV))
        {
            yield return www.SendWebRequest();
            if (www.result == UnityWebRequest.Result.Success)
            {
                callback(DownloadHandlerAudioClip.GetContent(www));
            }
        }
    }
}

Example Prompts

• A cheerful and melodic pop Christmas song featuring piano, acoustic guitar, and drums
• An energetic electronic trance track with synth bass and drums at 138 BPM
• A slow and emotional classical piece featuring cello and violin in C minor
• A cinematic electronic soundtrack with an epic and dark atmosphere
• Happy medieval tavern music with lute and flute

Local Development

pip install -r requirements.txt
python app.py

Credits

• Model: amaai-lab/text2midi
• Audio synthesis: FluidSynth with FluidR3 GM SoundFont