README.md

---
title: VR Game Music Generator
emoji: 🎵
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 3.50.2
app_file: app.py
python_version: 3.11
pinned: false
---

# VR Game Music Generator

Generate music from text descriptions using the text2midi AI model. Designed for integration with Unity and other game engines via the Gradio API.

## Features

- Text-to-music generation using AI
- Real-time audio streaming (no file persistence)
- RESTful API for game engine integration
- Supports various music styles and instruments

## API Usage

### Endpoint
```
POST https://YOUR-SPACE.hf.space/api/generate
```

### Request
```json
{
    "data": ["A cheerful pop song with piano and drums", 512, 0.9]
}
```

Parameters:
- `data[0]`: Music prompt (string)
- `data[1]`: Max length in tokens (256-2048, default: 512)
- `data[2]`: Temperature (0.1-1.5, default: 0.9)

### Response
```json
{
    "data": [
        {"path": "/file=...", "url": "https://...", "orig_name": "audio.wav"},
        "AI-generated audio for: 'A cheerful pop song...'"
    ]
}
```

## Unity Integration

```csharp
using UnityEngine;
using UnityEngine.Networking;
using System.Collections;

public class MusicGenerator : MonoBehaviour
{
    private const string API_URL = "https://YOUR-SPACE.hf.space/api/generate";

    public IEnumerator GenerateMusic(string prompt, System.Action<AudioClip> callback)
    {
        string json = $"{{\"data\": [\"{prompt}\", 512, 0.9]}}";

        using (UnityWebRequest request = new UnityWebRequest(API_URL, "POST"))
        {
            byte[] bodyRaw = System.Text.Encoding.UTF8.GetBytes(json);
            request.uploadHandler = new UploadHandlerRaw(bodyRaw);
            request.downloadHandler = new DownloadHandlerBuffer();
            request.SetRequestHeader("Content-Type", "application/json");

            yield return request.SendWebRequest();

            if (request.result == UnityWebRequest.Result.Success)
            {
                // Parse response and download audio from returned URL
                var response = JsonUtility.FromJson<GradioResponse>(request.downloadHandler.text);
                yield return DownloadAudio(response.data[0].url, callback);
            }
        }
    }

    private IEnumerator DownloadAudio(string url, System.Action<AudioClip> callback)
    {
        using (UnityWebRequest www = UnityWebRequestMultimedia.GetAudioClip(url, AudioType.WAV))
        {
            yield return www.SendWebRequest();
            if (www.result == UnityWebRequest.Result.Success)
            {
                callback(DownloadHandlerAudioClip.GetContent(www));
            }
        }
    }
}
```

## Example Prompts

- A cheerful and melodic pop Christmas song featuring piano, acoustic guitar, and drums
- An energetic electronic trance track with synth bass and drums at 138 BPM
- A slow and emotional classical piece featuring cello and violin in C minor
- A cinematic electronic soundtrack with an epic and dark atmosphere
- Happy medieval tavern music with lute and flute

## Local Development

```bash
pip install -r requirements.txt
python app.py
```

## Credits

- Model: [amaai-lab/text2midi](https://huggingface.co/amaai-lab/text2midi)
- Audio synthesis: FluidSynth with FluidR3 GM SoundFont