Spaces:
Running
A newer version of the Gradio SDK is available: 6.12.0
ACE-Step Gradio ãã¢ãŠãŒã¶ãŒã¬ã€ã
Language / è¯èš / èšèª: English | äžæ | æ¥æ¬èª
æ¬ã¬ã€ãã¯ACE-Step Gradio Webã€ã³ã¿ãŒãã§ãŒã¹ã䜿çšãã鳿¥œçæã®å æ¬çãªããã¥ã¡ã³ããæäŸãããã¹ãŠã®æ©èœãšèšå®ãå«ã¿ãŸãã
ç®æ¬¡
- ã¯ããã«
- ãµãŒãã¹èšå®
- çæã¢ãŒã
- å ¥åãã©ã¡ãŒã¿
- é«åºŠãªèšå®
- çµæã»ã¯ã·ã§ã³
- LoRAãã¬ãŒãã³ã°
- ãã³ããšãã¹ããã©ã¯ãã£ã¹
ã¯ããã«
ãã¢ã®èµ·å
# åºæ¬çãªèµ·å
python app.py
# äºååæåä»ã
python app.py --config acestep-v15-turbo --init-llm
# ç¹å®ã®ããŒãã§
python app.py --port 7860
ã€ã³ã¿ãŒãã§ãŒã¹æŠèŠ
Gradioã€ã³ã¿ãŒãã§ãŒã¹ã®ã¬ã€ã¢ãŠãïŒ
- èšå®ïŒæãããã¿ã¢ã³ãŒãã£ãªã³ïŒ- ãµãŒãã¹èšå®ãDiT/LMãã©ã¡ãŒã¿ãåºåãªãã·ã§ã³
- çæã¿ã - ã¡ã€ã³ã¯ãŒã¯ã¹ããŒã¹ãäžéšã«çæã¢ãŒãã©ãžãªã»ã¬ã¯ã¿ãŒïŒ
- Turbo/SFTã¢ãã«ïŒSimpleãCustomãRemixãRepaint
- Baseã¢ãã«ïŒSimpleãCustomãRemixãRepaintãExtractãLegoãComplete
- çµæã»ã¯ã·ã§ã³ - çæããããªãŒãã£ãªã®åçãã¹ã³ã¢ãªã³ã°ããããããã²ãŒã·ã§ã³
- ãã¬ãŒãã³ã°ã¿ã - ããŒã¿ã»ãããã«ããŒãšLoRAãã¬ãŒãã³ã°
ãµãŒãã¹èšå®
ã¢ãã«éžæ
| èšå® | 説æ |
|---|---|
| ãã§ãã¯ãã€ã³ããã¡ã€ã« | ãã¬ãŒãã³ã°æžã¿ã¢ãã«ãã§ãã¯ãã€ã³ããéžæïŒå©çšå¯èœãªå ŽåïŒ |
| ã¡ã€ã³ã¢ãã«ãã¹ | DiTã¢ãã«èšå®ãéžæïŒäŸïŒacestep-v15-turboãacestep-v15-turbo-shift3ïŒ |
| ããã€ã¹ | åŠçããã€ã¹ïŒautoïŒæšå¥šïŒãcudaããŸã㯠cpu |
5Hz LMèšå®
| èšå® | 説æ |
|---|---|
| 5Hz LMã¢ãã«ãã¹ | èšèªã¢ãã«ãéžæãå©çšå¯èœãªã¢ãã«ã¯GPUãã£ã¢ã«åºã¥ããŠèªåãã£ã«ã¿ãªã³ã°ãããŸã â äŸïŒ6-8GB GPUã§ã¯0.6Bã®ã¿ã24GB+ GPUã§ã¯ãã¹ãŠã®ãµã€ãºïŒ0.6Bã1.7Bã4BïŒã衚瀺ãããŸãã |
| 5Hz LMããã¯ãšã³ã | vllmïŒããé«éãVRAM â¥8GBã®NVIDIA GPUæšå¥šïŒãptïŒPyTorchããŠãããŒãµã«ãã©ãŒã«ããã¯ïŒããŸã㯠mlxïŒApple SiliconïŒãVRAM <8GBã®GPUã§ã¯ pt/mlx ã«å¶éãããŸãïŒvllmã®KVãã£ãã·ã¥ãã¡ã¢ãªãæ¶è²»ããããããïŒã |
| 5Hz LMãåæå | åæåæã«LMãèªã¿èŸŒãããã«ãã§ãã¯ïŒthinkingã¢ãŒãã«å¿ èŠïŒãVRAM â€6GBã®GPUïŒTier 1-2ïŒã§ã¯ããã©ã«ãã§ãã§ãã¯ãªãã»ç¡å¹ã |
ã¢ãããã£ãããã©ã«ã: ãã¹ãŠã®LMèšå®ã¯GPUã®VRAMãã£ã¢ã«åºã¥ããŠèªåæ§æãããŸããæšå¥šLMã¢ãã«ãããã¯ãšã³ããåæåç¶æ ã¯æé©ãªããã©ãŒãã³ã¹ã«äºåèšå®ãããŠããŸããæåã§äžæžãã§ããŸãããGPUãšäºææ§ã®ãªãéžæãããå Žåãã·ã¹ãã ãèŠåã衚瀺ããŸãã
ããã©ãŒãã³ã¹ãªãã·ã§ã³
| èšå® | 説æ |
|---|---|
| Flash Attentionãäœ¿çš | ããé«éãªæšè«ã®ããã«æå¹åïŒflash_attnããã±ãŒãžãå¿ èŠïŒ |
| CPUã«ãªãããŒã | ã¢ã€ãã«æã«ã¢ãã«ãCPUã«ãªãããŒãããŠGPUã¡ã¢ãªãç¯çŽãVRAM <20GBã®GPUã§ããã©ã«ãèªåæå¹ã |
| DiTãCPUã«ãªãããŒã | DiTã¢ãã«ãç¹ã«CPUã«ãªãããŒããVRAM <12GBã®GPUã§ããã©ã«ãèªåæå¹ã |
| INT8éåå | INT8éã¿éååã§ã¢ãã«ã®VRAM䜿çšéãåæžãVRAM <20GBã®GPUã§ããã©ã«ãèªåæå¹ã |
| ã¢ãã«ã³ã³ãã€ã« | æé©åæšè«ã®ãã torch.compile ãæå¹åããã¹ãŠã®ãã£ã¢ã§ããã©ã«ãæå¹ïŒéååãã¢ã¯ãã£ããªå Žåã«å¿
èŠïŒã |
ãã£ã¢å¯Ÿå¿èšå®: ãªãããŒããéååãã³ã³ãã€ã«ãªãã·ã§ã³ã¯GPUãã£ã¢ã«åºã¥ããŠèªåèšå®ãããŸããå®å šãªãã£ã¢ããŒãã«ã¯ GPU_COMPATIBILITY.md ãåç §ããŠãã ããã
LoRAã¢ããã¿ãŒ
| èšå® | 説æ |
|---|---|
| LoRAãã¹ | ãã¬ãŒãã³ã°æžã¿LoRAã¢ããã¿ãŒãã£ã¬ã¯ããªãžã®ãã¹ |
| LoRAãèªã¿èŸŒã¿ | æå®ãããLoRAã¢ããã¿ãŒãèªã¿èŸŒã¿ |
| ã¢ã³ããŒã | çŸåšèªã¿èŸŒãŸããŠããLoRAãåé€ |
| LoRAãäœ¿çš | æšè«çšã®èªã¿èŸŒãŸããLoRAãæå¹å/ç¡å¹å |
â ïž æ³šæïŒ PEFTãšTorchAOéã®äºææ§ã®åé¡ã«ãããéååãããã¢ãã«ã«LoRAã¢ããã¿ãŒãèªã¿èŸŒãããšã¯ã§ããŸãããLoRAã䜿çšããå¿ èŠãããå Žåã¯ãã¢ããã¿ãŒãèªã¿èŸŒãåã« INT8éåå ã None ã«èšå®ããŠãã ããã
åæå
ãµãŒãã¹ãåæå ãã¯ãªãã¯ããŠã¢ãã«ãèªã¿èŸŒã¿ãŸããã¹ããŒã¿ã¹ããã¯ã¹ã«ä»¥äžãå«ã鲿ãšç¢ºèªã衚瀺ãããŸãïŒ
- æ€åºãããGPUãã£ã¢ãšVRAM
- æå€§èš±å®¹æéãšããããµã€ãºïŒLMãåæåããããã©ããã«åºã¥ããŠåçã«èª¿æŽïŒ
- èªåä¿®æ£ãããäºææ§ã®ãªãèšå®ã«é¢ããèŠå
åæååŸããªãŒãã£ãªæé ãš ããããµã€ãº ã¹ã©ã€ããŒã¯ãã£ã¢ã®å¶éãåæ ããããã«èªåæŽæ°ãããŸãã
çæã¢ãŒã
çæã¿ãäžéšã®çæã¢ãŒãã©ãžãªã»ã¬ã¯ã¿ãŒã§ã¯ãŒã¯ãããŒã決å®ããŸããTurboãšSFTã¢ãã«ã¯4ã€ã®ã¢ãŒããBaseã¢ãã«ã¯ããã«3ã€ã®ã¢ãŒãã远å ã§æäŸããŸãã
Simpleã¢ãŒã
è¿ éãªèªç¶èšèªããŒã¹ã®é³æ¥œçæçšã«èšèšãããŠããŸãã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Simple ãéžæ
- ãæ²ã®èª¬æããã£ãŒã«ãã«èªç¶èšèªã®èª¬æãå ¥å
- ããŒã«ã«ãäžèŠãªå Žåã¯ãã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãããªãã·ã§ã³ã§ãã§ãã¯
- ãªãã·ã§ã³ã§åžæããããŒã«ã«èšèªãéžæ
- ãµã³ãã«ãäœæ ãã¯ãªãã¯ããŠcaptionãæè©ãã¡ã¿ããŒã¿ãçæ
- å±éãããã»ã¯ã·ã§ã³ã§çæãããã³ã³ãã³ãã確èª
- 鳿¥œãçæ ãã¯ãªãã¯ããŠãªãŒãã£ãªãäœæ
説æã®äŸïŒ
- ãéããªå€ã®ããã®æããããã³ã¬ã«ã®ã©ããœã³ã°ã
- ãéãããŒã¹ããããã®ã¢ããããŒããªãšã¬ã¯ããããã¯ãã³ã¹ãã¥ãŒãžãã¯ã
- ãã¢ã³ãŒã¹ãã£ãã¯ã®ã¿ãŒã®ã¡ã©ã³ã³ãªãã¯ãªã€ã³ãã£ãŒãã©ãŒã¯ã
- ãç ããããŒã§æŒå¥ãããžã£ãºããªãªã
ã©ã³ãã ãµã³ãã«ïŒ ð² ãã¿ã³ãã¯ãªãã¯ããŠã©ã³ãã ãªäŸã®èª¬æãèªã¿èŸŒã¿ãŸãã
Customã¢ãŒã
ãã¹ãŠã®çæãã©ã¡ãŒã¿ã®å®å šãªå¶åŸ¡ãæäŸããŸãïŒtext2musicïŒã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Custom ãéžæ
- Captionãšæè©ãã£ãŒã«ããæåã§å ¥å
- ãªãã·ã§ã³ã§åç §ãªãŒãã£ãªãã¢ããããŒãããŠã¹ã¿ã€ã«ã¬ã€ãã³ã¹
- ãªãã·ã§ã³ã®ã¡ã¿ããŒã¿ãèšå®ïŒBPMãããŒãDurationãªã©ïŒ
- ãªãã·ã§ã³ã§ ãã©ãŒããã ãã¯ãªãã¯ããŠLMã䜿çšããŠå ¥åã匷å
- å¿ èŠã«å¿ããŠé«åºŠãªèšå®ãæ§æ
- 鳿¥œãçæ ãã¯ãªãã¯ããŠãªãŒãã£ãªãäœæ
Remixã¢ãŒã
æ¢åã®ãªãŒãã£ãªã®ã¡ããã£æ§é ãç¶æããªããã¹ã¿ã€ã«ã倿Žã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Remix ãéžæ
- ãœãŒã¹ãªãŒãã£ãªãã¢ããããŒãïŒãªããã¯ã¹ããæ²ïŒ
- ã¿ãŒã²ããã¹ã¿ã€ã«ã説æããCaptionãèšè¿°
- ãªãã·ã§ã³ã§æè©ãä¿®æ£
- Remix匷床ïŒ0.0-1.0ïŒã調æŽïŒé«ã = å ã®æ§é ã«è¿ã
- 鳿¥œãçæ ãã¯ãªãã¯
ãŠãŒã¹ã±ãŒã¹ïŒ ã«ããŒããŒãžã§ã³ã®äœæãã¹ã¿ã€ã«è»¢éãæ²ã®ããªãšãŒã·ã§ã³çæã
Repaintã¢ãŒã
ãªãŒãã£ãªã®ç¹å®ã®æéã»ã°ã¡ã³ããåçæããæ®ãã¯ç¶æã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Repaint ãéžæ
- ãœãŒã¹ãªãŒãã£ãªãã¢ããããŒã
- ãªãã€ã³ãéå§ãšãªãã€ã³ãçµäºãèšå®ïŒç§ïŒ-1ã§ãã¡ã€ã«çµç«¯ïŒ
- ãªãã€ã³ãã»ã¯ã·ã§ã³ã®åžæããã³ã³ãã³ãã説æããCaptionãèšè¿°
- 鳿¥œãçæ ãã¯ãªãã¯
ãŠãŒã¹ã±ãŒã¹ïŒ åé¡ã®ããã»ã¯ã·ã§ã³ã®ä¿®æ£ãã»ã°ã¡ã³ãå ã®æè©å€æŽãæ²ã®å»¶é·ã
Extractã¢ãŒãïŒBaseã¢ãã«ã®ã¿ïŒ
ããã¯ã¹ãªãŒãã£ãªããç¹å®ã®æ¥œåšãã©ãã¯ãæœåº/åé¢ã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Extract ãéžæ
- ãœãŒã¹ãªãŒãã£ãªãã¢ããããŒã
- ããããããŠã³ããæœåºãããã©ãã¯åãéžæ
- 鳿¥œãçæ ãã¯ãªãã¯
å©çšå¯èœãªãã©ãã¯ïŒ vocalsãbacking_vocalsãdrumsãbassãguitarãkeyboardãpercussionãstringsãsynthãfxãbrassãwoodwinds
Legoã¢ãŒãïŒBaseã¢ãã«ã®ã¿ïŒ
æ¢åã®ãªãŒãã£ãªã«æ°ããæ¥œåšãã©ãã¯ã远å ã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Lego ãéžæ
- ãœãŒã¹ãªãŒãã£ãªãã¢ããããŒã
- ããããããŠã³ãã远å ãããã©ãã¯åãéžæ
- ãã©ãã¯ç¹æ§ã説æããCaptionãèšè¿°
- 鳿¥œãçæ ãã¯ãªãã¯
Completeã¢ãŒãïŒBaseã¢ãã«ã®ã¿ïŒ
æå®ãããæ¥œåšã§éšåçãªãã©ãã¯ã宿ïŒèªåã¢ã¬ã³ãžïŒã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã§ Complete ãéžæ
- ãœãŒã¹ãªãŒãã£ãªãã¢ããããŒã
- 远å ããè€æ°ã®ãã©ãã¯åãéžæ
- åžæããã¹ã¿ã€ã«ã説æããCaptionãèšè¿°
- 鳿¥œãçæ ãã¯ãªãã¯
å ¥åãã©ã¡ãŒã¿
ãªãŒãã£ãªå ¥å
| ãã£ãŒã«ã | 説æ |
|---|---|
| åç §ãªãŒãã£ãª | ã¹ã¿ã€ã«/é³è²ã¬ã€ãã³ã¹çšã®ãªãã·ã§ã³ãªãŒãã£ãªïŒCustomã¢ãŒãã§è¡šç€ºïŒ |
| ãœãŒã¹ãªãŒãã£ãª | RemixãRepaintãExtractãLegoãCompleteã¢ãŒãã«å¿ é |
| ã³ãŒãã«å€æ | ãœãŒã¹ãªãŒãã£ãªãã5Hzã»ãã³ãã£ãã¯ã³ãŒããæœåº |
LMã³ãŒããã³ãïŒCustomã¢ãŒãïŒ
äºåèšç®ããããªãŒãã£ãªã»ãã³ãã£ãã¯ã³ãŒããããã«è²Œãä»ããŠçæãã¬ã€ãã§ããŸãããã©ã³ã¹ã¯ã©ã€ã ãã¿ã³ã䜿çšããŠã³ãŒããåæãã¡ã¿ããŒã¿ãæœåºããŸããããã¯ãœãŒã¹ãªãŒãã£ãªãã¢ããããŒãããã«ã¡ããã£æ§é ãå¶åŸ¡ããããã®é«åºŠãªæ©èœã§ãã
鳿¥œãã£ãã·ã§ã³
åžæãã鳿¥œã®ããã¹ã説æã以äžã«ã€ããŠå ·äœçã«ïŒ
- ãžã£ã³ã«ãšã¹ã¿ã€ã«
- 楜åš
- ã ãŒããšé°å²æ°
- ãã³ãæïŒBPMãæå®ããªãå ŽåïŒ
äŸïŒ ããšã¬ãã®ã¿ãŒãå匷ããã©ã ããã£ãããŒãªã·ã³ã»ããã¯ã®ã¢ããããŒããªãããããã¯ã
ð² ãã¯ãªãã¯ããŠã©ã³ãã ãªäŸã®captionãèªã¿èŸŒã¿ãŸãã
æè©
æ§é ã¿ã°ä»ãã®æè©ãå ¥åïŒ
[Verse 1]
仿¥è¡ãæ©ããŠããŠ
åãèšã£ãŠããèšèãæãåºããŠãã
[Chorus]
åã«é²ãã§ããã匷ããã
ãããåã®å±
å Žæ
[Verse 2]
...
ã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãã§ãã¯ããã¯ã¹ïŒ ããããã§ãã¯ãããšãæè©ã®å 容ã«é¢ä¿ãªãã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«é³æ¥œãçæããŸãã
ããŒã«ã«èšèªïŒ ããŒã«ã«ã®èšèªãéžæãèªåæ€åºãŸãã¯ã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãã©ãã¯ã«ã¯ãunknownãã䜿çšã
ãã©ãŒããããã¿ã³ïŒ ã¯ãªãã¯ããŠ5Hz LMã䜿çšããŠcaptionãšæè©ã匷åã
ãªãã·ã§ã³ãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| BPM | èªå | 1åãããã®ããŒãæ°ïŒ30-300ïŒ |
| ããŒã¹ã±ãŒã« | èªå | 鳿¥œããŒïŒäŸïŒãC MajorãããAmãããF# minorãïŒ |
| æåèšå· | èªå | æåèšå·ïŒ2ïŒ2/4ïŒã3ïŒ3/4ïŒã4ïŒ4/4ïŒã6ïŒ6/8ïŒ |
| ãªãŒãã£ãªé· | èªå/-1 | ç®æšé·ïŒç§ïŒïŒ10-600ïŒã-1ã§èªå |
| ããããµã€ãº | 2 | çæãããªãŒãã£ãªããªãšãŒã·ã§ã³ã®æ°ïŒ1-8ïŒ |
é«åºŠãªèšå®
DiTãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| æšè«ã¹ããã | 8 | ããã€ãºã¹ããããTurboïŒ1-20ãBaseïŒ1-200 |
| ã¬ã€ãã³ã¹ã¹ã±ãŒã« | 7.0 | CFG匷床ïŒbaseã¢ãã«ã®ã¿ïŒãé«ã = ããã³ããã«ããåŸã |
| ã·ãŒã | -1 | ã©ã³ãã ã·ãŒãããããã«ã¯ã«ã³ãåºåãã®å€ãäœ¿çš |
| ã©ã³ãã ã·ãŒã | â | ãã§ãã¯æã«ã©ã³ãã ã·ãŒããçæ |
| ãªãŒãã£ãªåœ¢åŒ | mp3 | åºå圢åŒïŒmp3ãflac |
| ã·ãã | 3.0 | ã¿ã€ã ã¹ãããã·ããä¿æ°ïŒ1.0-5.0ïŒãturboã«ã¯3.0æšå¥š |
| æšè«æ¹æ³ | ode | odeïŒEulerãããé«éïŒãŸãã¯sdeïŒç¢ºççïŒ |
| ã«ã¹ã¿ã ã¿ã€ã ã¹ããã | - | ã¿ã€ã ã¹ãããããªãŒããŒã©ã€ãïŒäŸïŒã0.97,0.76,0.615,0.5,0.395,0.28,0.18,0.085,0ãïŒ |
Baseã¢ãã«ã®ã¿ã®ãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| ADGãäœ¿çš | â | ããè¯ãå質ã®ããã«é©å¿ãã¥ã¢ã«ã¬ã€ãã³ã¹ãæå¹å |
| CFGåºééå§ | 0.0 | CFGãé©çšãå§ããã¿ã€ãã³ã°ïŒ0.0-1.0ïŒ |
| CFGåºéçµäº | 1.0 | CFGã®é©çšã忢ããã¿ã€ãã³ã°ïŒ0.0-1.0ïŒ |
LMãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| LM枩床 | 0.85 | ãµã³ããªã³ã°æž©åºŠïŒ0.0-2.0ïŒãé«ã = ããåµé ç |
| LM CFGã¹ã±ãŒã« | 2.0 | LMã¬ã€ãã³ã¹åŒ·åºŠïŒ1.0-3.0ïŒ |
| LM Top-K | 0 | Top-Kãµã³ããªã³ã°ã0ã§ç¡å¹ |
| LM Top-P | 0.9 | æ žãµã³ããªã³ã°ïŒ0.0-1.0ïŒ |
| LMãã¬ãã£ãããã³ãã | "NO USER INPUT" | CFGçšã®ãã¬ãã£ãããã³ãã |
CoTïŒæèã®é£éïŒãªãã·ã§ã³
| ãªãã·ã§ã³ | ããã©ã«ã | 説æ |
|---|---|---|
| CoT Metas | â | LMæšè«ã§ã¡ã¿ããŒã¿ãçæ |
| CoT Language | â | LMã§ããŒã«ã«èšèªãæ€åº |
| å¶çŽä»ããã³ãŒãã£ã³ã°ãããã° | â | ãããã°ãã°ãæå¹å |
çæãªãã·ã§ã³
| ãªãã·ã§ã³ | ããã©ã«ã | 説æ |
|---|---|---|
| LMã³ãŒã匷床 | 1.0 | LMã³ãŒããçæã«äžãã圱é¿ã®åŒ·ãïŒ0.0-1.0ïŒ |
| èªåã¹ã³ã¢ | â | å質ã¹ã³ã¢ãèªåèšç® |
| èªåLRC | â | æè©ã¿ã€ã ã¹ã¿ã³ããèªåçæ |
| LMããããã£ã³ã¯ãµã€ãº | 8 | LMããããããã®æå€§ã¢ã€ãã æ°ïŒGPUã¡ã¢ãªïŒ |
ã¡ã€ã³çæã³ã³ãããŒã«
| ã³ã³ãããŒã« | 説æ |
|---|---|
| Think | ã³ãŒãçæãšã¡ã¿ããŒã¿çšã®5Hz LMãæå¹å |
| ParallelThinking | 䞊åLMãããåŠçãæå¹å |
| CaptionRewrite | LMã«å ¥åcaptionã匷åããã |
| AutoGen | å®äºåŸã«æ¬¡ã®ããããèªåéå§ |
çµæã»ã¯ã·ã§ã³
çæããããªãŒãã£ãª
ããããµã€ãºã«åºã¥ããŠæå€§8ã€ã®ãªãŒãã£ãªãµã³ãã«ã衚瀺ãããŸããåãµã³ãã«ã«ã¯ä»¥äžãå«ãŸããŸãïŒ
- ãªãŒãã£ãªãã¬ãŒã€ãŒ - çæããããªãŒãã£ãªã®åçãäžæåæ¢ãããŠã³ããŒã
- ãœãŒã¹ã«éä¿¡ - ãã®ãªãŒãã£ãªããœãŒã¹ãªãŒãã£ãªå ¥åã«éä¿¡ããŠããã«åŠç
- ä¿å - ãªãŒãã£ãªãšã¡ã¿ããŒã¿ãJSONãã¡ã€ã«ã«ä¿å
- ã¹ã³ã¢ - ããŒãã¬ãã·ãã£ããŒã¹ã®å質ã¹ã³ã¢ãèšç®
- LRC - æè©ã¿ã€ã ã¹ã¿ã³ããçæïŒLRC圢åŒïŒ
詳现ã¢ã³ãŒãã£ãªã³
ãã¹ã³ã¢ & LRC & LMã³ãŒãããã¯ãªãã¯ããŠå±éãã以äžã衚瀺ïŒ
- LMã³ãŒã - ãã®ãµã³ãã«ã®5Hzã»ãã³ãã£ãã¯ã³ãŒã
- å質ã¹ã³ã¢ - ããŒãã¬ãã·ãã£ããŒã¹ã®å質ã¡ããªãã¯
- æè©ã¿ã€ã ã¹ã¿ã³ã - LRC圢åŒã®ã¿ã€ãã³ã°ããŒã¿
ãããããã²ãŒã·ã§ã³
| ã³ã³ãããŒã« | 説æ |
|---|---|
| â åãž | åã®ãããã衚瀺 |
| ãããã€ã³ãžã±ãŒã¿ãŒ | çŸåšã®ãããäœçœ®ã衚瀺ïŒäŸïŒãããã 1 / 3ãïŒ |
| 次ãããã¹ããŒã¿ã¹ | ããã¯ã°ã©ãŠã³ãçæã®é²æã衚瀺 |
| 次㞠ⶠ| 次ã®ãããã衚瀺ïŒAutoGenããªã³ã®å Žåã¯çæãããªã¬ãŒïŒ |
ãã©ã¡ãŒã¿ã®åŸ©å
ãããã®èšå®ãUIã«é©çš ãã¯ãªãã¯ããŠãçŸåšã®ããããããã¹ãŠã®çæãã©ã¡ãŒã¿ãå ¥åãã£ãŒã«ãã«åŸ©å ãè¯ãçµæãå埩ããã®ã«äŸ¿å©ã
ãããçµæ
ããããçµæãšçæè©³çްãã¢ã³ãŒãã£ãªã³ã«ã¯ä»¥äžãå«ãŸããŸãïŒ
- ãã¹ãŠã®çæãã¡ã€ã« - ãã¹ãŠã®ããããããã¹ãŠã®ãã¡ã€ã«ãããŠã³ããŒã
- çæè©³çް - çæããã»ã¹ã«é¢ãã詳现æ å ±
LoRAãã¬ãŒãã³ã°
LoRAãã¬ãŒãã³ã°ã¿ãã¯ã«ã¹ã¿ã LoRAã¢ããã¿ãŒãäœæããããã®ããŒã«ãæäŸããŸãã
ð å æ¬çãªã¹ããããã€ã¹ãããã¬ã€ãïŒããŒã¿æºåãã¢ãããŒã·ã§ã³ãååŠçãèšç·Žããšã¯ã¹ããŒãïŒã«ã€ããŠã¯ãLoRA ãã¬ãŒãã³ã°ãã¥ãŒããªã¢ã«ãåç §ããŠãã ããã
ããŒã¿ã»ãããã«ããŒã¿ã
ã¹ããã1ïŒèªã¿èŸŒã¿ãŸãã¯ã¹ãã£ã³
ãªãã·ã§ã³AïŒæ¢åã®ããŒã¿ã»ãããèªã¿èŸŒã¿
- 以åä¿åããããŒã¿ã»ããJSONãžã®ãã¹ãå ¥å
- èªã¿èŸŒã¿ ãã¯ãªãã¯
ãªãã·ã§ã³BïŒæ°ãããã£ã¬ã¯ããªãã¹ãã£ã³
- ãªãŒãã£ãªãã©ã«ããžã®ãã¹ãå ¥å
- ã¹ãã£ã³ ãã¯ãªãã¯ããŠãªãŒãã£ãªãã¡ã€ã«ãæ€çŽ¢ïŒwavãmp3ãflacãoggãopusïŒ
ã¹ããã2ïŒããŒã¿ã»ããã®èšå®
| èšå® | 説æ |
|---|---|
| ããŒã¿ã»ããå | ããŒã¿ã»ããã®åå |
| ãã¹ãŠã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã« | ãã¹ãŠã®ãã©ãã¯ã«ããŒã«ã«ããªãå Žåã«ãã§ã㯠|
| ã«ã¹ã¿ã ã¢ã¯ãã£ããŒã·ã§ã³ã¿ã° | ãã®LoRAã®ã¹ã¿ã€ã«ãã¢ã¯ãã£ãã«ãããŠããŒã¯ãªã¿ã° |
| ã¿ã°äœçœ® | ã¿ã°ãé 眮ããå ŽæïŒåã«è¿œå ãåŸã«è¿œå ããŸãã¯captionã眮æ |
ã¹ããã3ïŒèªåã©ãã«
ãã¹ãŠèªåã©ãã« ãã¯ãªãã¯ããŠãã¹ãŠã®ãªãŒãã£ãªãã¡ã€ã«ã®ã¡ã¿ããŒã¿ãçæïŒ
- CaptionïŒé³æ¥œã®èª¬æïŒ
- BPM
- ããŒ
- æåèšå·
Metasãã¹ããã ãªãã·ã§ã³ã¯LLMã©ããªã³ã°ãã¹ãããããŠN/Aå€ã䜿çšããŸãã
ã¹ããã4ïŒãã¬ãã¥ãŒãšç·šé
ã¹ã©ã€ããŒã䜿çšããŠãµã³ãã«ãéžæããæåã§ç·šéïŒ
- Caption
- æè©
- BPMãããŒãæåèšå·
- èšèª
- ã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãã©ã°
倿Žãä¿å ãã¯ãªãã¯ããŠãµã³ãã«ãæŽæ°ã
ã¹ããã5ïŒããŒã¿ã»ãããä¿å
ä¿åãã¹ãå ¥åããããŒã¿ã»ãããä¿å ãã¯ãªãã¯ããŠJSONãšããŠãšã¯ã¹ããŒãã
ã¹ããã6ïŒååŠç
é«éãã¬ãŒãã³ã°ã®ããã«ããŒã¿ã»ãããäºåèšç®ãã³ãœã«ã«å€æïŒ
- ãªãã·ã§ã³ã§æ¢åã®ããŒã¿ã»ããJSONãèªã¿èŸŒã¿
- ãã³ãœã«åºåãã£ã¬ã¯ããªãèšå®
- ååŠç ãã¯ãªãã¯
ããã«ãããªãŒãã£ãªãVAEæœåšå€æ°ã«ãšã³ã³ãŒããããããã¹ããåã蟌ã¿ã«ãšã³ã³ãŒããããæ¡ä»¶ãšã³ã³ãŒããŒãå®è¡ãããŸãã
LoRAãã¬ãŒãã³ã°ã¿ã
ããŒã¿ã»ããéžæ
ååŠçããããã³ãœã«ãã£ã¬ã¯ããªãžã®ãã¹ãå ¥åããããŒã¿ã»ãããèªã¿èŸŒã¿ ãã¯ãªãã¯ã
LoRAèšå®
| èšå® | ããã©ã«ã | 説æ |
|---|---|---|
| LoRAã©ã³ã¯ (r) | 64 | LoRAã®å®¹éãé«ã = ããå€ãã®å®¹éãããå€ãã®ã¡ã¢ãª |
| LoRA Alpha | 128 | ã¹ã±ãŒãªã³ã°ä¿æ°ïŒéåžžã¯ã©ã³ã¯ã®2åïŒ |
| LoRA Dropout | 0.1 | æ£ååã®ããã®ããããã¢ãŠãç |
ãã¬ãŒãã³ã°ãã©ã¡ãŒã¿
| èšå® | ããã©ã«ã | 説æ |
|---|---|---|
| åŠç¿ç | 1e-4 | æé©ååŠç¿ç |
| æå€§ãšãã㯠| 500 | æå€§ãã¬ãŒãã³ã°ãšãã㯠|
| ããããµã€ãº | 1 | ãã¬ãŒãã³ã°ããããµã€ãº |
| åŸé çŽ¯ç© | 1 | æå¹ããã = batch_size à accumulation |
| Nãšããã¯ããšã«ä¿å | 200 | ãã§ãã¯ãã€ã³ãä¿åé »åºŠ |
| ã·ãã | 3.0 | turboã¢ãã«ã®ã¿ã€ã ã¹ãããã·ãã |
| ã·ãŒã | 42 | åçŸæ§ã®ããã®ã©ã³ãã ã·ãŒã |
ãã¬ãŒãã³ã°ã³ã³ãããŒã«
- ãã¬ãŒãã³ã°éå§ - ãã¬ãŒãã³ã°ããã»ã¹ãéå§
- ãã¬ãŒãã³ã°åæ¢ - ãã¬ãŒãã³ã°ãäžæ
- ãã¬ãŒãã³ã°é²æ - çŸåšã®ãšããã¯ãšãã¹ã衚瀺
- ãã¬ãŒãã³ã°ãã° - 詳现ãªãã¬ãŒãã³ã°åºå
- ãã¬ãŒãã³ã°ãã¹ãããã - èŠèŠçãªãã¹æ²ç·
LoRAã®ãšã¯ã¹ããŒã
ãã¬ãŒãã³ã°åŸãæçµã¢ããã¿ãŒããšã¯ã¹ããŒãïŒ
- ãšã¯ã¹ããŒããã¹ãå ¥å
- LoRAããšã¯ã¹ããŒã ãã¯ãªãã¯
ãã³ããšãã¹ããã©ã¯ãã£ã¹
æé«å質ã®ããã«
- thinkingã¢ãŒããäœ¿çš - LM匷åçæã®ããã«ãThinkããã§ãã¯ããã¯ã¹ãæå¹ã«ä¿ã€
- captionãå ·äœçã« - ãžã£ã³ã«ã楜åšãã ãŒããã¹ã¿ã€ã«ã®è©³çްãå«ãã
- LMã«ã¡ã¿ããŒã¿ãæ€åºããã - èªåæ€åºã®ããã«BPM/ããŒ/Durationã空ã®ãŸãŸã«ãã
- ãããçæãäœ¿çš - 2-4ã®ããªãšãŒã·ã§ã³ãçæããæè¯ã®ãã®ãéžã¶
ããé«éãªçæã®ããã«
- turboã¢ãã«ãäœ¿çš -
acestep-v15-turboãŸãã¯acestep-v15-turbo-shift3ãéžæ - æšè«ã¹ãããã8ã«ä¿ã€ - turboã«æé©ãªããã©ã«ã
- ããããµã€ãºãæžãã - è¿ éãªçµæãå¿ èŠãªå Žåã¯ããããµã€ãºãäžãã
- AutoGenãç¡å¹å - ãããçæã®æåå¶åŸ¡
äžè²«ããçµæã®ããã«
- ç¹å®ã®ã·ãŒããèšå® - ãã©ã³ãã ã·ãŒããã®ãã§ãã¯ãå€ããŠã·ãŒãå€ãå ¥å
- è¯ãçµæãä¿å - åçŸã®ããã«ãã©ã¡ãŒã¿ããšã¯ã¹ããŒãããããã«ãä¿åãã䜿çš
- ããããã®èšå®ãé©çšããäœ¿çš - è¯ãããããããã©ã¡ãŒã¿ã埩å
é·å°ºé³æ¥œã®ããã«
- æç€ºçãªdurationãèšå® - ç§åäœã§durationãæå®
- repaintã¿ã¹ã¯ãäœ¿çš - åæçæåŸã«åé¡ã®ããã»ã¯ã·ã§ã³ãä¿®æ£
- çæããã§ãŒã³ - 以åã®çµæã®äžã«æ§ç¯ããããã«ããœãŒã¹ã«éä¿¡ãã䜿çš
ã¹ã¿ã€ã«ã®äžè²«æ§ã®ããã«
- LoRAããã¬ãŒãã³ã° - ããªãã®ã¹ã¿ã€ã«çšã®ã«ã¹ã¿ã ã¢ããã¿ãŒãäœæ
- åç §ãªãŒãã£ãªãäœ¿çš - ãªãŒãã£ãªã¢ããããŒãã§ã¹ã¿ã€ã«åç §ãã¢ããããŒã
- äžè²«ããcaptionãäœ¿çš - é¡äŒŒã®èª¬æçãªèšèªãç¶æ
ãã©ãã«ã·ã¥ãŒãã£ã³ã°
ãªãŒãã£ãªãçæãããªãïŒ
- ã¢ãã«ãåæåãããŠããããšã確èªïŒç·ã®ã¹ããŒã¿ã¹ã¡ãã»ãŒãžïŒ
- thinkingã¢ãŒãã䜿çšããŠããå Žåã¯5Hz LMãåæåãããŠããããšã確èª
- ãšã©ãŒã¡ãã»ãŒãžã®ã¹ããŒã¿ã¹åºåã確èª
çµæã®åè³ªãæªãïŒ
- æšè«ã¹ããããå¢ããïŒbaseã¢ãã«ã®å ŽåïŒ
- ã¬ã€ãã³ã¹ã¹ã±ãŒã«ã調æŽ
- ç°ãªãã·ãŒãã詊ã
- captionãããå ·äœçã«ãã
ã¡ã¢ãªäžè¶³ïŒOOMïŒïŒ
- ã·ã¹ãã ã¯èªåVRAMã¬ãŒãïŒãããèªååæžïŒãšã¢ãããã£ãVAEãã³ãŒãïŒCPUãã©ãŒã«ããã¯ïŒãå«ã¿ãŸããããã§ãOOMãçºçããå ŽåïŒ
- æåã§ããããµã€ãºãæžãã
- CPUãªãããŒããæå¹åïŒVRAM <20GBã§ã¯èªåæå¹ã®ã¯ãïŒ
- INT8éååãæå¹åïŒVRAM <20GBã§ã¯èªåæå¹ã®ã¯ãïŒ
- LMããããã£ã³ã¯ãµã€ãºãæžãã
- åãã£ã¢ã®æšå¥šèšå®ã¯ GPU_COMPATIBILITY.md ãåç §
LMãæ©èœããªãïŒ
- åæåæã«ã5Hz LMãåæåãããã§ãã¯ãããŠããããšã確èªïŒVRAM â€6GBã®GPUã§ã¯ããã©ã«ãç¡å¹ïŒ
- æå¹ãªLMã¢ãã«ãã¹ãéžæãããŠããããšã確èªïŒãã£ã¢äºæã¢ãã«ã®ã¿è¡šç€ºïŒ
- vllmãŸãã¯PyTorchããã¯ãšã³ããå©çšå¯èœã§ããããšã確èªïŒVRAM <8GBã§ã¯vllmå¶éïŒ
- LMãã§ãã¯ããã¯ã¹ãã°ã¬ãŒã¢ãŠãããŠããå ŽåãGPUãã£ã¢ãLMããµããŒãããŠããŸãã â DiTã®ã¿ã¢ãŒãã䜿çš
ããŒããŒãã·ã§ãŒãã«ãã
Gradioã€ã³ã¿ãŒãã§ãŒã¹ã¯æšæºçãªWebã·ã§ãŒãã«ããããµããŒãïŒ
- Tab - å ¥åãã£ãŒã«ãéãç§»å
- Enter - ããã¹ãå ¥åãéä¿¡
- Space - ãã§ãã¯ããã¯ã¹ãåãæ¿ã
èšèªãµããŒã
ã€ã³ã¿ãŒãã§ãŒã¹ã¯è€æ°ã®UIèšèªããµããŒãïŒ
- è±èª (en)
- äžåœèª (zh)
- æ¥æ¬èª (ja)
ãµãŒãã¹èšå®ã»ã¯ã·ã§ã³ã§å¥œã¿ã®èšèªãéžæããŠãã ããã
詳现ã«ã€ããŠã¯ä»¥äžãåç §ïŒ
- ã¡ã€ã³READMEïŒ
../../README.md - REST APIããã¥ã¡ã³ãïŒ
API.md - Pythonæšè«APIïŒ
INFERENCE.md