Spaces:
Running
on
A100
ACE-Step Gradio ãã¢ãŠãŒã¶ãŒã¬ã€ã
Language / è¯èš / èšèª: English | äžæ | æ¥æ¬èª
æ¬ã¬ã€ãã¯ACE-Step Gradio Webã€ã³ã¿ãŒãã§ãŒã¹ã䜿çšãã鳿¥œçæã®å æ¬çãªããã¥ã¡ã³ããæäŸãããã¹ãŠã®æ©èœãšèšå®ãå«ã¿ãŸãã
ç®æ¬¡
- ã¯ããã«
- ãµãŒãã¹èšå®
- çæã¢ãŒã
- ã¿ã¹ã¯ã¿ã€ã
- å ¥åãã©ã¡ãŒã¿
- é«åºŠãªèšå®
- çµæã»ã¯ã·ã§ã³
- LoRAãã¬ãŒãã³ã°
- ãã³ããšãã¹ããã©ã¯ãã£ã¹
ã¯ããã«
ãã¢ã®èµ·å
# åºæ¬çãªèµ·å
python app.py
# äºååæåä»ã
python app.py --config acestep-v15-turbo --init-llm
# ç¹å®ã®ããŒãã§
python app.py --port 7860
ã€ã³ã¿ãŒãã§ãŒã¹æŠèŠ
Gradioã€ã³ã¿ãŒãã§ãŒã¹ã¯ä»¥äžã®äž»èŠã»ã¯ã·ã§ã³ã§æ§æãããŠããŸãïŒ
- ãµãŒãã¹èšå® - ã¢ãã«ã®èªã¿èŸŒã¿ãšåæå
- å¿ é å ¥å - ã¿ã¹ã¯ã¿ã€ãããªãŒãã£ãªã¢ããããŒããçæã¢ãŒã
- 鳿¥œãã£ãã·ã§ã³ãšæè© - çæçšã®ããã¹ãå ¥å
- ãªãã·ã§ã³ãã©ã¡ãŒã¿ - BPMãããŒãdurationãªã©ã®ã¡ã¿ããŒã¿
- é«åºŠãªèšå® - çæã®çްããå¶åŸ¡
- çµæ - çæããããªãŒãã£ãªã®åçãšç®¡ç
ãµãŒãã¹èšå®
ã¢ãã«éžæ
| èšå® | 説æ |
|---|---|
| ãã§ãã¯ãã€ã³ããã¡ã€ã« | ãã¬ãŒãã³ã°æžã¿ã¢ãã«ãã§ãã¯ãã€ã³ããéžæïŒå©çšå¯èœãªå ŽåïŒ |
| ã¡ã€ã³ã¢ãã«ãã¹ | DiTã¢ãã«èšå®ãéžæïŒäŸïŒacestep-v15-turboãacestep-v15-turbo-shift3ïŒ |
| ããã€ã¹ | åŠçããã€ã¹ïŒautoïŒæšå¥šïŒãcudaããŸã㯠cpu |
5Hz LMèšå®
| èšå® | 説æ |
|---|---|
| 5Hz LMã¢ãã«ãã¹ | èšèªã¢ãã«ãéžæïŒäŸïŒacestep-5Hz-lm-0.6Bãacestep-5Hz-lm-1.7BïŒ |
| 5Hz LMããã¯ãšã³ã | vllmïŒããé«éãæšå¥šïŒãŸã㯠ptïŒPyTorchãäºææ§ãé«ãïŒ |
| 5Hz LMãåæå | åæåæã«LMãèªã¿èŸŒãããã«ãã§ãã¯ïŒthinkingã¢ãŒãã«å¿ èŠïŒ |
ããã©ãŒãã³ã¹ãªãã·ã§ã³
| èšå® | 説æ |
|---|---|
| Flash Attentionãäœ¿çš | ããé«éãªæšè«ã®ããã«æå¹åïŒflash_attnããã±ãŒãžãå¿ èŠïŒ |
| CPUã«ãªãããŒã | ã¢ã€ãã«æã«ã¢ãã«ãCPUã«ãªãããŒãããŠGPUã¡ã¢ãªãç¯çŽ |
| DiTãCPUã«ãªãããŒã | DiTã¢ãã«ãç¹ã«CPUã«ãªãããŒã |
LoRAã¢ããã¿ãŒ
| èšå® | 説æ |
|---|---|
| LoRAãã¹ | ãã¬ãŒãã³ã°æžã¿LoRAã¢ããã¿ãŒãã£ã¬ã¯ããªãžã®ãã¹ |
| LoRAãèªã¿èŸŒã¿ | æå®ãããLoRAã¢ããã¿ãŒãèªã¿èŸŒã¿ |
| ã¢ã³ããŒã | çŸåšèªã¿èŸŒãŸããŠããLoRAãåé€ |
| LoRAãäœ¿çš | æšè«çšã®èªã¿èŸŒãŸããLoRAãæå¹å/ç¡å¹å |
åæå
ãµãŒãã¹ãåæå ãã¯ãªãã¯ããŠã¢ãã«ãèªã¿èŸŒã¿ãŸããã¹ããŒã¿ã¹ããã¯ã¹ã«é²æãšç¢ºèªã衚瀺ãããŸãã
çæã¢ãŒã
ã·ã³ãã«ã¢ãŒã
ã·ã³ãã«ã¢ãŒãã¯ãè¿ éãªèªç¶èšèªããŒã¹ã®é³æ¥œçæçšã«èšèšãããŠããŸãã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã©ãžãªãã¿ã³ã§ãã·ã³ãã«ããéžæ
- ãæ²ã®èª¬æããã£ãŒã«ãã«èªç¶èšèªã®èª¬æãå ¥å
- ããŒã«ã«ãäžèŠãªå Žåã¯ãã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãããªãã·ã§ã³ã§ãã§ãã¯
- ãªãã·ã§ã³ã§åžæããããŒã«ã«èšèªãéžæ
- ãµã³ãã«ãäœæ ãã¯ãªãã¯ããŠcaptionãæè©ãã¡ã¿ããŒã¿ãçæ
- å±éãããã»ã¯ã·ã§ã³ã§çæãããã³ã³ãã³ãã確èª
- 鳿¥œãçæ ãã¯ãªãã¯ããŠãªãŒãã£ãªãäœæ
説æã®äŸïŒ
- ãéããªå€ã®ããã®æããããã³ã¬ã«ã®ã©ããœã³ã°ã
- ãéãããŒã¹ããããã®ã¢ããããŒããªãšã¬ã¯ããããã¯ãã³ã¹ãã¥ãŒãžãã¯ã
- ãã¢ã³ãŒã¹ãã£ãã¯ã®ã¿ãŒã®ã¡ã©ã³ã³ãªãã¯ãªã€ã³ãã£ãŒãã©ãŒã¯ã
- ãç ããããŒã§æŒå¥ãããžã£ãºããªãªã
ã©ã³ãã ãµã³ãã«ïŒ ð² ãã¿ã³ãã¯ãªãã¯ããŠã©ã³ãã ãªäŸã®èª¬æãèªã¿èŸŒã¿ãŸãã
ã«ã¹ã¿ã ã¢ãŒã
ã«ã¹ã¿ã ã¢ãŒãã¯ãã¹ãŠã®çæãã©ã¡ãŒã¿ã®å®å šãªå¶åŸ¡ãæäŸããŸãã
äœ¿çšæ¹æ³ïŒ
- çæã¢ãŒãã©ãžãªãã¿ã³ã§ãã«ã¹ã¿ã ããéžæ
- Captionãšæè©ãã£ãŒã«ããæåã§å ¥å
- ãªãã·ã§ã³ã®ã¡ã¿ããŒã¿ãèšå®ïŒBPMãããŒãDurationãªã©ïŒ
- ãªãã·ã§ã³ã§ ãã©ãŒããã ãã¯ãªãã¯ããŠLMã䜿çšããŠå ¥åã匷å
- å¿ èŠã«å¿ããŠé«åºŠãªèšå®ãæ§æ
- 鳿¥œãçæ ãã¯ãªãã¯ããŠãªãŒãã£ãªãäœæ
ã¿ã¹ã¯ã¿ã€ã
text2musicïŒããã©ã«ãïŒ
ããã¹ã説æããã³/ãŸãã¯æè©ãã鳿¥œãçæã
ãŠãŒã¹ã±ãŒã¹ïŒ ããã³ããã«åºã¥ããŠæ°ãã鳿¥œããŒãããäœæã
å¿ é å ¥åïŒ CaptionãŸãã¯æè©ïŒå°ãªããšã1ã€ïŒ
cover
æ¢åã®ãªãŒãã£ãªãæ§é ãç¶æããªããã¹ã¿ã€ã«ã倿ŽããŠå€æã
ãŠãŒã¹ã±ãŒã¹ïŒ ç°ãªãã¹ã¿ã€ã«ã®ã«ããŒããŒãžã§ã³ãäœæã
å¿ é å ¥åïŒ
- ãœãŒã¹ãªãŒãã£ãªïŒãªãŒãã£ãªã¢ããããŒãã»ã¯ã·ã§ã³ã§ã¢ããããŒãïŒ
- ã¿ãŒã²ããã¹ã¿ã€ã«ã説æããCaption
éèŠãªãã©ã¡ãŒã¿ïŒ ãªãŒãã£ãªã«ããŒåŒ·åºŠïŒ0.0-1.0ïŒ
- é«ãå€ã¯å ã®æ§é ãããå€ãç¶æ
- äœãå€ã¯ããåµé çãªèªç±ãèš±å¯
repaint
ãªãŒãã£ãªã®ç¹å®ã®æéã»ã°ã¡ã³ããåçæã
ãŠãŒã¹ã±ãŒã¹ïŒ çæããã鳿¥œã®ç¹å®ã®ã»ã¯ã·ã§ã³ãä¿®æ£ãŸãã¯å€æŽã
å¿ é å ¥åïŒ
- ãœãŒã¹ãªãŒãã£ãª
- ãªãã€ã³ãéå§ïŒç§ïŒ
- ãªãã€ã³ãçµäºïŒç§ããã¡ã€ã«çµç«¯ã«ã¯-1ïŒ
- åžæããã³ã³ãã³ãã説æããCaption
legoïŒBaseã¢ãã«ã®ã¿ïŒ
æ¢åã®ãªãŒãã£ãªã®ã³ã³ããã¹ãã§ç¹å®ã®æ¥œåšãã©ãã¯ãçæã
ãŠãŒã¹ã±ãŒã¹ïŒ ãããã³ã°ãã©ãã¯ã«æ¥œåšã¬ã€ã€ãŒã远å ã
å¿ é å ¥åïŒ
- ãœãŒã¹ãªãŒãã£ãª
- ãã©ãã¯åïŒããããããŠã³ããéžæïŒ
- ãã©ãã¯ç¹æ§ã説æããCaption
å©çšå¯èœãªãã©ãã¯ïŒ vocalsãbacking_vocalsãdrumsãbassãguitarãkeyboardãpercussionãstringsãsynthãfxãbrassãwoodwinds
extractïŒBaseã¢ãã«ã®ã¿ïŒ
ããã¯ã¹ãªãŒãã£ãªããç¹å®ã®æ¥œåšãã©ãã¯ãæœåº/åé¢ã
ãŠãŒã¹ã±ãŒã¹ïŒ ã¹ãã åé¢ã楜åšã®åé¢ã
å¿ é å ¥åïŒ
- ãœãŒã¹ãªãŒãã£ãª
- æœåºãããã©ãã¯å
completeïŒBaseã¢ãã«ã®ã¿ïŒ
æå®ãããæ¥œåšã§éšåçãªãã©ãã¯ã宿ã
ãŠãŒã¹ã±ãŒã¹ïŒ äžå®å šãªäœåã®èªåã¢ã¬ã³ãžã
å¿ é å ¥åïŒ
- ãœãŒã¹ãªãŒãã£ãª
- ãã©ãã¯åïŒè€æ°éžæïŒ
- åžæããã¹ã¿ã€ã«ã説æããCaption
å ¥åãã©ã¡ãŒã¿
å¿ é å ¥å
ã¿ã¹ã¯ã¿ã€ã
ããããããŠã³ããçæã¿ã¹ã¯ãéžæãéžæãããã¿ã¹ã¯ã«åºã¥ããŠæç€ºãã£ãŒã«ããèªåçã«æŽæ°ãããŸãã
ãªãŒãã£ãªã¢ããããŒã
| ãã£ãŒã«ã | 説æ |
|---|---|
| åç §ãªãŒãã£ãª | ã¹ã¿ã€ã«åç §çšã®ãªãã·ã§ã³ãªãŒãã£ãª |
| ãœãŒã¹ãªãŒãã£ãª | coverãrepaintãlegoãextractãcompleteã¿ã¹ã¯ã«å¿ é |
| ã³ãŒãã«å€æ | ãœãŒã¹ãªãŒãã£ãªãã5Hzã»ãã³ãã£ãã¯ã³ãŒããæœåº |
LMã³ãŒããã³ã
äºåèšç®ããããªãŒãã£ãªã»ãã³ãã£ãã¯ã³ãŒããããã«è²Œãä»ããŠçæãã¬ã€ãã§ããŸãããã©ã³ã¹ã¯ã©ã€ã ãã¿ã³ã䜿çšããŠã³ãŒããåæãã¡ã¿ããŒã¿ãæœåºããŸãã
鳿¥œãã£ãã·ã§ã³
åžæãã鳿¥œã®ããã¹ã説æã以äžã«ã€ããŠå ·äœçã«ïŒ
- ãžã£ã³ã«ãšã¹ã¿ã€ã«
- 楜åš
- ã ãŒããšé°å²æ°
- ãã³ãæïŒBPMãæå®ããªãå ŽåïŒ
äŸïŒ ããšã¬ãã®ã¿ãŒãå匷ããã©ã ããã£ãããŒãªã·ã³ã»ããã¯ã®ã¢ããããŒããªãããããã¯ã
ð² ãã¯ãªãã¯ããŠã©ã³ãã ãªäŸã®captionãèªã¿èŸŒã¿ãŸãã
æè©
æ§é ã¿ã°ä»ãã®æè©ãå ¥åïŒ
[Verse 1]
仿¥è¡ãæ©ããŠããŠ
åãèšã£ãŠããèšèãæãåºããŠãã
[Chorus]
åã«é²ãã§ããã匷ããã
ãããåã®å±
å Žæ
[Verse 2]
...
ã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãã§ãã¯ããã¯ã¹ïŒ ããããã§ãã¯ãããšãæè©ã®å 容ã«é¢ä¿ãªãã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«é³æ¥œãçæããŸãã
ããŒã«ã«èšèªïŒ ããŒã«ã«ã®èšèªãéžæãèªåæ€åºãŸãã¯ã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãã©ãã¯ã«ã¯ãunknownãã䜿çšã
ãã©ãŒããããã¿ã³ïŒ ã¯ãªãã¯ããŠ5Hz LMã䜿çšããŠcaptionãšæè©ã匷åã
ãªãã·ã§ã³ãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| BPM | èªå | 1åãããã®ããŒãæ°ïŒ30-300ïŒ |
| ããŒã¹ã±ãŒã« | èªå | 鳿¥œããŒïŒäŸïŒãC MajorãããAmãããF# minorãïŒ |
| æåèšå· | èªå | æåèšå·ïŒ2ïŒ2/4ïŒã3ïŒ3/4ïŒã4ïŒ4/4ïŒã6ïŒ6/8ïŒ |
| ãªãŒãã£ãªé· | èªå/-1 | ç®æšé·ïŒç§ïŒïŒ10-600ïŒã-1ã§èªå |
| ããããµã€ãº | 2 | çæãããªãŒãã£ãªããªãšãŒã·ã§ã³ã®æ°ïŒ1-8ïŒ |
é«åºŠãªèšå®
DiTãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| æšè«ã¹ããã | 8 | ããã€ãºã¹ããããTurboïŒ1-20ãBaseïŒ1-200 |
| ã¬ã€ãã³ã¹ã¹ã±ãŒã« | 7.0 | CFG匷床ïŒbaseã¢ãã«ã®ã¿ïŒãé«ã = ããã³ããã«ããåŸã |
| ã·ãŒã | -1 | ã©ã³ãã ã·ãŒãããããã«ã¯ã«ã³ãåºåãã®å€ãäœ¿çš |
| ã©ã³ãã ã·ãŒã | â | ãã§ãã¯æã«ã©ã³ãã ã·ãŒããçæ |
| ãªãŒãã£ãªåœ¢åŒ | mp3 | åºå圢åŒïŒmp3ãflac |
| ã·ãã | 3.0 | ã¿ã€ã ã¹ãããã·ããä¿æ°ïŒ1.0-5.0ïŒãturboã«ã¯3.0æšå¥š |
| æšè«æ¹æ³ | ode | odeïŒEulerãããé«éïŒãŸãã¯sdeïŒç¢ºççïŒ |
| ã«ã¹ã¿ã ã¿ã€ã ã¹ããã | - | ã¿ã€ã ã¹ãããããªãŒããŒã©ã€ãïŒäŸïŒã0.97,0.76,0.615,0.5,0.395,0.28,0.18,0.085,0ãïŒ |
Baseã¢ãã«ã®ã¿ã®ãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| ADGãäœ¿çš | â | ããè¯ãå質ã®ããã«é©å¿ãã¥ã¢ã«ã¬ã€ãã³ã¹ãæå¹å |
| CFGåºééå§ | 0.0 | CFGãé©çšãå§ããã¿ã€ãã³ã°ïŒ0.0-1.0ïŒ |
| CFGåºéçµäº | 1.0 | CFGã®é©çšã忢ããã¿ã€ãã³ã°ïŒ0.0-1.0ïŒ |
LMãã©ã¡ãŒã¿
| ãã©ã¡ãŒã¿ | ããã©ã«ã | 説æ |
|---|---|---|
| LM枩床 | 0.85 | ãµã³ããªã³ã°æž©åºŠïŒ0.0-2.0ïŒãé«ã = ããåµé ç |
| LM CFGã¹ã±ãŒã« | 2.0 | LMã¬ã€ãã³ã¹åŒ·åºŠïŒ1.0-3.0ïŒ |
| LM Top-K | 0 | Top-Kãµã³ããªã³ã°ã0ã§ç¡å¹ |
| LM Top-P | 0.9 | æ žãµã³ããªã³ã°ïŒ0.0-1.0ïŒ |
| LMãã¬ãã£ãããã³ãã | "NO USER INPUT" | CFGçšã®ãã¬ãã£ãããã³ãã |
CoTïŒæèã®é£éïŒãªãã·ã§ã³
| ãªãã·ã§ã³ | ããã©ã«ã | 説æ |
|---|---|---|
| CoT Metas | â | LMæšè«ã§ã¡ã¿ããŒã¿ãçæ |
| CoT Language | â | LMã§ããŒã«ã«èšèªãæ€åº |
| å¶çŽä»ããã³ãŒãã£ã³ã°ãããã° | â | ãããã°ãã°ãæå¹å |
çæãªãã·ã§ã³
| ãªãã·ã§ã³ | ããã©ã«ã | 説æ |
|---|---|---|
| LMã³ãŒã匷床 | 1.0 | LMã³ãŒããçæã«äžãã圱é¿ã®åŒ·ãïŒ0.0-1.0ïŒ |
| èªåã¹ã³ã¢ | â | å質ã¹ã³ã¢ãèªåèšç® |
| èªåLRC | â | æè©ã¿ã€ã ã¹ã¿ã³ããèªåçæ |
| LMããããã£ã³ã¯ãµã€ãº | 8 | LMããããããã®æå€§ã¢ã€ãã æ°ïŒGPUã¡ã¢ãªïŒ |
ã¡ã€ã³çæã³ã³ãããŒã«
| ã³ã³ãããŒã« | 説æ |
|---|---|
| Think | ã³ãŒãçæãšã¡ã¿ããŒã¿çšã®5Hz LMãæå¹å |
| ParallelThinking | 䞊åLMãããåŠçãæå¹å |
| CaptionRewrite | LMã«å ¥åcaptionã匷åããã |
| AutoGen | å®äºåŸã«æ¬¡ã®ããããèªåéå§ |
çµæã»ã¯ã·ã§ã³
çæããããªãŒãã£ãª
ããããµã€ãºã«åºã¥ããŠæå€§8ã€ã®ãªãŒãã£ãªãµã³ãã«ã衚瀺ãããŸããåãµã³ãã«ã«ã¯ä»¥äžãå«ãŸããŸãïŒ
- ãªãŒãã£ãªãã¬ãŒã€ãŒ - çæããããªãŒãã£ãªã®åçãäžæåæ¢ãããŠã³ããŒã
- ãœãŒã¹ã«éä¿¡ - ãã®ãªãŒãã£ãªããœãŒã¹ãªãŒãã£ãªå ¥åã«éä¿¡ããŠããã«åŠç
- ä¿å - ãªãŒãã£ãªãšã¡ã¿ããŒã¿ãJSONãã¡ã€ã«ã«ä¿å
- ã¹ã³ã¢ - ããŒãã¬ãã·ãã£ããŒã¹ã®å質ã¹ã³ã¢ãèšç®
- LRC - æè©ã¿ã€ã ã¹ã¿ã³ããçæïŒLRC圢åŒïŒ
詳现ã¢ã³ãŒãã£ãªã³
ãã¹ã³ã¢ & LRC & LMã³ãŒãããã¯ãªãã¯ããŠå±éãã以äžã衚瀺ïŒ
- LMã³ãŒã - ãã®ãµã³ãã«ã®5Hzã»ãã³ãã£ãã¯ã³ãŒã
- å質ã¹ã³ã¢ - ããŒãã¬ãã·ãã£ããŒã¹ã®å質ã¡ããªãã¯
- æè©ã¿ã€ã ã¹ã¿ã³ã - LRC圢åŒã®ã¿ã€ãã³ã°ããŒã¿
ãããããã²ãŒã·ã§ã³
| ã³ã³ãããŒã« | 説æ |
|---|---|
| â åãž | åã®ãããã衚瀺 |
| ãããã€ã³ãžã±ãŒã¿ãŒ | çŸåšã®ãããäœçœ®ã衚瀺ïŒäŸïŒãããã 1 / 3ãïŒ |
| 次ãããã¹ããŒã¿ã¹ | ããã¯ã°ã©ãŠã³ãçæã®é²æã衚瀺 |
| 次㞠ⶠ| 次ã®ãããã衚瀺ïŒAutoGenããªã³ã®å Žåã¯çæãããªã¬ãŒïŒ |
ãã©ã¡ãŒã¿ã®åŸ©å
ãããã®èšå®ãUIã«é©çš ãã¯ãªãã¯ããŠãçŸåšã®ããããããã¹ãŠã®çæãã©ã¡ãŒã¿ãå ¥åãã£ãŒã«ãã«åŸ©å ãè¯ãçµæãå埩ããã®ã«äŸ¿å©ã
ãããçµæ
ããããçµæãšçæè©³çްãã¢ã³ãŒãã£ãªã³ã«ã¯ä»¥äžãå«ãŸããŸãïŒ
- ãã¹ãŠã®çæãã¡ã€ã« - ãã¹ãŠã®ããããããã¹ãŠã®ãã¡ã€ã«ãããŠã³ããŒã
- çæè©³çް - çæããã»ã¹ã«é¢ãã詳现æ å ±
LoRAãã¬ãŒãã³ã°
LoRAãã¬ãŒãã³ã°ã¿ãã¯ã«ã¹ã¿ã LoRAã¢ããã¿ãŒãäœæããããã®ããŒã«ãæäŸããŸãã
ããŒã¿ã»ãããã«ããŒã¿ã
ã¹ããã1ïŒèªã¿èŸŒã¿ãŸãã¯ã¹ãã£ã³
ãªãã·ã§ã³AïŒæ¢åã®ããŒã¿ã»ãããèªã¿èŸŒã¿
- 以åä¿åããããŒã¿ã»ããJSONãžã®ãã¹ãå ¥å
- èªã¿èŸŒã¿ ãã¯ãªãã¯
ãªãã·ã§ã³BïŒæ°ãããã£ã¬ã¯ããªãã¹ãã£ã³
- ãªãŒãã£ãªãã©ã«ããžã®ãã¹ãå ¥å
- ã¹ãã£ã³ ãã¯ãªãã¯ããŠãªãŒãã£ãªãã¡ã€ã«ãæ€çŽ¢ïŒwavãmp3ãflacãoggãopusïŒ
ã¹ããã2ïŒããŒã¿ã»ããã®èšå®
| èšå® | 説æ |
|---|---|
| ããŒã¿ã»ããå | ããŒã¿ã»ããã®åå |
| ãã¹ãŠã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã« | ãã¹ãŠã®ãã©ãã¯ã«ããŒã«ã«ããªãå Žåã«ãã§ã㯠|
| ã«ã¹ã¿ã ã¢ã¯ãã£ããŒã·ã§ã³ã¿ã° | ãã®LoRAã®ã¹ã¿ã€ã«ãã¢ã¯ãã£ãã«ãããŠããŒã¯ãªã¿ã° |
| ã¿ã°äœçœ® | ã¿ã°ãé 眮ããå ŽæïŒåã«è¿œå ãåŸã«è¿œå ããŸãã¯captionã眮æ |
ã¹ããã3ïŒèªåã©ãã«
ãã¹ãŠèªåã©ãã« ãã¯ãªãã¯ããŠãã¹ãŠã®ãªãŒãã£ãªãã¡ã€ã«ã®ã¡ã¿ããŒã¿ãçæïŒ
- CaptionïŒé³æ¥œã®èª¬æïŒ
- BPM
- ããŒ
- æåèšå·
Metasãã¹ããã ãªãã·ã§ã³ã¯LLMã©ããªã³ã°ãã¹ãããããŠN/Aå€ã䜿çšããŸãã
ã¹ããã4ïŒãã¬ãã¥ãŒãšç·šé
ã¹ã©ã€ããŒã䜿çšããŠãµã³ãã«ãéžæããæåã§ç·šéïŒ
- Caption
- æè©
- BPMãããŒãæåèšå·
- èšèª
- ã€ã³ã¹ãã¥ã«ã¡ã³ã¿ã«ãã©ã°
倿Žãä¿å ãã¯ãªãã¯ããŠãµã³ãã«ãæŽæ°ã
ã¹ããã5ïŒããŒã¿ã»ãããä¿å
ä¿åãã¹ãå ¥åããããŒã¿ã»ãããä¿å ãã¯ãªãã¯ããŠJSONãšããŠãšã¯ã¹ããŒãã
ã¹ããã6ïŒååŠç
é«éãã¬ãŒãã³ã°ã®ããã«ããŒã¿ã»ãããäºåèšç®ãã³ãœã«ã«å€æïŒ
- ãªãã·ã§ã³ã§æ¢åã®ããŒã¿ã»ããJSONãèªã¿èŸŒã¿
- ãã³ãœã«åºåãã£ã¬ã¯ããªãèšå®
- ååŠç ãã¯ãªãã¯
ããã«ãããªãŒãã£ãªãVAEæœåšå€æ°ã«ãšã³ã³ãŒããããããã¹ããåã蟌ã¿ã«ãšã³ã³ãŒããããæ¡ä»¶ãšã³ã³ãŒããŒãå®è¡ãããŸãã
LoRAãã¬ãŒãã³ã°ã¿ã
ããŒã¿ã»ããéžæ
ååŠçããããã³ãœã«ãã£ã¬ã¯ããªãžã®ãã¹ãå ¥åããããŒã¿ã»ãããèªã¿èŸŒã¿ ãã¯ãªãã¯ã
LoRAèšå®
| èšå® | ããã©ã«ã | 説æ |
|---|---|---|
| LoRAã©ã³ã¯ (r) | 64 | LoRAã®å®¹éãé«ã = ããå€ãã®å®¹éãããå€ãã®ã¡ã¢ãª |
| LoRA Alpha | 128 | ã¹ã±ãŒãªã³ã°ä¿æ°ïŒéåžžã¯ã©ã³ã¯ã®2åïŒ |
| LoRA Dropout | 0.1 | æ£ååã®ããã®ããããã¢ãŠãç |
ãã¬ãŒãã³ã°ãã©ã¡ãŒã¿
| èšå® | ããã©ã«ã | 説æ |
|---|---|---|
| åŠç¿ç | 1e-4 | æé©ååŠç¿ç |
| æå€§ãšãã㯠| 500 | æå€§ãã¬ãŒãã³ã°ãšãã㯠|
| ããããµã€ãº | 1 | ãã¬ãŒãã³ã°ããããµã€ãº |
| åŸé çŽ¯ç© | 1 | æå¹ããã = batch_size à accumulation |
| Nãšããã¯ããšã«ä¿å | 200 | ãã§ãã¯ãã€ã³ãä¿åé »åºŠ |
| ã·ãã | 3.0 | turboã¢ãã«ã®ã¿ã€ã ã¹ãããã·ãã |
| ã·ãŒã | 42 | åçŸæ§ã®ããã®ã©ã³ãã ã·ãŒã |
ãã¬ãŒãã³ã°ã³ã³ãããŒã«
- ãã¬ãŒãã³ã°éå§ - ãã¬ãŒãã³ã°ããã»ã¹ãéå§
- ãã¬ãŒãã³ã°åæ¢ - ãã¬ãŒãã³ã°ãäžæ
- ãã¬ãŒãã³ã°é²æ - çŸåšã®ãšããã¯ãšãã¹ã衚瀺
- ãã¬ãŒãã³ã°ãã° - 詳现ãªãã¬ãŒãã³ã°åºå
- ãã¬ãŒãã³ã°ãã¹ãããã - èŠèŠçãªãã¹æ²ç·
LoRAã®ãšã¯ã¹ããŒã
ãã¬ãŒãã³ã°åŸãæçµã¢ããã¿ãŒããšã¯ã¹ããŒãïŒ
- ãšã¯ã¹ããŒããã¹ãå ¥å
- LoRAããšã¯ã¹ããŒã ãã¯ãªãã¯
ãã³ããšãã¹ããã©ã¯ãã£ã¹
æé«å質ã®ããã«
- thinkingã¢ãŒããäœ¿çš - LM匷åçæã®ããã«ãThinkããã§ãã¯ããã¯ã¹ãæå¹ã«ä¿ã€
- captionãå ·äœçã« - ãžã£ã³ã«ã楜åšãã ãŒããã¹ã¿ã€ã«ã®è©³çްãå«ãã
- LMã«ã¡ã¿ããŒã¿ãæ€åºããã - èªåæ€åºã®ããã«BPM/ããŒ/Durationã空ã®ãŸãŸã«ãã
- ãããçæãäœ¿çš - 2-4ã®ããªãšãŒã·ã§ã³ãçæããæè¯ã®ãã®ãéžã¶
ããé«éãªçæã®ããã«
- turboã¢ãã«ãäœ¿çš -
acestep-v15-turboãŸãã¯acestep-v15-turbo-shift3ãéžæ - æšè«ã¹ãããã8ã«ä¿ã€ - turboã«æé©ãªããã©ã«ã
- ããããµã€ãºãæžãã - è¿ éãªçµæãå¿ èŠãªå Žåã¯ããããµã€ãºãäžãã
- AutoGenãç¡å¹å - ãããçæã®æåå¶åŸ¡
äžè²«ããçµæã®ããã«
- ç¹å®ã®ã·ãŒããèšå® - ãã©ã³ãã ã·ãŒããã®ãã§ãã¯ãå€ããŠã·ãŒãå€ãå ¥å
- è¯ãçµæãä¿å - åçŸã®ããã«ãã©ã¡ãŒã¿ããšã¯ã¹ããŒãããããã«ãä¿åãã䜿çš
- ããããã®èšå®ãé©çšããäœ¿çš - è¯ãããããããã©ã¡ãŒã¿ã埩å
é·å°ºé³æ¥œã®ããã«
- æç€ºçãªdurationãèšå® - ç§åäœã§durationãæå®
- repaintã¿ã¹ã¯ãäœ¿çš - åæçæåŸã«åé¡ã®ããã»ã¯ã·ã§ã³ãä¿®æ£
- çæããã§ãŒã³ - 以åã®çµæã®äžã«æ§ç¯ããããã«ããœãŒã¹ã«éä¿¡ãã䜿çš
ã¹ã¿ã€ã«ã®äžè²«æ§ã®ããã«
- LoRAããã¬ãŒãã³ã° - ããªãã®ã¹ã¿ã€ã«çšã®ã«ã¹ã¿ã ã¢ããã¿ãŒãäœæ
- åç §ãªãŒãã£ãªãäœ¿çš - ãªãŒãã£ãªã¢ããããŒãã§ã¹ã¿ã€ã«åç §ãã¢ããããŒã
- äžè²«ããcaptionãäœ¿çš - é¡äŒŒã®èª¬æçãªèšèªãç¶æ
ãã©ãã«ã·ã¥ãŒãã£ã³ã°
ãªãŒãã£ãªãçæãããªãïŒ
- ã¢ãã«ãåæåãããŠããããšã確èªïŒç·ã®ã¹ããŒã¿ã¹ã¡ãã»ãŒãžïŒ
- thinkingã¢ãŒãã䜿çšããŠããå Žåã¯5Hz LMãåæåãããŠããããšã確èª
- ãšã©ãŒã¡ãã»ãŒãžã®ã¹ããŒã¿ã¹åºåã確èª
çµæã®åè³ªãæªãïŒ
- æšè«ã¹ããããå¢ããïŒbaseã¢ãã«ã®å ŽåïŒ
- ã¬ã€ãã³ã¹ã¹ã±ãŒã«ã調æŽ
- ç°ãªãã·ãŒãã詊ã
- captionãããå ·äœçã«ãã
ã¡ã¢ãªäžè¶³ïŒ
- ããããµã€ãºãæžãã
- CPUãªãããŒããæå¹å
- LMããããã£ã³ã¯ãµã€ãºãæžãã
LMãæ©èœããªãïŒ
- åæåæã«ã5Hz LMãåæåãããã§ãã¯ãããŠããããšã確èª
- æå¹ãªLMã¢ãã«ãã¹ãéžæãããŠããããšã確èª
- vllmãŸãã¯PyTorchããã¯ãšã³ããå©çšå¯èœã§ããããšã確èª
ããŒããŒãã·ã§ãŒãã«ãã
Gradioã€ã³ã¿ãŒãã§ãŒã¹ã¯æšæºçãªWebã·ã§ãŒãã«ããããµããŒãïŒ
- Tab - å ¥åãã£ãŒã«ãéãç§»å
- Enter - ããã¹ãå ¥åãéä¿¡
- Space - ãã§ãã¯ããã¯ã¹ãåãæ¿ã
èšèªãµããŒã
ã€ã³ã¿ãŒãã§ãŒã¹ã¯è€æ°ã®UIèšèªããµããŒãïŒ
- è±èª (en)
- äžåœèª (zh)
- æ¥æ¬èª (ja)
ãµãŒãã¹èšå®ã»ã¯ã·ã§ã³ã§å¥œã¿ã®èšèªãéžæããŠãã ããã
詳现ã«ã€ããŠã¯ä»¥äžãåç §ïŒ
- ã¡ã€ã³READMEïŒ
../../README.md - REST APIããã¥ã¡ã³ãïŒ
API.md - Pythonæšè«APIïŒ
INFERENCE.md