| https://rwkv.com | |
| https://x.com/BlinkDL_AI | |
| Here are some cutting-edge preview RWKV models: https://huggingface.co/BlinkDL/temp-latest-training-models/tree/main | |
| including RWKV-7 checkpts (already supported by https://pypi.org/project/rwkv/ and https://github.com/Ai00-X/ai00_server) | |
| Ref: https://huggingface.co/BlinkDL/rwkv-7-world | |
| "single_round_qa" are "states" (check https://x.com/BlinkDL_AI/status/1788354345807057035) and these are single round Q&A states | |
| data_sample is random subsample of world dataset. note: due to technical reasons (very complicated due to my horrible messy code), some distill instruct data are not included, and only subsamples of these instruct data are included: | |
| flan, Buzz-V12, WebInstructSub, SKGInstruct, PIPPA, COIG-PC-core |