Ilikemechuri/rwkv7-g1f-7.2b-transformers

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "Ilikemechuri/rwkv7-g1f-7.2b-transformers"

tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)
model.eval()

inputs = tok("User: hello\n\nAssistant:", return_tensors="pt").to("cuda")

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=50,
        do_sample=False,
        stop_strings=["\n\n"],
        tokenizer=tok,
    )

print(tok.decode(out[0].tolist()))

Requires transformers >= 5.6.2 you must use transformers >= 5.6.2

I will update readme this is my plan

1. Model Introduction (Why RWKV-7, O(1))
2. Quick Start (Inference)
3. Chat Template
4. State Manipulation (Key Points for Researchers)
5. Convert from .pth
6. Known Limitations
7. Researcher's Guide (Advanced)
8. BlinkDL Citation

Citation

If you use this model, please cite the original RWKV work: https://github.com/BlinkDL/RWKV-LM

@software{peng_bo_2021_5196578,
  author    = {PENG Bo},
  title     = {BlinkDL/RWKV-LM: 0.01},
  month     = aug,
  year      = 2021,
  publisher = {Zenodo},
  version   = {0.01},
  doi       = {10.5281/zenodo.5196577},
  url       = {https://doi.org/10.5281/zenodo.5196577}
}