Estwld commited on
Commit
4b052d9
·
verified ·
1 Parent(s): 985f650

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +0 -3
README.md CHANGED
@@ -84,7 +84,6 @@ model = AutoModelForCausalLM.from_pretrained(
84
  )
85
  tokenizer = AutoTokenizer.from_pretrained("Estwld/Qwen3.5-9B-base-rednote-200K")
86
 
87
- # Text generation
88
  inputs = tokenizer("Your prompt here", return_tensors="pt").to(model.device)
89
  outputs = model.generate(inputs.input_ids, max_new_tokens=1024)
90
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
@@ -95,8 +94,6 @@ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
95
  - **Inference (bf16)**: ~18 GB VRAM (single GPU)
96
  - **Fine-tuning (full)**: ~80+ GB VRAM per GPU (with full parameters, optimizer states, and activations at 200K context)
97
 
98
- For long-context inference at 200K tokens, ensure sufficient KV-cache memory. Using Flash Attention helps reduce the memory footprint significantly.
99
-
100
  ## Limitations
101
 
102
  - This is a research checkpoint at iteration 309. Further training steps may improve performance.
 
84
  )
85
  tokenizer = AutoTokenizer.from_pretrained("Estwld/Qwen3.5-9B-base-rednote-200K")
86
 
 
87
  inputs = tokenizer("Your prompt here", return_tensors="pt").to(model.device)
88
  outputs = model.generate(inputs.input_ids, max_new_tokens=1024)
89
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 
94
  - **Inference (bf16)**: ~18 GB VRAM (single GPU)
95
  - **Fine-tuning (full)**: ~80+ GB VRAM per GPU (with full parameters, optimizer states, and activations at 200K context)
96
 
 
 
97
  ## Limitations
98
 
99
  - This is a research checkpoint at iteration 309. Further training steps may improve performance.