LightningRodLabs
/

Trump-Forecaster

Text Generation

reinforcement-learning

mixture-of-experts

future-as-label

Eval Results (legacy)

Model card Files Files and versions

Bturtel commited on Feb 14

Commit

f061ea6

·

verified ·

1 Parent(s): 7a10263

Upload README.md with huggingface_hub

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -94,7 +94,7 @@ pip install torch transformers safetensors tqdm huggingface-hub
 python merge.py --output ./trump-forecaster-merged
 ```
-This downloads the base model (MXFP4, ~30 GB), dequantizes to bf16, applies the LoRA adapter, and saves the merged model (~300 GB bf16). Requires ~300 GB RAM, no GPU needed.
 ### Inference with the merged model
@@ -108,7 +108,7 @@ engine = sgl.Engine(
     tokenizer_path="openai/gpt-oss-120b",
     trust_remote_code=True,
     dtype="bfloat16",
-    tp_size=2,  # needs 2x 80GB GPUs for bf16
 )
 prompt = """You are a forecasting expert. Given the question and context below, predict the probability that the answer is "Yes".

 python merge.py --output ./trump-forecaster-merged
 ```
+This downloads the base model, dequantizes to bf16, applies the LoRA adapter, and saves the merged model.
 ### Inference with the merged model
     tokenizer_path="openai/gpt-oss-120b",
     trust_remote_code=True,
     dtype="bfloat16",
+    tp_size=2,
 )
 prompt = """You are a forecasting expert. Given the question and context below, predict the probability that the answer is "Yes".