Create README.md

README.md

@@ -0,0 +1,27 @@
+# Model for testing RM scripts
+This model is just GPT2 base (~100M param) with a value head appended, untrained.
+Use this for debugging RLHF setups (could make a smaller one too).
+The predictions should be somewhat random.
+
+Load the model as follows:
+```
+from transformers import AutoModelForSequenceClassification
+rm = AutoModelForSequenceClassification.from_pretrained("natolambert/gpt2-dummy-rm")
+```
+or as a pipeline
+```
+from Transformers import pipeline
+reward_pipe = pipeline(
+        "text-classification",
+        model="natolambert/gpt2-dummy-rm",
+        # revision=args.model_revision,
+        # model_kwargs={"load_in_8bit": True, "device_map": {"": current_device}, "torch_dtype": torch.float16},
+    )
+reward_pipeline_kwargs = {}
+pipe_outputs = reward_pipe(texts, **reward_pipeline_kwargs)
+```
+
+
+
+
+