Update README.md

README.md

@@ -1,3 +1,71 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- LatitudeGames/Wayfarer-2-12B
+tags:
+- text adventure
+- roleplay
+- nvfp4
+- tensorrt-llm
+model_size: 12B
+datasets:
+- agentlans/distilled-roleplay
+pipeline_tag: text-generation
+---
+![image/jpeg](Wayfarer-2-12B.jpg)
+
+# Wayfarer-2-12B
+
+Quantized NVFP4 weights of the [Wayfarer-2-12B](https://huggingface.co/LatitudeGames/Wayfarer-2-12B) model, for use with nVidia Blackwell GPUs.
+
+## Quantization details
+
+Quantized with TensorRT-Model-Optimizer 0.37.0
+
+Calibrated using the [distilled-roleplay](https://huggingface.co/datasets/agentlans/distilled-roleplay) dataset, tagged in the same ChatML format used to train the Wayfarer and Muse models in the first place. This was accomplished by adding the following code to the start of `hf_ptq.py`: 
+
+```
+from modelopt.torch.utils import dataset_utils
+dataset_utils.SUPPORTED_DATASET_CONFIG["distilled-roleplay"] = {
+    "config": {
+        "path": "agentlans/distilled-roleplay",
+        "split": ["train"],
+    },
+    "preprocess": lambda sample: "".join(
+        f"<|im_start|>{ {'system':'system','human':'user','gpt':'assistant'}[turn['from']] }\n"
+        f"{turn['value'].strip()}<|im_end|>\n"
+        for turn in sample["conversations"]
+    ),
+}
+```
+
+## Inference
+
+Tested on a RTX 5060 Ti 16GB with TensorRT-LLM, vLLM, and SGLang.
+
+Recommended generation settings (a mix of what it says on the Wayfarer-2-12B model card, the default AI Dungeon settings for Wayfarer-2-12B, and the [AI Dungeon Model Guide](https://help.aidungeon.com/ai-models-and-their-differences) entry for the original Wayfarer-12B):
+- Temperature: 1.2
+- Top K: 50
+- Top P: 0.9
+- Min P: 0.025
+- Repetition Penalty: 1.05
+- Presence Penalty: 0.2
+
+## Prompt Format
+
+As mentioned above, the calibration data was provided with the same ChatML tags as had been used to finetune Latitude's 12B models:
+```
+<|im_start|>system
+You're a masterful storyteller and gamemaster. Write in second person present tense (You are), crafting vivid, engaging narratives with authority and confidence.<|im_end|>
+<|im_start|>user
+> You peer into the darkness.<|im_end|>
+<|im_start|>assistant
+You have been eaten by a grue.<|im_end|>
+```
+As such, I would recommend using that format for inference.
+
+## Credits
+
+Wayfarer-2-12B was made by [Latitude Games](https://huggingface.co/LatitudeGames) with help from [Gryphe Padar](https://huggingface.co/Gryphe)