stevelionheart commited on
Commit
bdc1699
·
verified ·
1 Parent(s): b164d43

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +114 -1
README.md CHANGED
@@ -10,4 +10,117 @@ tags:
10
  - Ollama
11
  datasets:
12
  - Tesleum/Fable-5-traces-Harmony
13
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  - Ollama
11
  datasets:
12
  - Tesleum/Fable-5-traces-Harmony
13
+ ---
14
+
15
+
16
+ # Highlights
17
+
18
+ * **Configurable reasoning effort:** Easily adjust the reasoning effort (low, medium, high) based on your specific use case and latency needs.
19
+ * **Full chain-of-thought:** Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
20
+ * **Claude Fable 5:** Fully Train model to Calude Fable 5 specific use case through parameter fine-tuning.
21
+ * **Agentic capabilities:** Use the models’ native capabilities for function calling, [web browsing](https://github.com/openai/gpt-oss/tree/main?tab=readme-ov-file#browser), [Python code execution](https://github.com/openai/gpt-oss/tree/main?tab=readme-ov-file#python), and Structured Outputs.
22
+ * **MXFP4 quantization:** The models were post-trained with MXFP4 quantization of the MoE weights, making model run within 16GB of memory. All evals were performed with the same MXFP4 quantization.
23
+
24
+ ---
25
+
26
+ # Inference examples
27
+
28
+ ## Transformers
29
+
30
+ You can use `Claude-OSS` with Transformers. If you use the Transformers chat template, it will automatically apply the [harmony response format](https://github.com/openai/harmony). If you use `model.generate` directly, you need to apply the harmony format manually using the chat template or use our [openai-harmony](https://github.com/openai/harmony) package.
31
+
32
+ To get started, install the necessary dependencies to setup your environment:
33
+
34
+ ```
35
+ pip install -U transformers kernels torch
36
+ ```
37
+
38
+ Once, setup you can proceed to run the model by running the snippet below:
39
+
40
+ ```py
41
+ from transformers import pipeline
42
+ import torch
43
+
44
+ model_id = "Tesleum/Claude-OSS"
45
+
46
+ pipe = pipeline(
47
+ "text-generation",
48
+ model=model_id,
49
+ torch_dtype="auto",
50
+ device_map="auto",
51
+ )
52
+
53
+ messages = [
54
+ {"role": "user", "content": "Explain quantum mechanics clearly and concisely."},
55
+ ]
56
+
57
+ outputs = pipe(
58
+ messages,
59
+ max_new_tokens=256,
60
+ )
61
+ print(outputs[0]["generated_text"][-1])
62
+ ```
63
+
64
+ Alternatively, you can run the model via [`Transformers Serve`](https://huggingface.co/docs/transformers/main/serving) to spin up a OpenAI-compatible webserver:
65
+
66
+ ```
67
+ transformers serve
68
+ transformers chat localhost:8000 --model-name-or-path Tesleum/Claude-OSS
69
+ ```
70
+
71
+ ## vLLM
72
+
73
+ vLLM recommends using [uv](https://docs.astral.sh/uv/) for Python dependency management. You can use vLLM to spin up an OpenAI-compatible webserver. The following command will automatically download the model and start the server.
74
+
75
+ ```bash
76
+ uv pip install --pre vllm==0.10.1+gptoss \
77
+ --extra-index-url https://wheels.vllm.ai/gpt-oss/ \
78
+ --extra-index-url https://download.pytorch.org/whl/nightly/cu128 \
79
+ --index-strategy unsafe-best-match
80
+
81
+ vllm serve Tesleum/Claude-OSS
82
+ ```
83
+
84
+ ## PyTorch / Triton
85
+
86
+ To learn about how to use this model with PyTorch and Triton, check out our [reference implementations in the gpt-oss repository](https://github.com/openai/gpt-oss?tab=readme-ov-file#reference-pytorch-implementation).
87
+
88
+ ## Ollama
89
+
90
+ If you are trying to run gpt-oss on consumer hardware, you can use Ollama by running the following commands after [installing Ollama](https://ollama.com/download).
91
+
92
+ ```bash
93
+ # Tesleum/claude-oss
94
+ ollama pull Tesleum/claude-oss
95
+ ollama run Tesleum/claude-oss
96
+ ```
97
+
98
+ [Learn more about how to use gpt-oss with Ollama.](https://cookbook.openai.com/articles/gpt-oss/run-locally-ollama)
99
+
100
+ #### LM Studio
101
+
102
+ If you are using [LM Studio](https://lmstudio.ai/) you can use the following commands to download.
103
+
104
+ ```bash
105
+ # claude-oss
106
+ lms get Tesleum/claude-oss
107
+ ```
108
+
109
+ ---
110
+
111
+ # Reasoning levels
112
+
113
+ You can adjust the reasoning level that suits your task across three levels:
114
+
115
+ * **Low:** Fast responses for general dialogue.
116
+ * **Medium:** Balanced speed and detail.
117
+ * **High:** Deep and detailed analysis.
118
+
119
+ The reasoning level can be set in the system prompts, e.g., "Reasoning: high".
120
+
121
+ # Tool use
122
+
123
+ The claude-oss models are excellent for:
124
+ * Agentic operations like browser tasks
125
+ * Function calling with defined schemas
126
+ * Web browsing (using built-in browsing tools)