Update README.md

README.md

@@ -17,32 +17,27 @@ pipeline_tag: image-text-to-text
 library_name: transformers
 ---
 
-# **OxCoder-9B-fp8**
+# **OxCoder-9B-FP8**
 
-> This repository contains an FP8 dynamic-quantized version of [OrionLLM/OxCoder-9B](https://huggingface.co/OrionLLM/OxCoder-9B) (built on `Qwen/Qwen3.5-9B`), optimized for high-throughput inference and agentic coding workflows using **vLLM** and **compressed-tensors**. Quantization was executed using [llm-compressor](https://github.com/vllm-project/llm-compressor) targeting linear projection layers while preserving critical token embeddings, heads, and specialized layers at full precision.
+> This repository contains an FP8 dynamic quantized version of **[OrionLLM/OxCoder-9B](https://huggingface.co/OrionLLM/OxCoder-9B)** (built on `Qwen/Qwen3.5-9B`), optimized for high-throughput inference and reduced VRAM footprint using `llm-compressor` and the `compressed-tensors` format. OxCoder-9B is a compact, frontier-class coding and reasoning model tailored for long-horizon agentic workflows, complex terminal operations, multi-file codebases, and interactive software development.
 
 ## Model Summary
 
-- **Base Model:** [OrionLLM/OxCoder-9B](https://huggingface.co/OrionLLM/OxCoder-9B)
-- **Foundational Architecture:** Qwen/Qwen3.5-9B
-- **Quantization Format:** `compressed-tensors` (FP8_DYNAMIC)
-- **Context Window:** 262,144 tokens (262K native)
-- **Primary Use Cases:** Agentic coding, multi-file codebase reasoning, terminal tool execution, and frontend code generation
+| Attribute | Details |
+|---|---|
+| **Base Model** | [OrionLLM/OxCoder-9B](https://huggingface.co/OrionLLM/OxCoder-9B) (Foundation: `Qwen/Qwen3.5-9B`) |
+| **Quantized Model** | `prithivMLmods/OxCoder-9B-fp8` |
+| **Quantization Method** | [LLM Compressor](https://github.com/vllm-project/llm-compressor) |
+| **Quantization Scheme** | `FP8_DYNAMIC` |
+| **Output Format** | `compressed-tensors` |
+| **Native Context Length** | 262,144 tokens (262K) |
+| **License** | Apache-2.0 |
 
 ## Quantization Details
 
-| Property | Specification |
-| :--- | :--- |
-| **Tool / Framework** | `llmcompressor` |
-| **Format** | `compressed-tensors` |
-| **Quantization Scheme** | `FP8_DYNAMIC` |
-| **Target Layers** | `Linear` |
-| **Calibration Required** | No (`requires_calibration_data: false`) |
-| **Excluded Components** | `lm_head`, `embed_tokens`, `visual`, `linear_attn` |
-
-Linear layers are dynamically scaled per-tensor during runtime. Sensitive components—such as the language model head, input embedding tables, vision modules, and linear attention projections—remain at full precision to retain fidelity in syntax generation and long-context agentic reasoning.
+The model was quantized to FP8 using `llm-compressor` with dynamic per-tensor activation scaling. Sensitive architecture components—including the language model head, input embeddings, vision modules, and linear attention layers—were excluded to preserve fidelity, reasoning stability, and code generation precision.
 
-### Recipe Used
+### Quantization Recipe
 
 ```yaml
 default_stage:
@@ -60,43 +55,59 @@ default_stage:
       requires_calibration_data: false
 ```
 
-## Quickstart & Deployment
+* **Targeted Layers:** All standard `Linear` projections (MLP and attention projections).
+* **Excluded Layers:** `lm_head`, `embed_tokens`, `visual`, `model.visual`, and `linear_attn`.
+* **Calibration Required:** No (utilizes runtime dynamic activation scaling).
 
-### 1. High-Throughput Serving with vLLM
+## Deployment & Inference
 
-The recommended engine for serving `compressed-tensors` FP8 checkpoints is [vLLM](https://github.com/vllm-project/vllm).
+### 1. vLLM (Recommended)
 
-#### Launch OpenAI-Compatible Server
+`compressed-tensors` FP8 checkpoints run natively in [vLLM](https://github.com/vllm-project/vllm):
 
 ```bash
-vllm serve prithivMLmods/OxCoder-9B-fp8 \
-  --max-model-len 262144 \
-  --tensor-parallel-size 1 \
-  --gpu-memory-utilization 0.95
+pip install vllm
+```
+
+Launch an OpenAI-compatible API server:
 
+```bash
+vllm serve prithivMLmods/OxCoder-9B-fp8 \
+    --max-model-len 65536 \
+    --trust-remote-code
 ```
 
-#### Python Inference via vLLM
+Or execute via Python:
 
 ```python
 from vllm import LLM, SamplingParams
 
+sampling_params = SamplingParams(
+    temperature=0.6,
+    top_p=0.95,
+    max_tokens=4096
+)
+
 llm = LLM(
     model="prithivMLmods/OxCoder-9B-fp8",
-    max_model_len=65536,  # Adjust to your GPU VRAM limit
-    trust_remote_code=True
+    trust_remote_code=True,
+    max_model_len=65536
 )
 
-prompt = "Write a React component that manages an editable data grid with undo/redo capabilities."
-sampling_params = SamplingParams(temperature=0.2, max_tokens=2048)
+prompts = [
+    "Write a Python script using asyncio to run a rate-limited web scraper with exponential backoff."
+]
 
-outputs = llm.generate([prompt], sampling_params)
-print(outputs[0].outputs[0].text)
+outputs = llm.generate(prompts, sampling_params)
+for output in outputs:
+    print(output.outputs[0].text)
 ```
 
-### 2. Loading with Hugging Face Transformers
+### 2. Transformers & Compressed Tensors
 
-Ensure you have `transformers` and `accelerate` installed:
+```bash
+pip install transformers compressed-tensors accelerate
+```
 
 ```python
 import torch
@@ -112,12 +123,22 @@ model = AutoModelForCausalLM.from_pretrained(
     trust_remote_code=True
 )
 
-inputs = tokenizer("def parse_git_diff(patch_str: str) -> list[dict]:", return_tensors="pt").to(model.device)
-outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2)
+prompt = "Implement a LRU cache with O(1) runtime for get and put operations in Python."
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+with torch.no_grad():
+    output_tokens = model.generate(
+        **inputs,
+        max_new_tokens=1024,
+        temperature=0.6,
+        top_p=0.95,
+        do_sample=True
+    )
 
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+print(tokenizer.decode(output_tokens[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
 ```
 
-## License
+## Attribution & License
 
-This model card and weights follow the upstream [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
+* **Base Architecture & Weights:** Developed by [OrionLLM](https://huggingface.co/OrionLLM) based on `Qwen/Qwen3.5-9B`.
+* **License:** Released under the [Apache 2.0 License](https://www.google.com/search?q=LICENSE).