Upload merged FP8_DYNAMIC checkpoint + model card

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,156 @@
+---
+base_model: google/gemma-3-27b-it
+library_name: vllm
+pipeline_tag: text-generation
+tags:
+  - gemma
+  - gemma3
+  - text-rewrite
+  - fp8
+  - quantized
+  - compressed-tensors
+  - vllm
+license: gemma
+---
+
+# Entropy v1 FP8 (Gemma 3 27B IT)
+
+Entropy v1 FP8 is a **merged + FP8-quantized** checkpoint based on `google/gemma-3-27b-it`, fine-tuned to rewrite AI-polished text into more human-sounding prose while preserving meaning.
+
+This repo is intended for **efficient inference in vLLM** without runtime LoRA.
+
+## What It Does
+
+Given an AI-sounding passage, the model rewrites it to be:
+
+- More human and textured (less generic "professional polish")
+- More varied in rhythm/word choice
+- Meaning-preserving (style change, not content change)
+
+## Prompt Trigger (Recommended)
+
+This is the pattern used in our fine-tuning data. Keep the passage after a newline.
+
+```text
+Polish this AI passage to feel more human:
+{passage}
+```
+
+Short variants that usually work similarly:
+
+- `Rephrase this AI passage to feel more human:\n{passage}`
+- `Convert this AI passage into a more human-sounding version:\n{passage}`
+
+## How To Run (vLLM)
+
+### 1) Start an OpenAI-compatible server
+
+```bash
+vllm serve ysong21/entropy-v1-fp8 \
+  --served-model-name entropy-v1-fp8 \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --dtype bfloat16 \
+  --max-model-len 8192
+```
+
+Notes:
+
+- This checkpoint is already quantized (compressed-tensors FP8_DYNAMIC). You do not need to pass `--quantization fp8`.
+- FP8 execution is hardware-dependent; see "Quantization" below.
+
+### 2) Send a request
+
+```bash
+curl http://127.0.0.1:8000/v1/chat/completions \
+  -H 'Content-Type: application/json' \
+  -H 'Authorization: Bearer sk-noop' \
+  -d '{
+    "model": "entropy-v1-fp8",
+    "messages": [
+      {
+        "role": "user",
+        "content": "Polish this AI passage to feel more human:\n" \
+                   "This is a highly polished paragraph that sounds generic and overly smooth..."
+      }
+    ],
+    "temperature": 0.7,
+    "top_p": 0.95,
+    "max_tokens": 512
+  }'
+```
+
+## Validation Benchmark (70 Gutenberg Examples)
+
+We evaluate by computing the **conditional negative log-likelihood** of the target (human) rewrite given the prompt, and report **character-normalized bits_per_char**:
+
+- Let `NLL` be the sum of token NLL over the target rewrite (teacher-forced).
+- Let `C` be the number of characters in the target rewrite.
+- `bits_per_char = (NLL / C) / ln(2)`.
+
+This char-normalization makes the score more comparable across models/tokenizers than token-based perplexity.
+
+Lower is better.
+
+### Results
+
+Baseline for relative improvement: `N8Programs/Unslopper-30B-A3B-bf16`.
+
+| System | bits_per_char (↓) | Relative vs Unslopper (↓) |
+|---|---:|---:|
+| **Entropy v1 FP8 (this repo)** | **0.35994** | **+4.07%** |
+| N8Programs/Unslopper-30B-A3B-bf16 | 0.37522 | +0.00% |
+| Base google/gemma-3-27b-it | 0.99565 | -165.35% |
+
+Interpretation:
+
+- Entropy v1 FP8 achieves the best bits_per_char on this 70-example Gutenberg validation study.
+
+## Quantization (Merged FP8_DYNAMIC)
+
+This checkpoint is produced in two steps:
+
+1. **Merge**: a PEFT LoRA adapter is merged into the base Gemma 3 27B IT weights (no runtime LoRA).
+2. **Quantize**: we apply **FP8_DYNAMIC (W8A8)** quantization with `llm-compressor`:
+
+- Targets: all `Linear` layers in the language model
+- Weights: FP8, static per-channel scaling
+- Activations: FP8, dynamic per-token scaling
+- Ignored: `lm_head` and the Gemma 3 vision tower (left in BF16)
+
+The model is saved in a vLLM-loadable **compressed-tensors** format.
+
+Hardware notes (vLLM):
+
+- Hopper/Ada/Blackwell-class NVIDIA GPUs can execute FP8 efficiently.
+- Other GPUs may fall back to less optimized modes.
+
+## Throughput (vLLM)
+
+Measured on a single NVIDIA RTX PRO 6000 Blackwell 96GB using `vllm/vllm-openai:v0.11.2` with random prompts:
+
+- Input length: 512 tokens
+- Output length: 256 tokens
+
+| max_concurrency | output tok/s | total tok/s |
+|---:|---:|---:|
+| 1 | 25.87 | 77.51 |
+| 20 | 412.60 | 1236.20 |
+
+## Limitations / Misuse
+
+- Trained primarily on literary/public-domain style passages; performance may vary on technical/legal writing.
+- Like other "humanizer" models, it can be misused for deceptive purposes. Use responsibly and follow applicable policies and disclosure norms.
+
+## Citation
+
+If you use this model in research, please cite:
+
+```bibtex
+@misc{entropy_v1_fp8,
+  title  = {Entropy v1 FP8 (Gemma 3 27B IT)},
+  author = {ysong21},
+  year   = {2026},
+  note   = {Merged + FP8_DYNAMIC quantized checkpoint for AI-to-human rewriting.}
+}
+```

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

config.json

@@ -0,0 +1,334 @@
+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "dtype": "bfloat16",
+  "eoi_token_index": 256000,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "format": "float-quantized",
+        "input_activations": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": true,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": null,
+          "observer_kwargs": {},
+          "strategy": "token",
+          "symmetric": true,
+          "type": "float"
+        },
+        "output_activations": null,
+        "targets": [
+          "Linear"
+        ],
+        "weights": {
+          "actorder": null,
+          "block_structure": null,
+          "dynamic": false,
+          "group_size": null,
+          "num_bits": 8,
+          "observer": "minmax",
+          "observer_kwargs": {},
+          "strategy": "channel",
+          "symmetric": true,
+          "type": "float"
+        }
+      }
+    },
+    "format": "float-quantized",
+    "global_compression_ratio": null,
+    "ignore": [
+      "model.vision_tower.vision_model.encoder.layers.0.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.0.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.0.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.0.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.0.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.0.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.1.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.1.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.1.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.1.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.1.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.1.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.2.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.2.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.2.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.2.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.2.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.2.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.3.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.3.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.3.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.3.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.3.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.3.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.4.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.4.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.4.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.4.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.4.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.4.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.5.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.5.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.5.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.5.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.5.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.5.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.6.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.6.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.6.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.6.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.6.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.6.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.7.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.7.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.7.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.7.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.7.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.7.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.8.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.8.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.8.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.8.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.8.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.8.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.9.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.9.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.9.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.9.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.9.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.9.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.10.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.10.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.10.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.10.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.10.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.10.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.11.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.11.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.11.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.11.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.11.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.11.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.12.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.12.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.12.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.12.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.12.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.12.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.13.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.13.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.13.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.13.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.13.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.13.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.14.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.14.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.14.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.14.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.14.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.14.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.15.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.15.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.15.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.15.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.15.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.15.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.16.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.16.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.16.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.16.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.16.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.16.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.17.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.17.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.17.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.17.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.17.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.17.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.18.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.18.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.18.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.18.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.18.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.18.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.19.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.19.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.19.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.19.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.19.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.19.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.20.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.20.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.20.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.20.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.20.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.20.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.21.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.21.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.21.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.21.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.21.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.21.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.22.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.22.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.22.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.22.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.22.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.22.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.23.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.23.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.23.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.23.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.23.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.23.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.24.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.24.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.24.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.24.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.24.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.24.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.25.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.25.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.25.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.25.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.25.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.25.mlp.fc2",
+      "model.vision_tower.vision_model.encoder.layers.26.self_attn.k_proj",
+      "model.vision_tower.vision_model.encoder.layers.26.self_attn.v_proj",
+      "model.vision_tower.vision_model.encoder.layers.26.self_attn.q_proj",
+      "model.vision_tower.vision_model.encoder.layers.26.self_attn.out_proj",
+      "model.vision_tower.vision_model.encoder.layers.26.mlp.fc1",
+      "model.vision_tower.vision_model.encoder.layers.26.mlp.fc2",
+      "lm_head"
+    ],
+    "kv_cache_scheme": null,
+    "quant_method": "compressed-tensors",
+    "quantization_status": "compressed",
+    "sparsity_config": {},
+    "transform_config": {},
+    "version": "0.13.0"
+  },
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "dtype": "bfloat16",
+    "final_logit_softcapping": null,
+    "head_dim": 128,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 5376,
+    "initializer_range": 0.02,
+    "intermediate_size": 21504,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 62,
+    "num_key_value_heads": 16,
+    "query_pre_attn_scalar": 168,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "use_bidirectional_attention": false,
+    "use_cache": true,
+    "vocab_size": 262208
+  },
+  "transformers_version": "4.57.3",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "vision_use_head": false
+  }
+}

entropy_manifest.json

@@ -0,0 +1,13 @@
+{
+  "base_model": "google/gemma-3-27b-it",
+  "adapter": "ysong21/entropy-v1-lora",
+  "quantization": "FP8_DYNAMIC",
+  "dtype_pre_quant": "bfloat16",
+  "ignore": [
+    "lm_head",
+    "re:^model\\.vision_tower\\."
+  ],
+  "model_class": "auto",
+  "saved_processor": true,
+  "output_dir": "/mnt/data_nvme/models/entropy-v1-fp8-dynamic-vllm"
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.57.3"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c36349e72bdd1fa4055a8e0d75af600116fb2caf83bdb81e5ea45682baaebc2e
+size 9927234488

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63f6abecf87e134ac10f9df7ee6694570c115960f98921eb44ec95d0ed608a48
+size 9913123656

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:918f6b8481e673a3213003b48a9a58c1b7a31593939d23debad0c685d674c2a8
+size 9433998440

preprocessor_config.json

@@ -0,0 +1,29 @@
+{
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_pan_and_scan": null,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_processor_type": "Gemma3ImageProcessor",
+  "image_seq_length": 256,
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "pan_and_scan_max_num_crops": null,
+  "pan_and_scan_min_crop_size": null,
+  "pan_and_scan_min_ratio_to_activate": null,
+  "processor_class": "Gemma3Processor",
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 896,
+    "width": 896
+  }
+}

processor_config.json

@@ -0,0 +1,4 @@
+{
+  "image_seq_length": 256,
+  "processor_class": "Gemma3Processor"
+}

recipe.yaml

@@ -0,0 +1,6 @@
+default_stage:
+  default_modifiers:
+    QuantizationModifier:
+      targets: [Linear]
+      ignore: [lm_head, 're:^model\.vision_tower\.']
+      scheme: FP8_DYNAMIC

special_tokens_map.json

@@ -0,0 +1,33 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074