11-47 commited on
Commit
85feebf
·
verified ·
1 Parent(s): a4221d0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +99 -9
README.md CHANGED
@@ -1,11 +1,101 @@
 
 
1
  ---
 
 
 
 
 
 
 
 
 
 
2
  library_name: transformers
3
- datasets:
4
- - WithinUsAI/Gemini_3.2_Pro_Distilled
5
- - WithinUsAI/gemini_3.5_flash_distilled_25k
6
- - WithinUsAI/codegemma_gemini_pro_32_distilled_25k
7
- - WithinUsAI/GeminiPro3.2_max_distill_god_seed_25k
8
- - WithinUsAI/DEEPMIND_Alpha_Distilled
9
- base_model:
10
- - google/codegemma-1.1-2b
11
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Here is a complete, professional, and well-structured Hugging Face model card (README.md) designed specifically for WithinUsAI/Gemini3.5-Code.Reasoner-2b-Distilled.
2
+ You can copy and paste the raw Markdown block below directly into your repository’s README.md file.
3
  ---
4
+ license: apache-2.0
5
+ tags:
6
+ - text-generation
7
+ - code
8
+ - reasoning
9
+ - gemma
10
+ - safe-tensors
11
+ - text-generation-inference
12
+ base_model: google/gemma-2-2b
13
+ pipeline_tag: text-generation
14
  library_name: transformers
15
+ language:
16
+ - en
17
+ ---
18
+
19
+ # Gemini3.5-Code.Reasoner-2b-Distilled
20
+
21
+ Gemini3.5-Code.Reasoner-2b-Distilled is a lightweight, high-performance reasoning model designed for code generation, debugging, and complex logical workflows. Built on top of the robust **Gemma 2 2B (`GemmaForCausalLM`)** architecture, this model integrates a specialized low-rank adaptation (LoRA) layer to embed advanced chain-of-thought (CoT) reasoning paths.
22
+
23
+ The model is distilled from larger frontier reasoning pipelines, bringing complex multi-step code planning capabilities into an efficient 2-billion parameter footprint that can easily run locally or on edge devices.
24
+
25
+ ## Model Details
26
+
27
+ - **Developed by:** WithinUsAI
28
+ - **Model Type:** Causal Language Model (Fine-tuned / Distilled)
29
+ - **Base Model:** `google/gemma-2-2b`
30
+ - **Architecture:** GemmaForCausalLM with LoRA Adapters
31
+ - **Language(s):** English (Primary), Code languages (Python, JavaScript, C++, Go, etc.)
32
+ - **License:** Apache 2.0
33
+
34
+ ## Intended Uses & Limitations
35
+
36
+ ### Ideal Use Cases
37
+ - **Local Code Assistance:** Fast autocompletion, code translation, and docstring generation on consumer-grade hardware.
38
+ - **Reasoning-focused Scripting:** Handling multi-step software logic tasks where the model must think step-by-step before writing out a block of code.
39
+ - **Mobile & Edge Deployments:** Thanks to its 2B parameter scale, it is optimal for low-latency setups.
40
+
41
+ ### Limitations
42
+ - **Parameter Constraints:** Being a 2B model, it may struggle with highly architectural software design patterns compared to massive 70B+ or closed-source models.
43
+ - **Hallucinations:** Like all LLMs, it can occasionally generate syntax errors or deprecations. Always test generated code in a secure sandboxed environment.
44
+
45
+ ## Quickstart Guide
46
+
47
+ ### Using Transformers Pipeline
48
+
49
+ ```python
50
+ from transformers import pipeline
51
+
52
+ pipe = pipeline("text-generation", model="WithinUsAI/Gemini3.5-Code.Reasoner-2b-Distilled", device_map="auto")
53
+
54
+ prompt = """<bos>Answer the following coding problem by thinking step-by-step:
55
+ Write a Python function to find the longest palindromic substring in a string.
56
+ """
57
+
58
+ outputs = pipe(prompt, max_new_tokens=512, temperature=0.3)
59
+ print(outputs[0]["generated_text"])
60
+
61
+ Direct Model Load (AutoModelForCausalLM)
62
+ from transformers import AutoTokenizer, AutoModelForCausalLM
63
+ import torch
64
+
65
+ model_id = "WithinUsAI/Gemini3.5-Code.Reasoner-2b-Distilled"
66
+
67
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
68
+ model = AutoModelForCausalLM.from_pretrained(
69
+ model_id,
70
+ torch_dtype=torch.bfloat16,
71
+ device_map="auto"
72
+ )
73
+
74
+ inputs = tokenizer("<bos>Write a quick bash script to parse logs for 404 errors.", return_tensors="pt").to("cuda")
75
+ outputs = model.generate(**inputs, max_new_tokens=256)
76
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
77
+
78
+ High-Performance Deployment via vLLM
79
+ For production setups or fast local servers, you can serve the model using vLLM:
80
+ # Install vLLM from pip
81
+ pip install vllm
82
+
83
+ # Start the OpenAI-compatible vLLM server
84
+ vllm serve "WithinUsAI/Gemini3.5-Code.Reasoner-2b-Distilled"
85
+
86
+ You can then query it via curl:
87
+ curl -X POST "http://localhost:8000/v1/completions" \
88
+ -H "Content-Type: application/json" \
89
+ --data '{
90
+ "model": "WithinUsAI/Gemini3.5-Code.Reasoner-2b-Distilled",
91
+ "prompt": "<bos>Explain the time complexity of QuickSort.",
92
+ "max_tokens": 512,
93
+ "temperature": 0.2
94
+ }'
95
+
96
+ Repository Structure
97
+ The weight distribution includes both the base model layers and structural adapter targets for easy loading:
98
+ * model.safetensors (~5.01 GB) - The primary model weights.
99
+ * adapter_model.safetensors (~49.9 MB) & adapter_config.json - LoRA configuration fine-tuned for code extraction and reasoning properties.
100
+ * tokenizer.json & tokenizer_config.json - Tokenizer structures mapped to the base Gemma profile.
101
+ ---