Add GGUF quantized versions for Ollama/llama.cpp compatibility with API instructions

gguf/README.md

@@ -0,0 +1,75 @@
+# GGUF Models - Qwen3-0.6B Tensor-Slayer Enhanced
+
+This folder contains GGUF (GPT-Generated Unified Format) versions of the Tensor-Slayer enhanced Qwen3-0.6B model, optimized for inference with llama.cpp and Ollama.
+
+## Available Formats
+
+| File | Quantization | Size | Use Case |
+|------|-------------|------|----------|
+| `qwen3-0.6b-tensorslayer-f16.gguf` | FP16 | ~1.2GB | Maximum quality |
+| `qwen3-0.6b-tensorslayer-q8_0.gguf` | 8-bit | ~650MB | High quality, smaller |
+| `qwen3-0.6b-tensorslayer-q5_k_m.gguf` | 5-bit K-quant | ~450MB | Balanced quality/size |
+| `qwen3-0.6b-tensorslayer-q4_0.gguf` | 4-bit | ~350MB | Fastest inference |
+
+## Usage with Ollama
+
+### Download and Setup
+```bash
+# Download recommended Q4_0 version
+wget https://huggingface.co/TheFireHacker/Qwen3-0.6b-TensorSlayerPatch/resolve/main/gguf/qwen3-0.6b-tensorslayer-q4_0.gguf
+
+# Create Ollama model
+ollama create qwen3-enhanced -f - <<EOF
+FROM ./qwen3-0.6b-tensorslayer-q4_0.gguf
+TEMPLATE """<|im_start|>system
+You are a helpful assistant with enhanced semantic understanding.<|im_end|>
+<|im_start|>user
+{{ .Prompt }}<|im_end|>
+<|im_start|>assistant
+"""
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+EOF
+```
+
+### Test Enhanced Semantics
+```bash
+# Test 1: Synonym similarity (should be much better now)
+ollama run qwen3-enhanced "Rate the similarity between 'understanding' and 'comprehension' on a scale of 1-10 and explain why."
+
+# Test 2: Antonym differentiation
+ollama run qwen3-enhanced "Explain the difference between 'surface-level' and 'deep' understanding."
+
+# Test 3: Conceptual relationships
+ollama run qwen3-enhanced "What is the relationship between 'semantic meaning' and 'conceptual understanding'?"
+```
+
+## Ollama API Access
+```python
+import requests
+
+# Using Ollama API (replace with your endpoint)
+OLLAMA_API = "2612b573cd924d148095de291b70bd98.MDGkuS-nd3Ms0a3tBQdpkk-Z"
+response = requests.post(f"{OLLAMA_API}/api/generate", json={
+    "model": "qwen3-enhanced",
+    "prompt": "Understanding and comprehension are",
+    "stream": False
+})
+print(response.json())
+```
+
+## Performance Improvements
+
+This enhanced model shows significant improvements:
+- **Synonym similarity**: 0.07 → 0.25-0.40 (+257-471% improvement)
+- **Better antonym differentiation**: More precise semantic boundaries
+- **Conceptual clustering**: Semantic rather than lexical token relationships
+- **Enhanced reasoning**: 44 tensor patches targeting embedding, attention, and MLP layers
+
+## Technical Details
+- **Base Model**: Qwen/Qwen3-0.6B
+- **Enhancement Method**: Tensor-Slayer framework with 44 strategic patches
+- **Target Areas**: Embeddings, Attention projections, MLP gates
+- **Validation**: HumanEval benchmark improvements demonstrated
+
+See parent directory README.md for complete technical analysis.