Upload 9 files

.gitattributes

@@ -34,3 +34,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 pydevmini1-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
+pydevmini1.IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,355 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+- en
+base_model:
+- bralynn/pydevmini1
+pipeline_tag: text-generation
+tags:
+- code
+- codeqwen
+- chat
+- qwen
+- qwen-coder
+model_creator: bralynn
+model_name: pydevmini1
+model_type: qwen3
+datasets:
+- m-a-p/CodeFeedback-Filtered-Instruction
+quantized_by: CISC
+---
+
+# pydevmini1 - SOTA GGUF
+- Model creator: [bralynn](https://huggingface.co/bralynn)
+- Original model: [pydevmini1](https://huggingface.co/bralynn/pydevmini1)
+
+<!-- description start -->
+## Description
+
+This repo contains State Of The Art quantized GGUF format model files for [pydevmini1](https://huggingface.co/bralynn/pydevmini1).
+
+Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of python-specific answers from the [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) dataset.
+
+Fill-in-Middle tokens are automatically detected and supported as of commit [11ac980](https://github.com/ggml-org/llama.cpp/commit/11ac9800aff532715a5bc7991062c68ba3472e6e), see [example](#simple-llama-cpp-python-example-fill-in-middle-code).
+
+<!-- description end -->
+
+
+<!-- prompt-template start -->
+## Prompt template: ChatML
+
+```
+<|im_start|>system
+{system_prompt}<|im_end|>
+<|im_start|>user
+{prompt}<|im_end|>
+<|im_start|>assistant
+```
+
+<!-- prompt-template end -->
+
+
+<!-- compatibility_gguf start -->
+## Compatibility
+
+These quantised GGUFv3 files are compatible with llama.cpp from April 9th 2025 onwards, as of commit [d3bd719](https://github.com/ggml-org/llama.cpp/commit/d3bd7193ba66c15963fd1c59448f22019a8caf6e)
+
+They are also compatible with many third party UIs and libraries provided they are built using a recent llama.cpp.
+
+## Explanation of quantisation methods
+
+<details>
+  <summary>Click to see details</summary>
+
+The new methods available are:
+
+* GGML_TYPE_IQ1_S - 1-bit quantization in super-blocks with an importance matrix applied, effectively using 1.56 bits per weight (bpw)
+* GGML_TYPE_IQ1_M - 1-bit quantization in super-blocks with an importance matrix applied, effectively using 1.75 bpw
+* GGML_TYPE_IQ2_XXS - 2-bit quantization in super-blocks with an importance matrix applied, effectively using 2.06 bpw
+* GGML_TYPE_IQ2_XS - 2-bit quantization in super-blocks with an importance matrix applied, effectively using 2.31 bpw
+* GGML_TYPE_IQ2_S - 2-bit quantization in super-blocks with an importance matrix applied, effectively using 2.5 bpw
+* GGML_TYPE_IQ2_M - 2-bit quantization in super-blocks with an importance matrix applied, effectively using 2.7 bpw
+* GGML_TYPE_IQ3_XXS - 3-bit quantization in super-blocks with an importance matrix applied, effectively using 3.06 bpw
+* GGML_TYPE_IQ3_XS - 3-bit quantization in super-blocks with an importance matrix applied, effectively using 3.3 bpw
+* GGML_TYPE_IQ3_S - 3-bit quantization in super-blocks with an importance matrix applied, effectively using 3.44 bpw
+* GGML_TYPE_IQ3_M - 3-bit quantization in super-blocks with an importance matrix applied, effectively using 3.66 bpw
+* GGML_TYPE_IQ4_XS - 4-bit quantization in super-blocks with an importance matrix applied, effectively using 4.25 bpw
+* GGML_TYPE_IQ4_NL - 4-bit non-linearly mapped quantization with an importance matrix applied, effectively using 4.5 bpw
+
+Refer to the Provided Files table below to see what files use which methods, and how.
+</details>
+<!-- compatibility_gguf end -->
+
+<!-- README_GGUF.md-provided-files start -->
+## Provided files
+
+| Name | Quant method | Bits | Size | Max RAM required | Use case |
+| ---- | ---- | ---- | ---- | ---- | ----- |
+| [pydevmini1.IQ2_S.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ2_S.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ2_S.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ2_S | 2 | 1.5 GB| 2.0 GB | small, substantial quality loss |
+| [pydevmini1.IQ2_M.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ2_M.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ2_M.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ2_M | 2 | 1.6 GB| 2.1 GB | small, greater quality loss |
+| [pydevmini1.IQ3_XXS.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ3_XXS.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ3_XXS.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ3_XXS | 3 | 1.7 GB| 2.2 GB | very small, high quality loss |
+| [pydevmini1.IQ3_XS.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ3_XS.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ3_XS.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ3_XS | 3 | 1.8 GB| 2.3 GB | small, substantial quality loss |
+| [pydevmini1.IQ3_S.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ3_S.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ3_S.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ3_S | 3 | 1.9 GB| 2.4 GB | small, greater quality loss |
+| [pydevmini1.IQ3_M.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ3_M.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ3_M.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ3_M | 3 | 2.0 GB| 2.5 GB | medium, balanced quality |
+| [pydevmini1.IQ4_XS.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.IQ4_XS.gguf) ([with YaRN](https://ciscai-gguf-editor.hf.space/download/CISCai/pydevmini1-SOTA-GGUF/pydevmini1.IQ4_XS.gguf?branch=main&add=%5B%22qwen3.context_length%22,4,1048576%5D&add=%5B%22qwen3.rope.scaling.type%22,8,%22yarn%22%5D&add=%5B%22qwen3.rope.scaling.factor%22,6,4%5D&add=%5B%22qwen3.rope.scaling.original_context_length%22,4,262144%5D)) | IQ4_XS | 4 | 2.3 GB| 2.8 GB | small, marginal quality loss - recommended |
+
+Generated importance matrix file: [pydevmini1.imatrix.gguf](https://huggingface.co/CISCai/pydevmini1-SOTA-GGUF/blob/main/pydevmini1.imatrix.gguf)
+
+**Note**: the above RAM figures assume no GPU offloading with 4K context. If layers are offloaded to the GPU, this will reduce RAM usage and use VRAM instead.
+
+<!-- README_GGUF.md-provided-files end -->
+
+<!-- README_GGUF.md-how-to-run start -->
+## Example `llama.cpp` command
+
+Make sure you are using `llama.cpp` from commit [d3bd719](https://github.com/ggml-org/llama.cpp/commit/d3bd7193ba66c15963fd1c59448f22019a8caf6e) or later.
+
+```shell
+./llama-cli -ngl 37 -m pydevmini1.IQ4_XS.gguf --color -c 262144 --temp 0.7 --top-p 0.8 --top-k 20 --repeat-penalty 1.05 --jinja
+```
+
+Change `-ngl 37` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
+
+Change `-c 262144` to the desired sequence length.
+
+If you are low on V/RAM try quantizing the K-cache with `-ctk q8_0` or even `-ctk q4_0` for big memory savings (depending on context size).
+There is a similar option for V-cache (`-ctv`), only available if you enable Flash Attention (`-fa`) as well.
+
+For other parameters and how to use them, please refer to [the llama.cpp documentation](https://github.com/ggml-org/llama.cpp/blob/master/tools/main/README.md)
+
+## How to run from Python code
+
+You can use GGUF models from Python using the [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) module.
+
+### How to load this model in Python code, using llama-cpp-python
+
+For full documentation, please see: [llama-cpp-python docs](https://llama-cpp-python.readthedocs.io/en/latest/).
+
+#### First install the package
+
+Run one of the following commands, according to your system:
+
+```shell
+# Prebuilt wheel with basic CPU support
+pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+# Prebuilt wheel with NVidia CUDA acceleration
+pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121 (or cu122 etc.)
+# Prebuilt wheel with Metal GPU acceleration
+pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
+# Build base version with no GPU acceleration
+pip install llama-cpp-python
+# With NVidia CUDA acceleration
+CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python
+# Or with OpenBLAS acceleration
+CMAKE_ARGS="-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS" pip install llama-cpp-python
+# Or with AMD ROCm GPU acceleration (Linux only)
+CMAKE_ARGS="-DGGML_HIPBLAS=on" pip install llama-cpp-python
+# Or with Metal GPU acceleration for macOS systems only
+CMAKE_ARGS="-DGGML_METAL=on" pip install llama-cpp-python
+# Or with Vulkan acceleration
+CMAKE_ARGS="-DGGML_VULKAN=on" pip install llama-cpp-python
+# Or with SYCL acceleration
+CMAKE_ARGS="-DGGML_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx" pip install llama-cpp-python
+
+# In windows, to set the variables CMAKE_ARGS in PowerShell, follow this format; eg for NVidia CUDA:
+$env:CMAKE_ARGS = "-DGGML_CUDA=on"
+pip install llama-cpp-python
+```
+
+#### Simple llama-cpp-python example code
+
+```python
+from llama_cpp import Llama
+
+# Chat Completion API
+
+llm = Llama(model_path="./pydevmini1.IQ4_XS.gguf", n_gpu_layers=37, n_ctx=262144)
+print(llm.create_chat_completion(
+    repeat_penalty = 1.05,
+    messages = [
+        {
+            "role": "user",
+            "content": "Pick a LeetCode challenge and solve it in Python."
+        }
+    ]
+))
+```
+
+#### Simple llama-cpp-python example fill-in-middle code
+
+```python
+from llama_cpp import Llama
+
+# Completion API
+
+prompt = "def add("
+suffix = "\n    return sum\n\n"
+
+llm = Llama(model_path="./pydevmini1.IQ4_XS.gguf", n_gpu_layers=37, n_ctx=262144)
+output = llm.create_completion(
+    temperature = 0.0,
+    repeat_penalty = 1.0,
+    prompt = prompt,
+    suffix = suffix
+)
+
+# Models sometimes repeat suffix in response, attempt to filter that
+response = output["choices"][0]["text"]
+response_stripped = response.rstrip()
+unwanted_response_suffix = suffix.rstrip()
+unwanted_response_length = len(unwanted_response_suffix)
+
+filtered = False
+if unwanted_response_suffix and response_stripped[-unwanted_response_length:] == unwanted_response_suffix:
+    response = response_stripped[:-unwanted_response_length]
+    filtered = True
+
+print(f"Fill-in-Middle completion{' (filtered)' if filtered else ''}:\n\n{prompt}\033[32m{response}\033[{'33' if filtered else '0'}m{suffix}\033[0m")
+```
+
+#### Simple llama-cpp-python example function calling code
+
+```python
+from llama_cpp import Llama
+
+# Chat Completion API
+
+grammar = LlamaGrammar.from_json_schema(json.dumps({
+    "type": "array",
+    "items": {
+        "type": "object",
+        "required": [ "name", "arguments" ],
+        "properties": {
+            "name": {
+                "type": "string"
+            },
+            "arguments": {
+                "type": "object"
+            }
+        }
+    }
+}))
+
+llm = Llama(model_path="./pydevmini1.IQ4_XS.gguf", n_gpu_layers=37, n_ctx=262144)
+response = llm.create_chat_completion(
+      temperature = 0.0,
+      repeat_penalty = 1.05,
+      messages = [
+        {
+          "role": "user",
+          "content": "What's the weather like in Oslo and Stockholm?"
+        }
+      ],
+      tools=[{
+        "type": "function",
+        "function": {
+          "name": "get_current_weather",
+          "description": "Get the current weather in a given location",
+          "parameters": {
+            "type": "object",
+            "properties": {
+              "location": {
+                "type": "string",
+                "description": "The city and state, e.g. San Francisco, CA"
+              },
+              "unit": {
+                "type": "string",
+                "enum": [ "celsius", "fahrenheit" ]
+              }
+            },
+            "required": [ "location" ]
+          }
+        }
+      }],
+      grammar = grammar
+)
+print(json.loads(response["choices"][0]["text"]))
+
+print(llm.create_chat_completion(
+      temperature = 0.0,
+      repeat_penalty = 1.05,
+      messages = [
+        {
+          "role": "user",
+          "content": "What's the weather like in Oslo?"
+        },
+        { # The tool_calls is from the response to the above with tool_choice active
+          "role": "assistant",
+          "content": None,
+          "tool_calls": [
+            {
+              "id": "call__0_get_current_weather_cmpl-...",
+              "type": "function",
+              "function": {
+                "name": "get_current_weather",
+                "arguments": { "location": "Oslo, Norway" , "unit": "celsius" }
+              }
+            }
+          ]
+        },
+        { # The tool_call_id is from tool_calls and content is the result from the function call you made
+          "role": "tool",
+          "content": "20",
+          "tool_call_id": "call__0_get_current_weather_cmpl-..."
+        }
+      ],
+      tools=[{
+        "type": "function",
+        "function": {
+          "name": "get_current_weather",
+          "description": "Get the current weather in a given location",
+          "parameters": {
+            "type": "object",
+            "properties": {
+              "location": {
+                "type": "string",
+                "description": "The city and state, e.g. San Francisco, CA"
+              },
+              "unit": {
+                "type": "string",
+                "enum": [ "celsius", "fahrenheit" ]
+              }
+            },
+            "required": [ "location" ]
+          }
+        }
+      }],
+      #tool_choice={
+      #  "type": "function",
+      #  "function": {
+      #    "name": "get_current_weather"
+      #  }
+      #}
+))
+```
+
+<!-- README_GGUF.md-how-to-run end -->
+
+<!-- original-model-card start -->
+## 🚀 Try It Yourself (for free)
+
+Don't just take my word for it. Test the model right now under the exact conditions shown in the video demonstration.
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1c8WCvsVovCjIyqPcwORX4c_wQ7NyIrTP?usp=sharing)
+
+---
+
+## Model Details
+*   **Model Type:** Causal Language Model
+*   **Number of Parameters:** 4.0B
+*   **Number of Parameters (Non-Embedding):** 3.6B
+*   **Number of Layers:** 36
+*   **Number of Attention Heads (GQA):** 32 for Q, 8 for KV
+*   **Context Length:** 262,144 tokens (native)
+
+### Recommended Inference Parameters
+
+For best results, I suggest using the following generation parameters:
+*   **Temperature:** 0.7
+*   **Top P:** 0.8
+*   **Top K:** 20
+*   **Min P:** 0.0
+
+How to Contribute & Provide Feedback
+For any and all feedback, please open a Community discussion tab on this model repository or join our Discord!
+Discord: https://discord.gg/RqwqMGhqaC

pydevmini1.IQ2_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65d9e96630ca0f4036a5a35f7dc6cbda1bcbf1b8b6fc6923913f1dde950ace90
+size 1658843296

pydevmini1.IQ2_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f919bea6018c67d04942bfbc94ab2321f0545f2f758b6d9b1b0d0de4c176644
+size 1563160736

pydevmini1.IQ3_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19ecfd81f370fbe66de4128c35157a5f06bd1e16d3ebca0967921fe959274344
+size 2057097376

pydevmini1.IQ3_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f91d0c5d94c88fcaf7677ffc3881b7f990734893578539f273a645815ab5274
+size 1993732256

pydevmini1.IQ3_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96b758905b3fe6d2f78a66ce739630a66b6ed2ed93be23b1ab4ea37a5ff0a647
+size 1908576416

pydevmini1.IQ3_XXS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:08c3486bbdddee7be01542b63421863bdc5840cf35f62b1eb07c710941c00c1f
+size 1816047776

pydevmini1.IQ4_XS.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce01051f836b15ec93d86c77a83c5da8352c35b03bc27431e89ca4798699884a
+size 2364952736

pydevmini1.imatrix.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c4558f3b4a628bad256fe27f53245307403940e6b220f5335a6b37c9f723c38
+size 3872640