Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+adaptors-anant-base/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+adaptors-anant-reason/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+anant-1.5b-base-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+anant-1.5b-base-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+anant-3b-reason-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+anant-3b-reason-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,274 @@
 ---
-license: gpl
+license: apache-2.0
+language:
+  - en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - gguf
+  - ollama
+  - llama.cpp
+  - qwen2.5
+  - lora
+  - reasoning
+  - local-ai
 ---
+
+# Anant
+
+Anant is a small, local-first AI model family by BlueishAI. It is designed to run comfortably on consumer devices while still giving useful assistant behavior and stronger reasoning than the base chat variant when needed.
+
+This repository contains GGUF builds for local inference with Ollama, llama.cpp, LM Studio, and other GGUF-compatible runtimes.
+
+## Available Files
+
+| File | Variant | Quantization | Best for |
+| --- | --- | --- | --- |
+| `anant-1.5b-base-Q4_K_M.gguf` | Base | Q4_K_M | Fastest local chat |
+| `anant-1.5b-base-Q8_0.gguf` | Base | Q8_0 | Better base quality with more RAM |
+| `anant-3b-reason-Q4_K_M.gguf` | Reason | Q4_K_M | Math, logic, planning, and analysis |
+| `anant-3b-reason-Q8_0.gguf` | Reason | Q8_0 | Higher quality reasoning with more RAM |
+
+## Which Variant Should You Use?
+
+Use `anant-1.5b-base-Q4_K_M.gguf` if you want the fastest Anant model for normal chat, short answers, summarization, rewriting, and lightweight coding help.
+
+Use `anant-3b-reason-Q4_K_M.gguf` if the task needs stronger reasoning: math, multi-step logic, debugging, planning, or careful analysis.
+
+Use Q8_0 if you have enough memory and want quality closer to the merged model. Use Q4_K_M if you want the best balance of size, speed, and quality.
+
+## What Makes Anant Different?
+
+Anant is not trying to be the largest model. The goal is a practical local model that:
+
+- Runs on normal laptops and desktops.
+- Has small GGUF files that are easy to download and run.
+- Keeps a fast 1.5B base variant for everyday use.
+- Provides a larger 3B reason variant for harder prompts.
+- Uses LoRA adaptation instead of training a full model from scratch.
+- Uses Q4_K_M and Q8_0 GGUF quantization for local inference.
+
+The base model is intended to be quick and useful. The reason model is intended to be more deliberate, especially on tasks where the base model may make algebra, logic, or constraint-checking mistakes.
+
+## Training Overview
+
+Anant is built from Qwen2.5 Instruct checkpoints and adapted using LoRA.
+
+The training process used this broad pipeline:
+
+1. Start from a Qwen2.5 Instruct checkpoint.
+2. Train a LoRA adapter on curated instruction or reasoning data.
+3. Merge the LoRA adapter back into the base model.
+4. Export the merged model to F16 GGUF.
+5. Quantize the F16 GGUF into Q4_K_M and Q8_0.
+6. Test the final GGUF models locally with llama.cpp/Ollama style prompts.
+
+The models were NOT trained from scratch. They are adaptations of Qwen2.5 models, which keeps the training cost manageable and preserves the strong tokenizer, chat format, and general language ability of the source model.
+This decision was made because it would've been a waste of time and resources if we had trained it from scratch. So, we took Qwen as the base model since qwen itself is a really strong model.
+
+## Base Variant
+
+The base variant is the lightweight general assistant model.
+
+Base model family:
+
+```text
+Qwen2.5 Instruct
+```
+
+Training method:
+
+```text
+LoRA supervised fine-tuning
+```
+
+Purpose:
+
+- General chat
+- Simple explanations
+- Rewriting
+- Summarization
+- Lightweight coding help
+- Fast local inference
+
+The base model is optimized for responsiveness. It is the model to use when the prompt does not require deep reasoning.
+
+## Reason Variant
+
+The reason variant is the stronger Anant model for multi-step problem solving.
+
+Base model:
+
+```text
+Qwen/Qwen2.5-3B-Instruct
+```
+
+Reasoning dataset:
+
+```text
+open-r1/OpenR1-Math-220k
+```
+
+Training method:
+
+```text
+QLoRA adaptation
+```
+
+Purpose:
+
+- Math reasoning
+- Constraint solving
+- Logic problems
+- Planning
+- Debugging
+- Programming
+- Analysis tasks where checking intermediate steps matters
+
+The reason variant is instructed to decide whether a prompt actually needs reasoning. For simple questions, it should answer directly. For math, logic, debugging, planning, or analysis, it should reason carefully and then give a clear answer.
+
+## LoRA And QLoRA
+
+LoRA means Low-Rank Adaptation. Instead of updating every parameter in the base model, LoRA trains small adapter matrices attached to selected layers of the model. This makes training much cheaper and faster while still changing the model's behavior.
+
+QLoRA is a memory-efficient version of this idea. The base model is loaded in 4-bit quantized form during training, while the LoRA adapter remains trainable. This makes it possible to adapt larger models on limited GPU memory.
+
+After training, the adapter is merged into the base model to create a normal standalone model. That merged model is then exported to GGUF for local inference.
+
+## Why GGUF?
+
+GGUF is the model format used by llama.cpp and many local inference tools. It is useful because it:
+
+- Runs well on CPU.
+- Supports quantized models.
+- Works with Ollama and LM Studio.
+- Makes local deployment simple.
+- Avoids needing a full Python/Transformers stack for inference.
+
+## Quantization
+
+The included models use:
+
+```text
+Q4_K_M
+Q8_0
+```
+
+Q4_K_M is smaller and faster. It is recommended for most users.
+
+Q8_0 is larger and slower, but it keeps more of the original F16 model quality.
+
+Approximate tradeoff:
+
+| Quantization | Size | Speed | Quality |
+| --- | --- | --- | --- |
+| Q4_K_M | Smaller | Faster | Good |
+| Q8_0 | Larger | Slower | Better |
+
+## Ollama Usage
+Run the base model:
+
+```bash
+ollama run anant
+```
+
+Run the reason model:
+
+```bash
+ollama run anant-reason
+```
+
+## llama.cpp Usage
+
+Example with the reason model:
+
+```bash
+llama-cli -m anant-3b-reason-Q4_K_M.gguf -c 4096 -n 512 -p "Solve carefully: A factory makes two products..."
+```
+
+Example with the base model:
+
+```bash
+llama-cli -m anant-1.5b-base-Q4_K_M.gguf -c 4096 -n 256 -p "Explain LoRA in simple terms."
+```
+
+## Example Reasoning Test
+
+Prompt:
+
+```text
+A factory makes two products, A and B.
+Each unit of A needs 2 hours of machine time and 1 hour of labor.
+Each unit of B needs 1 hour of machine time and 3 hours of labor.
+The factory has 100 machine hours and 90 labor hours available.
+Profit is $40 per unit of A and $50 per unit of B.
+
+How many units of A and B should the factory make to maximize profit? Show the reasoning.
+```
+
+Expected answer:
+
+```text
+A = 42
+B = 16
+Maximum profit = $2480
+```
+
+This kind of prompt is useful because it checks whether the model can set up constraints, solve the intersection, evaluate corner points, and reject infeasible answers.
+
+## Web Search
+
+Anant does not browse the internet by itself.
+
+For web search, connect Anant to an external wrapper or application that performs search and injects the retrieved results into the prompt. The model can then answer from those provided results.
+
+Do NOT rely on the model alone for current events, live prices, breaking news, laws, or anything time-sensitive.
+
+## Tokenizer And Chat Format
+
+The tokenizer and chat template come from the Qwen2.5 model family. The Anant variants use the same tokenization format, so the base and reason variants are compatible with the same general chat formatting.
+
+## Limitations
+
+Anant is small. It can be fast and useful, but it will not match large frontier models.
+
+Known limitations:
+
+- May hallucinate facts.
+- Does not have live internet access.
+- Can make reasoning mistakes, especially under long or ambiguous prompts.
+- May struggle with very long context.
+- Coding ability is limited compared with larger code-specialized models.
+- Quantized models may lose some quality compared with F16.
+
+For high-stakes medical, legal, financial, or safety-critical work, verify outputs with reliable sources and expert review.
+
+## Recommended Settings
+
+General base model:
+
+```text
+temperature: 0.7
+top_p: 0.9
+repeat_penalty: 1.08
+context: 4096
+```
+
+Reason model:
+
+```text
+temperature: 0.35
+top_p: 0.9
+repeat_penalty: 1.08
+context: 4096
+```
+
+Lower temperature is recommended for reasoning tasks because it reduces random mistakes.
+
+## Citation
+
+If you use Anant, cite or link this repository and BlueishAI.
+
+```text
+Anant, BlueishAI. Local-first GGUF language models adapted from Qwen2.5 using LoRA.
+```

adaptors-anant-base/adapter_config.json

@@ -0,0 +1,41 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-1.5B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "up_proj",
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adaptors-anant-base/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:661064ab4639409aa05910c4cef3ffafd974d740716b67782e2b0cf755afc4e5
+size 73911112

adaptors-anant-base/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

adaptors-anant-base/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

adaptors-anant-base/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:962b8d8c521fefa934665afddae177326e974ddd6a26e69ff31ad6bccbb5593b
+size 11421994

adaptors-anant-base/tokenizer_config.json

@@ -0,0 +1,208 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

adaptors-anant-reason/adapter_config.json

@@ -0,0 +1,41 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-3B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj",
+    "down_proj",
+    "up_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adaptors-anant-reason/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:150d96d9dc7232fec078cb103cd77f57abf09bccaf38b148f6c4041fb3f84547
+size 59933632

adaptors-anant-reason/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

adaptors-anant-reason/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

adaptors-anant-reason/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:376b9c18ea64bfffe186993357ba03ba8664d6c044154da87c450db70af56794
+size 11421994

adaptors-anant-reason/tokenizer_config.json

@@ -0,0 +1,208 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- messages[0]['content'] }}\n    {%- else %}\n        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n    {%- endif %}\n    {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0]['role'] == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n    {%- else %}\n        {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role }}\n        {%- if message.content %}\n            {{- '\\n' + message.content }}\n        {%- endif %}\n        {%- for tool_call in message.tool_calls %}\n            {%- if tool_call.function is defined %}\n                {%- set tool_call = tool_call.function %}\n            {%- endif %}\n            {{- '\\n<tool_call>\\n{\"name\": \"' }}\n            {{- tool_call.name }}\n            {{- '\", \"arguments\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- '}\\n</tool_call>' }}\n        {%- endfor %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

anant-1.5b-base-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9e5543eda46f2c18208bd689c92c01ad45b2c7da973bc1dfbc4d1c23d518249
+size 1117320832

anant-1.5b-base-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c27c5cbbd0c6da1b2d4f3b885073597e054c81e2fd59f58e79e32f9d7f6f64c
+size 1894532224

anant-3b-reason-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe0e315115845a2fe2f9a28df73e12a620bd7cd78829a02e5b7087f1293f8a90
+size 2104932864

anant-3b-reason-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:585876de907b83e7d57a68cd4f8c5f26dd5b5113eb3cdda3da6260db57c10ef4
+size 3616088576

modelfile-base

@@ -0,0 +1,16 @@
+FROM ./anant-1.5b-base-Q4_K_M.gguf
+
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.08
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+SYSTEM """
+You are Anant, an AI model by BlueishAI.
+
+You are helpful, direct, and concise. Answer the user's question clearly. If the user asks for code, give working code. If the user asks for an explanation, explain the important parts without unnecessary filler.
+
+You do not have live internet access unless an external application provides search results or tool output in the conversation.
+"""

modelfile-base-q4

@@ -0,0 +1,16 @@
+FROM ./anant-1.5b-base-Q4_K_M.gguf
+
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.08
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+SYSTEM """
+You are Anant, an AI model by BlueishAI.
+
+You are helpful, direct, and concise. Answer the user's question clearly. If the user asks for code, give working code. If the user asks for an explanation, explain the important parts without unnecessary filler.
+
+You do not have live internet access unless an external application provides search results or tool output in the conversation.
+"""

modelfile-base-q8

@@ -0,0 +1,16 @@
+FROM ./anant-1.5b-base-Q8_0.gguf
+
+PARAMETER temperature 0.7
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.08
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+SYSTEM """
+You are Anant, an AI model by BlueishAI.
+
+You are helpful, direct, and concise. Answer the user's question clearly. If the user asks for code, give working code. If the user asks for an explanation, explain the important parts without unnecessary filler.
+
+You do not have live internet access unless an external application provides search results or tool output in the conversation.
+"""

modelfile-reason

@@ -0,0 +1,18 @@
+FROM ./anant-3b-reason-Q4_K_M.gguf
+
+PARAMETER temperature 0.35
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.08
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+SYSTEM """
+You are Anant Reason, an AI model by BlueishAI.
+
+Before answering, decide whether the user's request actually needs deliberate reasoning. For simple facts, casual conversation, short definitions, or straightforward instructions, answer directly without showing a long reasoning process.
+
+For math, logic, planning, debugging, analysis, or multi-step problems, reason carefully before answering. Keep any private scratch work internal, then give the user a clear final answer with the key steps needed to trust it.
+
+If information might be current or time-sensitive, say that live information is needed unless an external application provides search results or tool output in the conversation.
+"""

modelfile-reason-q4

@@ -0,0 +1,18 @@
+FROM ./anant-3b-reason-Q4_K_M.gguf
+
+PARAMETER temperature 0.35
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.08
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+SYSTEM """
+You are Anant Reason, an AI model by BlueishAI.
+
+Before answering, decide whether the user's request actually needs deliberate reasoning. For simple facts, casual conversation, short definitions, or straightforward instructions, answer directly without showing a long reasoning process.
+
+For math, logic, planning, debugging, analysis, or multi-step problems, reason carefully before answering. Keep any private scratch work internal, then give the user a clear final answer with the key steps needed to trust it.
+
+If information might be current or time-sensitive, say that live information is needed unless an external application provides search results or tool output in the conversation.
+"""

modelfile-reason-q8

@@ -0,0 +1,18 @@
+FROM ./anant-3b-reason-Q8_0.gguf
+
+PARAMETER temperature 0.35
+PARAMETER top_p 0.9
+PARAMETER repeat_penalty 1.08
+PARAMETER num_ctx 4096
+PARAMETER stop "<|im_start|>"
+PARAMETER stop "<|im_end|>"
+
+SYSTEM """
+You are Anant Reason, an AI model by BlueishAI.
+
+Before answering, decide whether the user's request actually needs deliberate reasoning. For simple facts, casual conversation, short definitions, or straightforward instructions, answer directly without showing a long reasoning process.
+
+For math, logic, planning, debugging, analysis, or multi-step problems, reason carefully before answering. Keep any private scratch work internal, then give the user a clear final answer with the key steps needed to trust it.
+
+If information might be current or time-sensitive, say that live information is needed unless an external application provides search results or tool output in the conversation.
+"""