File size: 4,546 Bytes
301a324 9376dea | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | ---
tags:
- text-generation
- fine-tuned
- deepseek-coder
- spigot
- minecraft
- code-generation
- instruct
- gguf
- ollama
datasets:
- custom_spigot_dataset # You can change this if your dataset has a public Hugging Face ID
license: apache-2.0 # Assuming Apache 2.0, common for DeepSeek. Please verify or change.
model-index:
- name: LunarAI
results: [] # You can add evaluation results here if you have them
---
# LunarAI - Fine-tuned DeepSeek Coder V2 Lite for Spigot Plugin Development
## Model Description
**LunarAI** is a custom language model fine-tuned from the `deepseek-ai/DeepSeek-Coder-V2-Lite-Base` model. It has been specialized to act as an AI programming assistant, with a particular focus on **Spigot/Minecraft plugin development**.
This model is designed to provide accurate code examples, explanations, and guidance related to the Spigot API and general Java programming concepts relevant to creating Minecraft server plugins.
## Training Details
* **Base Model:** `deepseek-ai/DeepSeek-Coder-V2-Lite-Base`
* **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
* **Dataset:** A custom dataset (`spigot_dataset.jsonl`) focused on Spigot/Minecraft plugin development, including common tasks, event handling, and API usage.
* **Adapter Size:** Approximately 1.1 GB (LoRA adapter before merge)
* **Training Framework:** Axolotl
## Model Files
This repository contains two main versions of the fine-tuned model:
1. **Full Merged Model (Safetensors):** The complete model with the LoRA adapter merged into the base model's weights. This is the standard Hugging Face format, ideal for further development or use with `transformers`.
* Files: `model-00001-of-00007.safetensors` through `model-00007-of-00007.safetensors` (totaling ~31.4 GB)
* Configuration files: `config.json`, `tokenizer.json`, `special_tokens_map.json`, etc.
2. **Quantized GGUF Model (for Ollama):** A highly optimized, quantized version of the merged model in GGUF format, specifically designed for efficient local inference with tools like Ollama.
* File: `model.gguf` (~16.7 GB, `q8_0` quantization)
## How to Use LunarAI with Ollama (Recommended for Local Inference)
To run **LunarAI** locally using Ollama, follow these steps:
1. **Ensure Ollama is Installed:** If you don't have Ollama, install it from [ollama.com](https://ollama.com/).
2. **Download `model.gguf`:** You can download the `model.gguf` file directly from this repository's "Files" tab, or use `ollama pull ThePegasusGroup/LunarAI` if Ollama supports direct pulling of GGUF files from the Hub (this might require a `Modelfile` first).
3. **Create a `Modelfile`:** In the same directory as your downloaded `model.gguf`, create a file named `Modelfile` with the following content:
```dockerfile
# Tell Ollama which GGUF file to use
FROM ./model.gguf
# Set the chat template for DeepSeek Coder
TEMPLATE """{% for message in messages %}{% if message['role'] == 'user' %}{{ 'You are an AI programming assistant, utilizing the Deepseek Coder model, developed by Deepseek Company. Follow the user\'s instructions carefully. Respond using markdown.' }}\n### Instruction:\n{{ message['content'] }}\n### Response:\n{% elif message['role'] == 'assistant' %}{{ message['content'] }}{% if not loop.last %}\n{% endif %}{% endif %}{% endfor %}"""
# Set a default parameter
PARAMETER temperature 0.7
```
4. **Create the Model in Ollama:**
```bash
ollama create LunarAI -f ./Modelfile
```
5. **Run LunarAI:**
```bash
ollama run LunarAI
```
You can then start asking it questions related to Spigot plugin development!
## How to Load the Merged Model with Hugging Face Transformers
If you wish to load the full, unquantized merged model for further development or advanced usage with the `transformers` library:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Replace 'ThePegasusGroup/LunarAI' with the actual repo ID if you renamed it
model_id = "ThePegasusGroup/LunarAI"
# Load the model
# Ensure you have sufficient VRAM (GPU memory) or RAM for this large model
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16, # Or torch.float16, or torch.float32 depending on your hardware
device_map="auto",
trust_remote_code=True # Required for DeepSeek-Coder-V2-Lite-Base architecture
)
# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
print("LunarAI model loaded successfully!") |