Upload 4 files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+train.json filter=lfs diff=lfs merge=lfs -text

README.es.md

@@ -0,0 +1,68 @@
+> [Ver en ingles/See in english](https://huggingface.co/Komma-LuisMiSanVe/LangToSQL/blob/main/README.md)
+
+<img src="https://raw.githubusercontent.com/LuisMiSanVe/LuisMiSanVe/refs/heads/main/Resources/LangToSQL/LangToSQLLLM_banner.png" style="width: 100%; height: auto;" alt="LangToSQL LLM Banner">
+
+# 🤖 Modelo de IA para sentencias PostgreSQL
+[![image](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge&logo=postgresql&logoColor=white)](https://www.postgresql.org/)
+[![image](https://img.shields.io/badge/json-5E5C5C?style=for-the-badge&logo=json&logoColor=white)](https://www.newtonsoft.com/json)
+[![image](https://img.shields.io/badge/Visual_Studio_Code-0078D4?style=for-the-badge&logo=visual%20studio%20code&logoColor=white)](https://code.visualstudio.com/)
+[![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)](https://www.python.org/)
+[![PyTorch](https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=for-the-badge&logo=PyTorch&logoColor=white)](https://pytorch.org/)
+[![NumPy](https://img.shields.io/badge/numpy-%23013243.svg?style=for-the-badge&logo=numpy&logoColor=white)](https://numpy.org/)
+[![HuggingFace](https://img.shields.io/badge/Hugging%20Face-%23000040.svg?style=for-the-badge&logo=Hugging%20Face&logoColor=ffdf00)](https://huggingface.co/Komma-LuisMiSanVe)
+
+>[!NOTE]
+> Dale un vistazo a las otras versiones del programa:
+>- [WinForms](https://github.com/LuisMiSanVe/LangToSQL/tree/main)
+>- [REST API](https://github.com/LuisMiSanVe/LangToSQL_API/tree/main) 
+>- [ChatBot](https://github.com/LuisMiSanVe/LangToSQL_ChatBot/tree/main)
+>- [NuGet](https://github.com/LuisMiSanVe/LangToSQL_NuGet/tree/main)
+>- [Android](https://github.com/LuisMiSanVe/GeminiLiteSQL/tree/main)
+
+El modelo de IA ha sido entrenado para convertir lenguaje natural a sentencias de PostgreSQL.
+
+## 📝 Explicación de Tecnología
+El modelo usa [DeepSeek Coder](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base) de base y refinado con los datasets de [Spider](https://yale-lily.github.io/spider).
+
+El dataset en archivo `JSON` contiene `train_spider.json` de **Spider**, ya que es el dataset principal.
+
+El modelo se puede exportar a `GGUF` con [llama.cpp](https://github.com/ggml-org/llama.cpp) para que puedas usarlo en programas como [LM Studio](https://lmstudio.ai/).
+
+## 🛠️ Instalación
+Para ejecutar el script de entrenamiento por tu cuenta, primero necesitas instalar [Python](https://www.python.org/) y ejecuta este comando:
+```
+pip install transformers datasets peft accelerate bitsandbytes trl
+```
+Dependiendo en la versión, es posible que necesites usar este en su lugar:
+```
+py -m pip install transformers datasets peft accelerate bitsandbytes trl
+```
+
+## 📂 Archivos
+Este repositorio incluye los archivos del modelo LLM entrenado, su script de entrenamiento y el dataset para entrenar.
+
+Puedes descargar el `GGUF` final desde los [Lanzamientos](https://github.com/LuisMiSanVe/LangToSQL_LLM/releases).
+
+## 🚀 Lanzamientos
+Una versión será lanzada solo cuando se cumplan los siguientes puntos:\
+Nuevas funciones importantes y arreglos de fallos criticos causarán la salida inmediata de una nueva versión, mientras que otros cambios/arreglos menores deberán esperar una semana desde que se incluyeron en el repositorio antes de ser incluidos en la nueva versión, para que otros posibles cambios puedan ser añadidos tambien.
+>[!NOTE]
+>Estos posibles nuevos cambios no alargarán la espera de la salida de la nueva versión a más de una semana.
+
+El número de la versión seguirá este formato: \
+\[Añadido Importante\].\[Añadido Menor\].\[Arreglos de Errores\]
+
+## 💻 Tecnologías usadas
+- Lenguaje de programación: [Python](https://www.python.org/)
+- Librerías:
+  - [transformers](https://pypi.org/project/transformers/)
+  - [datasets](https://pypi.org/project/datasets/)
+  - [peft](https://pypi.org/project/peft/)
+  - [acceletare](https://pypi.org/project/accelerate/)
+  - [bitsandbytes](https://pypi.org/project/bitsandbytes/)
+  - [trl](https://pypi.org/project/trl/)
+- Otros:
+  - [llama.cpp](https://lmstudio.ai/)
+  - [DeepSeek Coder](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base)
+  - [Spider](https://yale-lily.github.io/spider)
+- IDE Recomendado: [VS Code](https://code.visualstudio.com/)

README.md

@@ -1,3 +1,68 @@
----
-license: apache-2.0
----
+> [See in spanish/Ver en español](https://huggingface.co/Komma-LuisMiSanVe/LangToSQL/blob/main/README.es.md)
+
+<img src="https://raw.githubusercontent.com/LuisMiSanVe/LuisMiSanVe/refs/heads/main/Resources/LangToSQL/LangToSQLLLM_banner.png" style="width: 100%; height: auto;" alt="LangToSQL LLM Banner">
+
+# 🤖 AI Model for PostgreSQL queries
+[![image](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge&logo=postgresql&logoColor=white)](https://www.postgresql.org/)
+[![image](https://img.shields.io/badge/json-5E5C5C?style=for-the-badge&logo=json&logoColor=white)](https://www.newtonsoft.com/json)
+[![image](https://img.shields.io/badge/Visual_Studio_Code-0078D4?style=for-the-badge&logo=visual%20studio%20code&logoColor=white)](https://code.visualstudio.com/)
+[![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)](https://www.python.org/)
+[![PyTorch](https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=for-the-badge&logo=PyTorch&logoColor=white)](https://pytorch.org/)
+[![NumPy](https://img.shields.io/badge/numpy-%23013243.svg?style=for-the-badge&logo=numpy&logoColor=white)](https://numpy.org/)
+[![HuggingFace](https://img.shields.io/badge/Hugging%20Face-%23000040.svg?style=for-the-badge&logo=Hugging%20Face&logoColor=ffdf00)](https://huggingface.co/Komma-LuisMiSanVe)
+
+>[!NOTE]
+> Check out other versions of this program:
+>- [WinForms](https://github.com/LuisMiSanVe/LangToSQL/tree/main)
+>- [REST API](https://github.com/LuisMiSanVe/LangToSQL_API/tree/main) 
+>- [ChatBot](https://github.com/LuisMiSanVe/LangToSQL_ChatBot/tree/main)
+>- [NuGet](https://github.com/LuisMiSanVe/LangToSQL_NuGet/tree/main)
+>- [Android](https://github.com/LuisMiSanVe/GeminiLiteSQL/tree/main)
+
+The AI model has been trained for turning natural language to PostgreSQL queries.
+
+## 📝 Technology Explanation
+This model uses [DeepSeek Coder](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base) as a base and then is fine tuned with [Spider](https://yale-lily.github.io/spider) datasets.
+
+The `JSON` dataset file contains **Spider**'s `train_spider.json` as is the main dataset.
+
+The model can be exported to `GGUF` with [llama.cpp](https://github.com/ggml-org/llama.cpp) so it can be used by programs like [LM Studio](https://lmstudio.ai/).
+
+## 🛠️ Setup
+In order to execute the training script for your own, you first need to install [Python](https://www.python.org/) and run this command:
+```
+pip install transformers datasets peft accelerate bitsandbytes trl
+```
+Depending on the version, you may have to use this instead:
+```
+py -m pip install transformers datasets peft accelerate bitsandbytes trl
+```
+
+## 📂 Files
+This repository includes the trained LLM model's files, its training script and the training dataset.
+
+You can download the final `GGUF` in the [Releases](https://github.com/LuisMiSanVe/LangToSQL_LLM/releases).
+
+## 🚀 Releases
+The version will be released using these versioning policies:\
+New major features and critical bug fixes will cause the immediate release of a new version, while other minor changes or fixes will wait one week since the time the change is introduced in the repository before being included in the new version, so that other potential changes can be added.
+>[!NOTE]
+>These potencial new changes will not increase the wait time for the new version beyond one week.
+
+The version number will follow this format: \
+\[Major Feature\].\[Minor Feature\].\[Bug Fixes\]
+
+## 💻 Technologies Used
+- Programming Language: [Python](https://www.python.org/)
+- Libraries:
+  - [transformers](https://pypi.org/project/transformers/)
+  - [datasets](https://pypi.org/project/datasets/)
+  - [peft](https://pypi.org/project/peft/)
+  - [acceletare](https://pypi.org/project/accelerate/)
+  - [bitsandbytes](https://pypi.org/project/bitsandbytes/)
+  - [trl](https://pypi.org/project/trl/)
+- Other: 
+  - [llama.cpp](https://github.com/ggml-org/llama.cpp)
+  - [DeepSeek Coder](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base)
+  - [Spider](https://yale-lily.github.io/spider)
+- Recommended IDE: [VS Code](https://code.visualstudio.com/)

train.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c43d0d72e59e1a9e1a60837da9bf70d5a6277226bdb7f634d544f380646f527a
+size 24928884

trainer.py

@@ -0,0 +1,92 @@
+import torch
+from datasets import load_dataset
+from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments
+from peft import LoraConfig, PeftModel
+from trl import SFTTrainer
+
+model_name = "deepseek-ai/deepseek-coder-1.3b-base"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+tokenizer.pad_token = tokenizer.eos_token
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.float32,
+    device_map={"": "cpu"} # Sets CPU for training, you can change it to use the GPU instead
+)
+
+dataset = load_dataset("json", data_files="train.json", split="train")
+
+def format_example(example):
+    return {
+        "instruction": example["question"],
+        "input": "",
+        "output": example["query"]
+    }
+
+dataset = dataset.map(format_example)
+
+def tokenize(example):
+    prompt_ids = tokenizer(
+        example["instruction"],
+        padding="max_length",
+        truncation=True,
+        max_length=512
+    ).input_ids
+
+    label_ids = tokenizer(
+        example["output"],
+        padding="max_length",
+        truncation=True,
+        max_length=512
+    ).input_ids
+
+    attention_mask = [1 if id != tokenizer.pad_token_id else 0 for id in prompt_ids]
+
+    return {
+        "input_ids": prompt_ids,
+        "attention_mask": attention_mask,
+        "labels": label_ids
+    }
+
+dataset = dataset.map(tokenize, batched=False)
+
+peft_config = LoraConfig(
+    r=16,
+    lora_alpha=32,
+    target_modules=["q_proj", "v_proj"],
+    lora_dropout=0.05,
+    bias="none",
+    task_type="CAUSAL_LM"
+)
+
+training_args = TrainingArguments(
+    output_dir="./sql-model",
+    per_device_train_batch_size=1,
+    gradient_accumulation_steps=4,
+    learning_rate=2e-4,
+    num_train_epochs=1, # More epochs -> better accuracy but longer training
+    logging_steps=10,
+    save_strategy="epoch",
+    fp16=False
+)
+
+trainer = SFTTrainer(
+    model=model,
+    train_dataset=dataset,
+    peft_config=peft_config,
+    args=training_args
+)
+
+trainer.train()
+
+trainer.model.save_pretrained("./sql-model")
+tokenizer.save_pretrained("./sql-model")
+
+base_model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    torch_dtype=torch.float32,
+    device_map={"": "cpu"}
+)
+model_merged = PeftModel.from_pretrained(base_model, "./sql-model")
+model_merged = model_merged.merge_and_unload()
+model_merged.save_pretrained("./sql-model-merged")