Update README.md

README.md

@@ -1,201 +1,257 @@
 ---
 library_name: transformers
-tags: []
+tags:
+- text-to-SQL
+- SQL
+- code-generation
+- NLQ-to-SQL
+- text2SQL
+inference:
+  parameters:
+    max_length: 200
+widget:
+- text: |-
+    CREATE TABLE Loans 
+    { 
+        loan_id number, 
+        client_id number, 
+        budget real, 
+        duration number, 
+        interest real, 
+        status varchar 
+    } 
+     CREATE TABLE Clients 
+    { 
+        client_id number, 
+        first_name varchar, 
+        last_name varchar, 
+        email varchar, 
+        city varchar, 
+        year_of_birth number 
+    } 
+     CREATE TABLE Accounts 
+    { 
+        account_id number, 
+        client_id number, 
+        balance real, 
+        type varchar 
+    } 
+     CREATE TABLE Deposits 
+    { 
+        deposit_id number, 
+        account_id number, 
+        source varchar, 
+        amount real 
+    } 
+     -- Using valid SQLite, answer the following question for the tables provided above. 
+     -- What is the duration and budget of the loan id 16342? 
+     SELECT 
+  example_title: Loan duration
+- text: |-
+    CREATE TABLE Transactions 
+    { 
+        transaction_id number, 
+        timestamp_id number, 
+        primary_contract_id number, 
+        client_id number, 
+        beneficiary_id number, 
+        transaction_amount real, 
+        is_fraudulent boolean, 
+        product_family_code varchar, 
+        amount_currency varchar 
+    } 
+     CREATE TABLE Beneficiary 
+    { 
+        beneficiary_id number, 
+        bank_branch_id number, 
+        country_name varchar, 
+        country_code varchar 
+    } 
+     CREATE TABLE Source 
+    { 
+        primary_contract_id number, 
+        client_id number, 
+        counterparty_bank_branch_id number, 
+        counterparty_donor_id number 
+    } 
+     CREATE TABLE Time 
+    { 
+        timestamp_id number, 
+        week_number number, 
+        day_number number, 
+        hour_number number, 
+        day_name varchar, 
+        year number, 
+        month_number number 
+    } 
+     -- Using valid SQLite, answer the following question for the tables provided above. 
+     -- How many transactions for the client id 15482? 
+     SELECT 
+  example_title: Client Transactions
+datasets:
+- salmane11/BanQies
+language:
+- en
+base_model:
+- bigcode/starcoder2-3b
 ---
 
-# Model Card for Model ID
+# BanQL-3B
 
-<!-- Provide a quick summary of what the model is/does. -->
+## Model Description
 
+BanQL is a family of Code LLMs dedicated solely for the text-to-SQL task in the Financial domain. 
 
+The checkpoint included in this repository is based on [bigcode/starcoder2-3b](https://huggingface.co/bigcode/starcoder2-3b) and further finetuned on [BanQies](https://huggingface.co/datasets/salmane11/BanQies), a dataset generated using [SelectCraft](https://github.com/ezzini/SelectCraft) composed of NLQ-SQL pairs from the financial domain.
 
-## Model Details
 
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure 
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+## Finetuning Procedure
+
+BanQL was fine-tuned using PEFT (Parameter-Efficient Fine-Tuning) techniques, specifically LoRA (Low-Rank Adaptation) adapters. 
 
+## Intended Use and Limitations
+
+The model was designed as a use case to prove the efficiency of SelectCraft in generating large-scale good quality domain-specific text-to-SQL datasets. The model is mainly finetuned on the database schemas displayed above. The prompt format is defined below.
+
+## How to Use
+
+Example 1: Loans_DB
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+device="cuda"
+tokenizer = AutoTokenizer.from_pretrained("salmane11/BanQL-3b")
+model = AutoModelForCausalLM.from_pretrained("salmane11/BanQL-3b").to(device)
+
+input_text = """
+      CREATE TABLE Loans {
+            loan_id number, 
+            client_id number, 
+            budget real, 
+            duration number, 
+            interest real, 
+            status varchar
+        }
 
+        CREATE TABLE Clients {
+            client_id number, 
+            first_name varchar, 
+            last_name varchar, 
+            email varchar, 
+            city varchar, 
+            year_of_birth number
+        }
+
+        CREATE TABLE Accounts {
+            account_id number, 
+            client_id number, 
+            balance real, 
+            type varchar
+        }
+
+        CREATE TABLE Deposits{
+            deposit_id number, 
+            account_id number, 
+            source varchar, 
+            amount real
+        }
+
+        -- Using valid SQLite, answer the following question for the tables provided above.
+
+        -- What is the duration and budget of the loan id 16342?
+
+        SELECT"""
+
+encoding = tokenizer.encode_plus(input_text,pad_to_max_length=True, return_tensors="pt").to(device)
+input_ids, attention_masks = encoding["input_ids"].to(device), encoding["attention_mask"].to(device)
+    
+
+outputs = model.generate(
+    input_ids=input_ids, attention_mask=attention_masks,
+    max_length=512,
+    do_sample=True,
+    top_k=120,
+    top_p=0.95,
+    early_stopping=True,
+)
+line = tokenizer.decode(outputs[0], skip_special_tokens=True,clean_up_tokenization_spaces=True)
+query_begining = line.find("SELECT")
+print(line[query_begining:])
+```
+
+Example 2: Transactions_DB
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+device="cuda"
+tokenizer = AutoTokenizer.from_pretrained("salmane11/BanQL-3b")
+model = AutoModelForCausalLM.from_pretrained("salmane11/BanQL-3b").to(device)
+
+input_text = """
+        CREATE TABLE Transactions {
+            transaction_id number, 
+            timestamp_id number, 
+            primary_contract_id number, 
+            client_id number, 
+            beneficiary_id number, 
+            transaction_amount real, 
+            is_fraudulent boolean, 
+            product_family_code varchar, 
+            amount_currency varchar
+        }
+
+        CREATE TABLE Beneficiary {
+            beneficiary_id number, 
+            bank_branch_id number, 
+            country_name varchar, 
+            country_code varchar, 
+        }
+
+        CREATE TABLE Source {
+            primary_contract_id number, 
+            client_id number, 
+            counterparty_bank_branch_id number, 
+            counterparty_donor_id number, 
+        }
+
+        CREATE TABLE Time{
+            timestamp_id number, 
+            week_number number, 
+            day_number number, 
+            hour_number number, 
+            day_name varchar, 
+            year number,
+            month_number number
+        }
+
+        -- Using valid SQLite, answer the following question for the tables provided above.
+
+        -- How many transactions for the client id 15482?
+
+        SELECT"""
+
+
+encoding = tokenizer.encode_plus(input_text,pad_to_max_length=True, return_tensors="pt").to(device)
+input_ids, attention_masks = encoding["input_ids"].to(device), encoding["attention_mask"].to(device)
+    
+
+outputs = model.generate(
+    input_ids=input_ids, attention_mask=attention_masks,
+    max_length=512,
+    do_sample=True,
+    top_k=120,
+    top_p=0.95,
+    early_stopping=True,
+)
+line = tokenizer.decode(outputs[0], skip_special_tokens=True,clean_up_tokenization_spaces=True)
+query_begining = line.find("SELECT")
+print(line[query_begining:])
+```
+
+
+
+## Cite our work
+
+Citation