Spaces:
Build error
Build error
Merge branch 'main' of https://huggingface.co/spaces/BitBasher/EduConnect
Browse files- ipynb/llm/1. F1_Llama2_QnA.ipynb +0 -0
- ipynb/llm/2. Hyper_Fine_tune_Llama_2_with_MCQ.ipynb +0 -0
- ipynb/llm/3. Integrated_model.ipynb +0 -0
- ipynb/llm/4. Chosen_model.ipynb +0 -0
- ipynb/llm/Langchain_router_RAG.ipynb +0 -0
- ipynb/llm/README.md +62 -1
- ipynb/llm/create_instr_FT_data.ipynb +0 -0
- ipynb/llm/llama2_base_lora_finetune.ipynb +0 -0
- ipynb/llm/llama2_chat_lora_finetune.ipynb +0 -0
ipynb/llm/1. F1_Llama2_QnA.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/2. Hyper_Fine_tune_Llama_2_with_MCQ.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/3. Integrated_model.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/4. Chosen_model.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/Langchain_router_RAG.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/README.md
CHANGED
|
@@ -1 +1,62 @@
|
|
| 1 |
-
# LLM Transfer Learning / Fine tuning
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# LLM Transfer Learning / Fine tuning
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+

|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
* Location of model development logs > https://drive.google.com/drive/folders/1OqUzYN_nfBSeZfbQ9qcB1YVkSNzi_gE3?usp=drive_link
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
__Dataset and Model Migration__
|
| 11 |
+
|
| 12 |
+
Overview
|
| 13 |
+
|
| 14 |
+
This README also serves as a guide and legend for the migration and update of datasets and models previously hosted
|
| 15 |
+
under the ssoh namespace, now transferred to the BitBasher namespace on Hugging Face.
|
| 16 |
+
|
| 17 |
+
The purpose of this guide is to facilitate easy referencing with respect to the submitted codes.
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
Migration Details
|
| 21 |
+
|
| 22 |
+
Below is a detailed list of the datasets and models that have been moved, including their old and new locations:
|
| 23 |
+
|
| 24 |
+
* The term 'base model' refers to 'meta-llama/Llama-2-7b-chat-hf' on Hugging Face.
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
1. **MCQ Dataset**
|
| 28 |
+
- Old Location: `ssoh/mcq_dataset_2`
|
| 29 |
+
- New Location: `BitBasher/mcq_dataset_2`
|
| 30 |
+
|
| 31 |
+
2. **Summarization and QnA Dataset**
|
| 32 |
+
- Old Location: `ssoh/sum_n_qna_dataset`
|
| 33 |
+
- New Location: `BitBasher/sum_n_qna_dataset`
|
| 34 |
+
|
| 35 |
+
3. **Model fine-tuned from a base model using the MCQ dataset**
|
| 36 |
+
- Old Location: `ssoh/llama-2-7b-mcq_2`
|
| 37 |
+
- New Location: `BitBasher/llama-2-7b-mcq_2`
|
| 38 |
+
|
| 39 |
+
4. **Model fine-tuned from the MCQ-tuned base model using the Summarization and QnA Dataset (also known as the Integrated model)**
|
| 40 |
+
- Old Location: `ssoh/llama-2-7b-combined_datasets`
|
| 41 |
+
- New Location: `BitBasher/llama-2-7b-combined_datasets`
|
| 42 |
+
|
| 43 |
+
5. **Integrated model converted to the GGUF format**
|
| 44 |
+
- Old Location: `ssoh/llama-2-7b-combined_datasets-GGUF`
|
| 45 |
+
- New Location: `BitBasher/llama-2-7b-combined_datasets-GGUF`
|
| 46 |
+
|
| 47 |
+
6. **Chosen model converted to the GGUF format**
|
| 48 |
+
- Old Location: `ssoh/llama-2-7b-mini-ibased-GGUF`
|
| 49 |
+
- New Location: `BitBasher/llama-2-7b-mini-ibased-GGUF`
|
| 50 |
+
|
| 51 |
+
7. **Chosen model**
|
| 52 |
+
- Old Location: `ssoh/llama-2-7b-mini-ibased`
|
| 53 |
+
- New Location: `BitBasher/llama-2-7b-mini-ibased`
|
| 54 |
+
|
| 55 |
+
8. **Initial dataset**
|
| 56 |
+
- Old Location: `ssoh/mini-ibased-dataset`
|
| 57 |
+
- New Location: `BitBasher/mini-ibased-dataset`
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
Accessing the Datasets and Models
|
| 61 |
+
|
| 62 |
+
* To ensure uninterrupted access, please visit the new locations listed above to access the migrated datasets and models.
|
ipynb/llm/create_instr_FT_data.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/llama2_base_lora_finetune.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
ipynb/llm/llama2_chat_lora_finetune.ipynb
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|