PulkundwarP commited on
Commit
81c9dfd
·
verified ·
1 Parent(s): b090756

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -26
README.md CHANGED
@@ -1,6 +1,6 @@
1
- # LangPWT
2
 
3
- This repository contains the implementation of a lightweight, modified version of the GPT architecture **LangPWT** trained from scratch using FineWeb-Edu, an open-source dataset. The project demonstrates the design, training, and optimization of a custom natural language model on local hardware.
4
 
5
  ## Features
6
  - **Custom GPT Architecture**: A miniaturized version of the GPT model tailored for efficient training on limited hardware.
@@ -10,7 +10,7 @@ This repository contains the implementation of a lightweight, modified version o
10
 
11
  <div align="center">
12
  <img src="LLM.drawio.png" alt="Description of the image" width="300">
13
- <p><strong>Figure 1: Architecture of LangPWT</p>
14
  </div>
15
 
16
  ## Implementation Details
@@ -19,7 +19,7 @@ This repository contains the implementation of a lightweight, modified version o
19
  - Incorporates modifications to parameter scaling to suit resource-constrained environments.
20
 
21
  2. **Training**
22
- - Training executed locally on NVIDIA GeForce RTX 3050 (Laptop) 4GB GPU, leveraging PyTorch.
23
 
24
  3. **Testing**
25
  - A simple Streamlit UI created for testing generation capability of the model.
@@ -66,26 +66,4 @@ This repository contains the implementation of a lightweight, modified version o
66
  - Dependencies listed in `requirements.txt`
67
  - **Note**: Different OS support different versions of PyTorch/Tensorflow to use CUDA (local GPU). Install only after verifying for your OS.
68
 
69
- ## Usage
70
- 1. Clone the repository:
71
- ```bash
72
- git clone https://github.com/pulkundwar29/LangPWT
73
- cd LangPWT
74
- ```
75
- 2. Create and activate a virtual environment:
76
- ```bash
77
- venv env
78
- env\scripts\activate
79
- ```
80
- 3. Install dependencies:
81
- ```bash
82
- pip install -r requirements.txt
83
- ```
84
- 4. Run the training file **trainpwt.py**
85
- 5. Run the streamlit file: **trial_pwt.py**
86
- 6. Enter your prompt and hit the Generate button.
87
 
88
- <div align="center">
89
- <img src="ex1.png" alt="example text">
90
- <p><strong>Figure 2: Example of Text Generated using LangPWT</p>
91
- </div>
 
1
+ # Leap-0
2
 
3
+ This repository contains the implementation of a lightweight, modified version of the GPT architecture **Leap-0** trained from scratch using FineWeb-Edu, an open-source dataset. The project demonstrates the design, training, and optimization of a custom natural language model on local hardware.
4
 
5
  ## Features
6
  - **Custom GPT Architecture**: A miniaturized version of the GPT model tailored for efficient training on limited hardware.
 
10
 
11
  <div align="center">
12
  <img src="LLM.drawio.png" alt="Description of the image" width="300">
13
+ <p><strong>Figure 1: Architecture of Leap</p>
14
  </div>
15
 
16
  ## Implementation Details
 
19
  - Incorporates modifications to parameter scaling to suit resource-constrained environments.
20
 
21
  2. **Training**
22
+ - Training executed locally on NVIDIA GeForce RTX 4500 ada 24GB GPU, leveraging PyTorch.
23
 
24
  3. **Testing**
25
  - A simple Streamlit UI created for testing generation capability of the model.
 
66
  - Dependencies listed in `requirements.txt`
67
  - **Note**: Different OS support different versions of PyTorch/Tensorflow to use CUDA (local GPU). Install only after verifying for your OS.
68
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69