File size: 1,704 Bytes
653af7e bb03644 653af7e 12ddbd8 653af7e 8b9d85a 6675312 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | ---
license: mit
---
# Model Card
## Overview
This model is a 156M-parameter English-language causal language model trained on a large-scale text corpus and instruction-tuned for general question answering and task completion.
---
## Running script
see github -> https://github.com/firdavsus/LLM_D2
## Model Details
* **Model size:** 156M parameters
* **Architecture:** Transformer (causal LM)
* **Tokenizer:** GPT-2 tokenizer
* **Languages:** English only
---
## Training Curves
## Training Data
### Pretraining
* Dataset: The Pile (10B token subset)
* Domain: mixed-domain text (web, books, articles, code, etc.)
### Instruction Fine-tuning
* Dataset: Alpaca (cleaned subset)
* Size: ~50,000 instruction–response examples
* Formatting: instruction-style prompt/response pairs
---
## Training Setup
### Pretraining
* Steps: **218,000**
* Final training loss: **2.6**
### Post-training (Instruction Fine-tuning)
* Steps: **2,500**
* Final training loss: **1.9**
---
## Evaluation
| Benchmark | Score |
| --------- | -------- |
| HellaSwag | **28.5** |
---
## Intended Use
* Instruction-style prompting
* Basic question answering
* Text generation and summarization
* Lightweight assistant-style tasks (English)
---
## Limitations
* Small model size limits reasoning and factual reliability
* May produce incorrect or inconsistent answers
* Instruction-following quality depends strongly on prompt format
* Not suitable for high-stakes or safety-critical use
# This model has not been safety-aligned. Please apply your own moderation and guardrails when deploying it ;)
FOR ADDITIONAL INFO CHEKC INFO.TXT
|