File size: 914 Bytes
a76c78c d04ade0 a76c78c d04ade0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | ---
license: mit
language:
- en
pipeline_tag: text-generation
tags:
- GPT-4
- GPT
---
# GPT-Fem-Compact
GPT-Fem-Compact is a version of GPT-Compact finetuned on 18 GB of text on and about women.
### Model Description
GPT-Fem-Compact has 16 heads, 24 layers, and 1280 embeddings, similar to GPT2-Large, with a context window of 16,384 tokens. This maximized GPU VRAM while balancing size and functionality.
It was trained on a diverse corpus of over 17 billion tokens.
### Model Architecture
| | |
|---------------------------------|----:|
|Layers |24|
|Heads |16|
|Embeddings |1280|
|Context Window |16384 tokens|
|Tokenizer |cl100k_base| |