File size: 2,615 Bytes
a2f7fb9 a3d6586 a2f7fb9 1f2beaa f20d35d 0339f83 1f2beaa 82c79d3 1f2beaa 82c79d3 1f2beaa 7ba2875 11b8123 4d562d1 11b8123 e4245db 7857901 11b8123 81f8fd9 4d562d1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
license: mit
datasets:
- HuggingFaceFW/fineweb-edu
- TIGER-Lab/Fineweb-Instruct
language:
- en
tags:
- gpt2
- gpt
- llm
- 100m
- 0.1b
- lm
- ai
- dront
---
**VdrontV1** (gpt-like) is the first full model (no experiment) that is neither a test nor an experimental version; it was trained on **FineWeb-Edu and FineWeb-Instruct**.
It can answer simple questions phrased like "What is Python?" or "How many people are on Earth?"; however, it lacks a sense of self ("I") and cannot handle small talk.
According to Dront Model Tester results, it outperformed **VdrontGPT50m-Base2 by a factor of two** and **came close to Supra50m-Base-Exp (which was trained on a significantly larger number of tokens)**.
Examples (**use.py in files**, t 0.4):
You: What is the most popular programming language?
Ai: The most popular programming language is Python. It is a general-purpose programming language that is used to create programs and applications. It is a powerful tool for creating complex programs, and it is also used by many computer programmers.
You: What is a Wi-Fi?
AI: Wi-Fi is a type of wireless network that connects your home to your phone's internet connection. It allows you to connect to a wide variety of devices, including laptops, tablets, and other devices. Wi-Fi is also known as Wi-Fi. You can connect to any device through Wi-Fi, such as the Internet, tablet, or smartphone. The internet is a great way to connect with friends, family, and colleagues. Wi-Fi is also known as "smart" or "smart" or "smart."
You: What is a Site?
AI: A Site is a web site that allows users to create, manage and share their own websites. It is used by many websites, including Facebook, Twitter, Instagram, and YouTube. The site is also used by many websites, such as Google, Yahoo, and others.
| Feature | Detail |
|---------|--------|
| **Architecture** | Transformer-based |
| **Parameters** | +-100,000,000 |
| **Precision** | F32 (Full float32) |
| **Tokenizer** | **GPT-2 Tokenizer** |
| **Format** | Hugging Face |
| **Special Tokens** | `<\|user\|>` `<\|assistant\|>` |
| Dront Model Tester| Supra1.5 Base Exp | VdrontGPT50m-Base2 | VdrontV1 |
|---------|--------|--------|--------|
| Gramm | 13% | 1% | 7% |
| Talk | 38% | 17% | 30% |
| Info | 17% | 6% | 12% |
| Train tokens | 23b | 0.5b | 2.1b|
P.S. next, the Vdront V1.5 (150m) is planned—an expanded version of this model.
P.S. train 21.04 token per param! (2.1b tokens)
:-) |