Text Generation
Transformers
Safetensors
English
gpt_neo
gpt-neo
tinystories
tiny
embedded
esp32
microcontroller
chat
small-talk
Instructions to use TheREZOR/TinyTalk with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use TheREZOR/TinyTalk with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="TheREZOR/TinyTalk")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("TheREZOR/TinyTalk") model = AutoModelForCausalLM.from_pretrained("TheREZOR/TinyTalk") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use TheREZOR/TinyTalk with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "TheREZOR/TinyTalk" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheREZOR/TinyTalk", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/TheREZOR/TinyTalk
- SGLang
How to use TheREZOR/TinyTalk with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "TheREZOR/TinyTalk" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheREZOR/TinyTalk", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "TheREZOR/TinyTalk" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "TheREZOR/TinyTalk", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use TheREZOR/TinyTalk with Docker Model Runner:
docker model run hf.co/TheREZOR/TinyTalk
Third-party components and attributions
The source code in this repository is MIT-licensed (see LICENSE). The embedded model and tokenizer artifacts derive from the following works:
Base model — TinyStories-Instruct-3M
- Ronen Eldan & Yuanzhi Li, TinyStories: How Small Can Language Models Be and Still Speak Coherent English? (arXiv:2305.07759).
- Weights: https://huggingface.co/roneneldan/TinyStories-Instruct-3M — published without an explicit license tag. The companion TinyStories dataset is CDLA-Sharing-1.0, which places no restrictions on results (e.g. trained models). If the licensing of the base weights matters for your use case, contact the model author.
Fine-tuning data
- SODA (https://huggingface.co/datasets/allenai/soda), CC BY 4.0.
Kim et al., SODA: Million-scale Dialogue Distillation with Social
Commonsense Contextualization (arXiv:2212.10465). The chat fine-tune
embedded in
model_data.cppwas trained on a filtered subset. - TinyStoriesInstruct (https://huggingface.co/datasets/roneneldan/TinyStoriesInstruct), CDLA-Sharing-1.0 (per the TinyStories dataset family).
Tokenizer
- GPT-2 byte-level BPE vocabulary and merges (
vocab.json/merges.txt), from OpenAI's GPT-2 release (https://github.com/openai/gpt-2), MIT (Modified MIT License, Copyright (c) 2019 OpenAI). Embedded here in pruned, re-encoded form insidetok_data.cpp.
Alternative model (not embedded by default)
- Maykeye/TinyLLama-v0 (https://huggingface.co/Maykeye/TinyLLama-v0),
Apache-2.0. Supported by the engine via
tools/convert_tinyllama_v0.py.
Vendored keyboard driver (main/keyboard/)
- Ported from M5Cardputer v1.1.1
(https://github.com/m5stack/M5Cardputer), MIT,
Copyright (c) 2025 M5Stack Technology CO LTD. Arduino GPIO/interrupt
calls were replaced with ESP-IDF
driver/gpioequivalents. - Includes M5Stack's adaptation of the Adafruit TCA8418 keypad driver (https://github.com/adafruit/Adafruit_TCA8418), BSD, Copyright (c) Limor Fried (Adafruit Industries).
Acknowledgements
- The inference engine follows the structure of Andrej Karpathy's llama2.c (https://github.com/karpathy/llama2.c), MIT. The code here is an independent implementation extended with Q4_0 quantization, a GPT-Neo forward path, int8 KV cache, and a flash-walking tokenizer.