AI & ML interests

Training small LLMs from scratch, mostly for fun · French NLP · Privacy-preserving clinical NLP

Recent Activity

capybara1337  updated a model 18 days ago
Dokitek/Dokichat-100M
capybara1337  updated a model 18 days ago
Dokitek/Dokichat-100M-Instruct
capybara1337  published a model 18 days ago
Dokitek/Dokichat-100M-Instruct
View all activity

Organization Card

Dokitek

Software and intelligent-systems engineering company based in Toulouse, France, and publisher of clinical software. Here, we share what we learn training small language models from scratch, mostly for fun.

Models

  • Dokichat-100M (from doki doki, the heartbeat in Japanese): a 100M-parameter French language model trained from scratch on a single RTX 4080. On our French benchmark, it matches models three to five times larger, and leads all models under 1B in grammar and vocabulary.
  • Dokichat-100M-Instruct: its chat version, fine-tuned on synthetic French conversations.

Coming soon

  • French synthetic conversations: the datasets we built to fine-tune it.

Our models are small, learning-oriented, and will happily make facts up.

dokitek.com · LinkedIn

datasets 0

None public yet