Update notebook.ipynb

notebook.ipynb

@@ -0,0 +1,174 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# NoTokenLM-MicroGen \u2014 Use Notebook\n",
+    "\n",
+    "This is the notebook to use if you just want to **run one of the MicroGen models**. Every model in the series ships as a real `transformers`-compatible checkpoint, so loading one is a single `from_pretrained(..., trust_remote_code=True)` call \u2014 no manual architecture code, no private repos.\n",
+    "\n",
+    "Because this single repository hosts six different checkpoints side by side (one per subfolder), Hugging Face's default \"Use this model\" widget doesn't represent it correctly \u2014 this notebook is the intended entry point instead.\n",
+    "\n",
+    "## What this notebook does\n",
+    "1. Lets you pick a model from a dropdown\n",
+    "2. Loads it directly from the Hub with `AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True)`\n",
+    "3. Runs text generation on your own prompt\n",
+    "\n",
+    "No token is required (the repo is public) \u2014 a GPU (T4 is enough) will make generation faster, but these models are small enough to run fine on CPU too.\n",
+    "\n",
+    "> These models have no tokenizer \u2014 input and output are raw UTF-8 bytes. `model.generate_bytes(prompt, ...)` handles that for you."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install -q -U transformers torch huggingface_hub\n",
+    "\n",
+    "import torch\n",
+    "from transformers import AutoModelForCausalLM\n",
+    "\n",
+    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
+    "print(f'Device: {device}')\n",
+    "if device == 'cuda':\n",
+    "    print(torch.cuda.get_device_name(0))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Model registry\n",
+    "\n",
+    "Used only to show languages/context length and to offer default prompts \u2014 not needed for loading, that part is fully generic."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "PUBLIC_REPO = \"omurberaisik/NoTokenLM-MicroGen\"\n",
+    "\n",
+    "MODEL_INFO = {\n",
+    "    \"2.5\": dict(languages=[\"en\"], prompts=[\"The \", \"I think that \", \"Once upon a time \"]),\n",
+    "    \"2.6\": dict(languages=[\"en\"], prompts=[\"The \", \"I think that \", \"Once upon a time \"]),\n",
+    "    \"2.7\": dict(languages=[\"en\"], prompts=[\"The \", \"I think that \", \"Once upon a time \"]),\n",
+    "    \"3.5\": dict(languages=[\"en\"], prompts=[\"The \", \"I think that \", \"Once upon a time \"]),\n",
+    "    \"3.6\": dict(languages=[\"en\"], prompts=[\"The \", \"I think that \", \"Once upon a time \"]),\n",
+    "    \"3.7\": dict(languages=[\"en\", \"es\", \"id\", \"it\"],\n",
+    "                prompts=[(\"en\", \"The \"), (\"en\", \"I think that \"),\n",
+    "                         (\"es\", \"El \"), (\"id\", \"Saya \"), (\"it\", \"Il \")]),\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Pick a model and load it"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "SELECTED_VERSION = \"3.6\"  #@param [\"2.5\", \"2.6\", \"2.7\", \"3.5\", \"3.6\", \"3.7\"]\n",
+    "\n",
+    "model = AutoModelForCausalLM.from_pretrained(\n",
+    "    PUBLIC_REPO,\n",
+    "    subfolder=SELECTED_VERSION,\n",
+    "    trust_remote_code=True,\n",
+    ")\n",
+    "model.to(device)\n",
+    "model.eval()\n",
+    "\n",
+    "n_params = sum(p.numel() for p in model.parameters())\n",
+    "info = MODEL_INFO[SELECTED_VERSION]\n",
+    "print(f\"Loaded NoTokenLM-MicroGen-{SELECTED_VERSION}: {n_params:,} parameters\")\n",
+    "print(f\"Languages: {info['languages']}\")\n",
+    "print(f\"Context length: {model.config.max_len} bytes\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generate text\n",
+    "\n",
+    "Edit `MY_PROMPT` and re-run. `temperature` controls randomness \u2014 lower is more predictable, higher is more chaotic; these models were evaluated at 0.5."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MY_PROMPT = \"The \"  #@param {type:\"string\"}\n",
+    "TEMPERATURE = 0.5  #@param {type:\"number\"}\n",
+    "N_NEW_BYTES = 200  #@param {type:\"integer\"}\n",
+    "\n",
+    "out = model.generate_bytes(MY_PROMPT, n_new_bytes=N_NEW_BYTES, temperature=TEMPERATURE)\n",
+    "print(f\"Model:  NoTokenLM-MicroGen-{SELECTED_VERSION} ({n_params:,} params)\")\n",
+    "print(f\"Prompt: {MY_PROMPT!r}\")\n",
+    "print(f\"Output: {out!r}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Try the model's default prompt set\n",
+    "\n",
+    "Each model has a small built-in prompt set (the same ones used for the README examples). Run this to see how the currently loaded model does on all of them at once."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for item in info[\"prompts\"]:\n",
+    "    if isinstance(item, tuple):\n",
+    "        lang, prompt = item\n",
+    "        tag = f\"[{lang}] \"\n",
+    "    else:\n",
+    "        prompt = item\n",
+    "        tag = \"\"\n",
+    "    out = model.generate_bytes(prompt, n_new_bytes=200, temperature=0.5)\n",
+    "    print(f\"{tag}{prompt!r} -> {out!r}\\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "\n",
+    "### A note on what to expect\n",
+    "\n",
+    "These are small, from-scratch, byte-level models with no instruction tuning. From 2.7 onward they tend to produce grammatically plausible continuations, but they are not chat assistants and will not reliably stay on topic for long. See the [model card](https://huggingface.co/omurberaisik/NoTokenLM-MicroGen) for a full, honest rundown of each version's limitations."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  },
+  "accelerator": "GPU"
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}