Upload folder using huggingface_hub

ANDREA-12M-best.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f67dc5f259b51e9cade5db845b71cdbbe15313cafb42914a5ba630278adc2f39
+size 153363464

ANDREA-12M.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:927baf98b44cdba986f69079a259cc3b8019eb4fab210bd5bcaad703a6d50626
+size 153363464

README.md

@@ -1,3 +1,120 @@
----
-license: agpl-3.0
----
+---
+license: agpl-3.0
+language:
+- en
+tags:
+- text-generation
+- smol
+- permacomputer
+- bandit-curriculum
+pipeline_tag: text-generation
+---
+
+# ANDREA-12M
+
+**A**utonomous **N**eural **D**ata **R**ecipe for **E**ducation and **A**gency
+
+A 12.8M parameter language model grown on a single RTX 4090 using a bandit-controlled curriculum.
+Part of the permacomputer project — open source, open data, open weights.
+
+## Model Details
+
+| Property | Value |
+|----------|-------|
+| Parameters | 12.8M |
+| Architecture | Transformer decoder, 384d/12h/6L |
+| Embedding dim | 384 |
+| Heads | 12 |
+| Layers | 6 |
+| Context | 1024 tokens |
+| Tokenizer | Harris morpheme (2048 segments, 2305 vocab) |
+| Training steps | 43,587 |
+| Final SMMA loss | 2.0 |
+| Best single-step loss | 0.21 |
+| Training time | ~72 hours |
+| Hardware | Single NVIDIA RTX 4090 (24GB VRAM, 1.4GB used) |
+| CUDA engine | microgpt_cuda.cu (custom, FP32) |
+| Born | 2026-03-21 12:53 UTC / 08:53 EST |
+| License | AGPL-3.0 |
+
+## Files
+
+| File | Step | Description |
+|------|------|-------------|
+| `ANDREA-12M.bin` | 43,587 | Final checkpoint (SMMA 2.0) |
+| `ANDREA-12M-best.bin` | 42,300 | Best checkpoint (lowest loss during training) |
+| `harris_segments.json` | — | Harris tokenizer segments (required for inference and fine-tuning) |
+
+### Checkpoint format
+
+Binary, little-endian: `[int32 step][int32 n_params][n_params × float32 weights][n_params × float32 m][n_params × float32 v]`
+
+- **Weights**: model parameters (12.8M floats, ~49MB)
+- **m**: Adam first moment (same size)
+- **v**: Adam second moment (same size)
+- Total: ~147MB per checkpoint
+
+Use either checkpoint to resume fine-tuning (weights + optimizer state preserved)
+or extract weights only for inference (first `n_params` floats after the 8-byte header).
+
+## Training Data
+
+Trained on a curated mix of open conversational and educational data:
+
+- **NousResearch/Hermes-3-Dataset** (general, creative, roleplay) — 590K conversations
+- **Dictionary** — 88K word definitions distilled from Hermes 3 8B
+- **Gutenberg** — public domain literature (Project Gutenberg)
+- Additional: chat, smoltalk, oasst, dolly, IRC, repo-docs
+
+Data mix controlled by a UCB1 multi-armed bandit with dice-based phase control.
+The bandit dynamically adjusts source weights during training based on per-source
+loss trajectories. Full curriculum specification in the white paper.
+
+## Training Recipe
+
+- Harris morpheme tokenizer (2048 segments)
+- Cosine LR schedule with warm restart at step 25K (0.0004 peak)
+- Phase-based bandit: 2 focus arms, 1d3 dice, source floors
+- Checkpoints every 100 steps, SIGTERM-safe
+- Per-source reward attribution, epoch penalty, coverage tracking
+
+## Capabilities
+
+ANDREA-12M learns patterns, not facts. At 12.8M parameters it produces:
+- Correct Q&A turn structure (`> question / < answer`)
+- Definition-style responses
+- Multi-sentence outputs with plausible grammar
+- Instruction-following scaffolding ("explain", "define", "describe")
+
+It does NOT produce factually accurate content — it's a pattern machine.
+Factual accuracy requires scaling to ANDREA-120M (planned).
+
+## Usage
+
+```python
+# Inference via microgpt
+from microgpt import load_model, generate_fast
+
+model = load_model('ANDREA-12M.json')
+results = generate_fast(model['state_dict'], model['uchars'], model['bos'],
+                        384, 12, 6, 1024, prefix='> what is an apple? / <')
+print(results[0][0])
+```
+
+## White Paper
+
+See `papers/ANDREA.md` in the [uncloseai-cli repository](https://git.unturf.com/engineering/unturf/uncloseai-cli).
+
+## Citation
+
+```
+ANDREA: Autonomous Neural Data Recipe for Education and Agency
+TimeHexOn, foxhop, russell@unturf
+March 2026, permacomputer.com
+```
+
+## License
+
+AGPL-3.0. Code outlasts authors. Infrastructure outlasts builders.
+
+● ○

harris_segments.json

@@ -0,0 +1 @@
+["the", "to", "you", "and", "of", "it", "is", "in", "that", "for", "are", "what", "have", "can", "with", "on", "do", "be", "was", "this", "but", "not", "me", "as", "my", "or", "your", "at", "we", "like", "so", "there", "no", "if", "from", "he", "yes", "how", "about", "all", "one", "an", "will", "by", "just", "some", "which", "would", "don", "good", "well", "they", "her", "time", "up", "his", "when", "know", "ted", "see", "The", "get", "out", "here", "more", "think", "had", "any", "she", "ll", "need", "now", "very", "som", "right", "go", "re", "could", "want", "were", "has", "then", "should", "make", "been", "use", "them", "oh", "ting", "am", "help", "did", "sure", "please", "inte", "really", "ve", "too", "him", "new", "who", "other", "great", "work", "two", "also", "our", "their", "going", "much", "only", "thanks", "first", "ok", "take", "tion", "why", "does", "way", "into", "de", "than", "thr", "let", "try", "red", "name", "yeah", "ething", "where", "ever", "file", "said", "got", "day", "people", "look", "thank", "after", "long", "its", "back", "ter", "spec", "before", "these", "most", "using", "must", "many", "python", "still", "find", "tly", "It", "lit", "may", "over", "anything", "diffe", "tell", "come", "morning", "things", "pas", "mean", "tle", "code", "give", "sorry", "bash", "say", "data", "done", "problem", "ding", "check", "wit", "ough", "home", "ion", "hello", "run", "cmd", "ned", "He", "answer", "com", "la", "est", "py", "next", "list", "last", "lot", "yet", "ring", "nce", "same", "ation", "man", "such", "though", "year", "today", "set", "used", "being", "made", "never", "wing", "every", "us", "world", "again", "open", "even", "room", "old", "line", "better", "else", "ask", "rent", "might", "doing", "thought", "class", "best", "number", "job", "en", "course", "because", "star", "que", "each", "https", "sounds", "love", "And", "kind", "thing", "tes", "inform", "both", "hope", "follo", "git", "ect", "nice", "maybe", "around", "This", "part", "question", "call", "sentence", "down", "always", "10", "off", "those", "idea", "salt", "found", "un", "place", "test", "between", "welcome", "another", "doesn", "own", "years", "hout", "ties", "didn", "few", "keep", "service", "main", "enough", "point", "while", "direc", "looking", "tant", "proba", "string", "req", "But", "In", "type", "read", "show", "user", "seems", "house", "isn", "You", "corr", "away", "art", "able", "hi", "quite", "three", "small", "opt", "running", "true", "coun", "top", "crea", "night", "conc", "What", "example", "tomorrow", "okay", "went", "pretty", "school", "gener", "sed", "book", "table", "far", "unde", "function", "hap", "el", "order", "days", "food", "working", "sing", "fine", "ded", "left", "write", "sir", "tran", "tor", "popular", "won", "app", "car", "start", "possible", "lly", "soon", "provide", "put", "med", "mr", "error", "nal", "nothing", "prog", "ning", "bly", "young", "pay", "real", "end", "five", "free", "If", "per", "rea", "change", "compl", "already", "rest", "wait", "obj", "docs", "hard", "company", "meet", "hey", "files", "king", "since", "ove", "wrong", "tions", "es", "least", "water", "big", "given", "compu", "full", "life", "feel", "server", "clock", "city", "size", "appr", "came", "easy", "led", "states", "ions", "value", "toge", "ther", "along", "often", "rather", "under", "hing", "impor", "second", "glad", "instr", "buy", "wri", "system", "afte", "mult", "minu", "ral", "happy", "afraid", "men", "ely", "ify", "light", "12", "instead", "succ", "han", "assis", "ident", "develo", "mind", "words", "ything", "called", "ready", "questions", "history", "until", "install", "ten", "almost", "case", "leave", "20", "eone", "actua", "money", "high", "side", "report", "envir", "four", "uest", "ah", "ai", "yourself", "rence", "build", "usually", "She", "add", "wonde", "moment", "trying", "coffee", "hours", "ific", "upon", "form", "lis", "include", "getting", "state", "key", "once", "hand", "ind", "cen", "based", "sum", "short", "seen", "later", "cool", "air", "si", "defin", "known", "There", "nginx", "late", "thinking", "exac", "means", "etimes", "looks", "ges", "saw", "How", "bit", "phone", "default", "clea", "por", "response", "simple", "resting", "api", "friends", "fox", "price", "play", "fas", "makes", "ement", "white", "bad", "having", "friend", "Mr", "turn", "live", "No", "clas", "either", "taking", "less", "stop", "rnoon", "explain", "team", "When", "version", "return", "foo", "15", "week", "whatever", "etc", "head", "support", "however", "lat", "page", "tal", "myself", "hig", "quic", "terr", "card", "create", "import", "source", "took", "ope", "miss", "building", "game", "office", "single", "certainly", "reason", "near", "low", "music", "bac", "So", "person", "hot", "pattern", "instal", "def", "fix", "worked", "whole", "print", "They", "looked", "anyone", "package", "hear", "expec", "matter", "text", "stuff", "ses", "window", "care", "His", "told", "bring", "public", "business", "believe", "haven", "We", "send", "seem", "experience", "talking", "current", "sleep", "coming", "works", "remember", "ters", "ries", "det", "ial", "eyes", "hin", "contai", "moun", "des", "onment", "aren", "para", "tem", "module", "path", "movie", "whose", "refe", "wish", "email", "incr", "wow", "half", "making", "pass", "rstand", "hour", "As", "range", "sense", "wanted", "veget", "Python", "beautiful", "date", "common", "kly", "language", "rly", "message", "cal", "speak", "hundred", "um", "watch", "exis", "capabili", "traffic", "commun", "guess", "github", "000", "ago", "hed", "pull", "master", "shall", "content", "difficult", "himself", "stan", "future", "students", "sent", "indeed", "inside", "section", "At", "move", "script", "black", "feeling", "connec", "excuse", "useful", "mer", "sort", "percussion", "ente", "None", "blue", "ive", "library", "english", "fact", "die", "wife", "cold", "party", "json", "color", "For", "woman", "yone", "project", "body", "general", "plea", "hotel", "rnet", "rful", "month", "among", "couldn", "father", "sugges", "early", "30", "become", "tment", "chinese", "step", "ground", "se", "hola", "cause", "town", "hmm", "favorite", "helpful", "family", "con", "mother", "ements", "model", "tten", "ional", "ist", "perfor", "born", "books", "speed", "including", "exercise", "hold", "aut", "taken", "meeting", "website", "custo", "restaurant", "street", "sound", "ing", "That", "whether", "process", "learn", "nee", "others", "extra", "11", "ice", "ete", "ible", "natu", "itely", "wed", "ping", "una", "datab", "pos", "00", "clear", "stay", "lines", "trouble", "girl", "ling", "ways", "ideas", "tive", "problems", "lear", "break", "times", "children", "ming", "bus", "account", "news", "paper", "dear", "uh", "address", "nea", "applic", "te", "weeks", "http", "hash", "chec", "publ", "ones", "Then", "non", "during", "config", "depen", "weather", "42", "fun", "services", "avoid", "doctor", "seve", "itself", "thirty", "absolutely", "save", "ally", "apply", "living", "spea", "against", "guy", "dead", "capi", "ntly", "rity", "added", "wonder", "command", "abc", "cat", "ran", "log", "poin", "post", "trip", "available", "im", "store", "fast", "drive", "suf", "american", "changes", "yed", "pick", "remove", "says", "50", "needs", "disti", "mary", "18", "eat", "web", "pment", "structure", "fire", "enjoy", "ration", "errors", "Is", "tried", "original", "united", "large", "door", "tain", "ram", "land", "zone", "issue", "suppose", "index", "ner", "mance", "finally", "memory", "south", "online", "packa", "dark", "christmas", "total", "update", "son", "les", "john", "expensive", "lots", "ers", "manager", "face", "except", "sor", "box", "via", "weekend", "lost", "copy", "improve", "js", "google", "minute", "cache", "ical", "cial", "shell", "search", "evening", "local", "kno", "began", "flowers", "scripts", "tory", "ces", "space", "supe", "Use", "cut", "watc", "past", "network", "therefore", "strong", "iple", "planning", "machine", "13", "talk", "sls", "dow", "Why", "bed", "ms", "eur", "ument", "answe", "wind", "nment", "numbers", "systems", "ansible", "hate", "safe", "further", "follow", "offer", "video", "deplo", "tools", "gave", "six", "china", "mine", "knew", "birthday", "effec", "ners", "conside", "trees", "brother", "walk", "obser", "issues", "certain", "gran", "travel", "lev", "chat", "madam", "mom", "besides", "tall", "tel", "shows", "opp", "java", "characters", "kept", "ked", "tree", "dev", "power", "sh", "setup", "API", "sports", "ate", "due", "orga", "syntax", "Well", "secu", "tance", "deep", "pi", "My", "dont", "various", "keys", "goo", "To", "engine", "heard", "heavy", "www", "tation", "shop", "major", "contain", "sig", "fish", "addit", "On", "tickets", "cannot", "output", "switch", "ved", "das", "consider", "org", "places", "front", "soft", "image", "16", "season", "ahead", "age", "bjs", "echo", "wasn", "require", "ies", "del", "link", "allo", "pip", "cities", "nse", "close", "catch", "edit", "pens", "results", "sign", "docker", "law", "forgot", "drink", "nor", "commit", "describe", "ica", "values", "rview", "shopping", "situation", "sword", "25", "Force", "women", "includes", "ifi", "perhaps", "ssh", "word", "across", "clean", "story", "bought", "logs", "particular", "fall", "train", "root", "speech", "route", "ase", "method", "types", "cross", "brea", "alone", "format", "calling", "returns", "note", "Mrs", "ride", "practi", "tool", "All", "baby", "cing", "exist", "takes", "qu", "em", "nct", "surpri", "perc", "Oh", "uses", "unless", "lunch", "style", "gift", "configu", "studying", "32", "worry", "prefer", "uct", "decision", "md", "flight", "pue", "famous", "choose", "rested", "ture", "mill", "phi", "feature", "array", "arrived", "javascript", "remote", "america", "healthy", "14", "scie", "natio", "fellow", "env", "turned", "begin", "grep", "points", "input", "url", "resources", "role", "group", "updated", "int", "tonight", "languages", "branch", "profes", "purpose", "visit", "simi", "lar", "fill", "dogs", "military", "lib", "access", "math", "pen", "result", "ving", "doubt", "boy", "hands", "spend", "wants", "der", "saying", "level", "asking", "assist", "corner", "dollars", "manner", "God", "freque", "above", "notice", "dry", "lian", "With", "bin", "final", "fresh", "easily", "college", "prevent", "load", "defence", "seven", "fil", "True", "appoin", "green", "ess", "members", "tired", "accept", "dle", "vars", "deal", "changed", "secre", "embed", "letter", "driving", "South", "provides", "apple", "pastebin", "settings", "fair", "shouldn", "testing", "messa", "recommend", "france", "study", "channel", "repo", "shor", "images", "finish", "propo", "produce", "develop", "area", "quick", "const", "essful", "mater", "requir", "charge", "seemed", "ich", "satu", "bye", "youtube", "attend", "numpy", "ima", "statement", "gone", "lived", "health", "holiday", "variable", "alright", "yea", "choice", "eu", "field", "wouldn", "helps", "windows", "names", "third", "shoes", "present", "asked", "physi", "twenty", "broken", "uests", "mode", "larg", "context", "hes", "none", "24", "origin", "apartment", "details", "related", "finished", "self", "host", "proper", "reser", "node", "matc", "dinner", "required", "pened", "condition", "square", "wai", "United", "cove", "client", "daughter", "shoul", "22", "chance", "subject", "surprise", "bob", "truth", "recommen", "stand", "Defe", "built", "ease", "perfect", "forty", "null", "amount", "These", "nced", "usr", "events", "aggr", "gets", "dirty", "column", "position", "earth", "pulled", "north", "curren", "produc", "quality", "armed", "exchange", "simply", "equ", "training", "sev", "loca", "personal", "ncy", "100", "sea", "cally", "san", "adap", "depar", "bread", "remain", "meal", "compr", "round", "excellent", "st", "calls", "CLI", "answers", "funny", "repor", "port", "ert", "lol", "guys", "wear", "eciate", "weight", "anyway", "valid", "formula", "methods", "rnational", "Not", "bas", "release", "hair", "rooms", "failed", "slo", "seeing", "child", "credit", "orden", "figure", "virt", "influe", "unturf", "unit", "task", "attent", "ially", "unique", "river", "defined", "alte", "plan", "ations", "caught", "Nat", "plate", "herself", "clothes", "mobile", "giving", "ball", "leaves", "id", "quiet", "didnt", "friday", "delive", "An", "serve", "orde", "platform", "Now", "brought", "notes", "ging", "adm", "religion", "Int", "Let", "disap", "hit", "appar", "comment", "icine", "sky", "zero", "adve", "bank", "jane", "effort", "cost", "las", "recen", "prin", "eating", "managed", "fried", "jogging", "swimming", "york", "steps", "market", "hardly", "lady", "park", "zones", "parts", "lice", "replace", "force", "alive", "exti", "comes", "src", "global", "tests", "rate", "active", "london", "svc", "elements", "spaces", "double", "till", "stopped", "mum", "mers", "States", "forward", "refl", "dress", "micr", "sof", "Before", "tries", "recogn", "ramming", "drop", "skills", "telling", "reached", "concl", "nable", "human", "reduce", "limit", "finding", "nature", "ita", "Reg", "conf", "bootstrap", "wledge", "argument", "worth", "native", "india", "commi", "feat", "sale", "Ins", "compati", "ble", "searc", "months", "activi", "txt", "elec", "rance", "kin", "below", "pillar", "instance", "integra", "features", "base", "reci", "dispo", "grat", "screen", "feelings", "worried", "Thanks", "lik", "smith", "necessary", "nume", "tu", "lie", "pure", "els", "accep", "helping", "reasons", "regular", "road", "yml", "ables", "normal", "commission", "goes", "tary", "bike", "sify", "plane", "mrs", "yel", "bpa", "letters", "itera", "moto", "greater", "plans", "acqu", "er", "held", "consis", "song", "satis", "domain", "dict", "impress", "univer", "carry", "gato", "peace", "exact", "thy", "solve", "floor", "nobody", "espec", "implemen", "join", "bright", "Opt", "hos", "chicken", "rice", "yment", "80", "war", "pages", "tent", "et", "stores", "fixed", "ewhere", "termi", "research", "Like", "ker", "provided", "conditio", "ress", "rmah", "ished", "milk", "huh", "west", "AND", "cases", "Miss", "immediately", "ectly", "cloud", "minion", "supposed", "guide", "perfec", "whom", "versions", "trave", "boot", "secret", "beyond", "unable", "cdigo", "mais", "receive", "previous", "dad", "28", "fully", "17", "restart", "ent", "rwise", "bar", "ligh", "theory", "rain", "block", "vil", "ao", "compet", "lead", "thousand", "ish", "share", "matters", "forth", "blood", "nation", "moved", "although", "fear", "proxy", "auto", "bug", "japanese", "cards", "debug", "19", "subway", "smaller", "hammer", "repr", "Makefile", "suppor", "kidding", "busy", "runs", "inves", "myapp", "eye", "product", "sun", "sion", "omat", "view", "fifty", "items", "solution", "rust", "met", "anywhere", "maxi", "taxi", "yesterday", "brown", "pool", "california", "sessions", "review", "represen", "tat", "location", "di", "adding", "fran", "actions", "itude", "By", "received", "pity", "operations", "fishing", "bag", "teach", "pyramid", "trade", "Does", "kernel", "sad", "length", "reaso", "fee", "combi", "coins", "hest", "gas", "agree", "average", "serious", "quar", "Manag", "gment", "strange", "weird", "regex", "lights", "falls", "eve", "sat", "cd", "nized", "tables", "ma", "define", "lock", "showing", "design", "kfast", "Are", "als", "machines", "python3", "match", "fifteen", "ad", "equal", "ity", "driver", "icate", "fan", "cisco", "dishes", "enter", "unr", "econo", "mic", "bringing", "exte", "detail", "arian", "agents", "areas", "god", "Of", "wine", "arguments", "2023", "solutions", "jour", "lists", "SDK", "tom", "molecular", "efficie", "products", "grow", "ified", "Test", "norma", "circumstances", "custom", "removed", "deligh", "logic", "count", "movies", "palabras", "cream", "complex", "site", "yours", "heh", "nights", "tag", "sligh", "civ", "tmp", "friendly", "db", "conve", "pregunta", "ybody", "hang", "Yes", "und", "responsible", "ders", "document", "impro", "depends", "Add", "period", "gracias", "airport", "walking", "italy", "venv", "purs", "damn", "websocket", "updates", "station", "wash", "staying", "replied", "wer", "patie", "bility", "conti", "requires", "haha", "chocolate", "impossible", "schedule", "integer", "furniture", "japan", "painting", "push", "gitlab", "express", "bunch", "reports", "records", "See", "parties", "jobs", "SnoopJ", "rode", "200", "lose", "ently", "couple", "abroad", "typically", "crime", "employees", "tips", "according", "singing", "bill", "tour", "capability", "wedding", "stream", "los", "entirely", "ection", "sailo", "rjerry", "ary", "excep", "court", "killed", "term", "applied", "othe", "suggest", "meaning", "amu", "William", "rich", "Req", "__URI", "volume", "ship", "async", "blocks", "rad", "lation", "processes", "lord", "wearing", "pair", "sweet", "grew", "bullet", "super", "camera", "query", "played", "lion", "php", "thats", "links", "func", "excited", "david", "direct", "grown", "songs", "selec", "essfully", "heart", "je", "tokens", "sales", "payment", "forms", "fruit", "dog", "desk", "smartos", "private", "curl", "construc", "status", "amb", "miles", "died", "ci", "oughout", "stac", "regis", "musi", "gives", "il", "vation", "Merge", "slate", "poetry", "museum", "sudde", "OS", "tv", "stupid", "target", "pressure", "calcu", "wild", "deposit", "tral", "ruby", "eight", "pero", "dencies", "filter", "40", "logging", "ser", "hungry", "marvel", "estoy", "win", "rday", "picture", "dir", "147", "salad", "2x", "fees", "map", "cheap", "Por", "OF", "assig", "Gover", "sets", "carried", "employed", "Prog", "fxhp", "examples", "mall", "British", "produced", "wable", "html", "css", "wheel", "ates", "ideal", "touc", "cir", "plus", "finan", "erned", "fields", "primary", "title", "affect", "expand", "luck", "sudo", "confused", "li", "lay", "2026", "deploy", "kup", "session", "redu", "extract", "evide", "former", "appea", "grand", "canyon", "mount", "reads", "likes", "dly", "One", "control", "podras", "colu", "False", "themselves", "pandas", "lambda", "vault", "mark", "anim", "maps", "behavior", "ications", "toy", "positions", "leading", "changing", "honest", "moon", "console", "fn"]