Blog | TinyMemoryLM

2026-03-21 Hardware Madness

I Flashed The Matrix VBIOS And Now I Train Models All Day

Yesterday I wrote about how AI failed to help me find the InfoROM for VBIOS flashing. It could not do it. I had to do it myself. I spent the night reading forums. Reading modding guides. Reading warnings that I should not be doing this.

2026-03-20 Hardware Fails

I Asked AI To Mod My VBIOS And It Choked At Step Four

TI have a RTX 5090 OC LC. It runs at 600W. I wanted 700W. Not because I need it. Not because it is safe. Because I can. Because the model said it could help. Because I have learned nothing from previous AI disappointments. The plan was simple. Four steps. Extract the VBIOS. Find the wattage limit. Modify it. Flash it back. How hard could it be? The answer is very hard. The AI failed at step four. It could not figure out how to get the InfoROM. It tried for an hour. It gave up. I am still at 600W.

2026-03-20 Not AI Related

I Watched Project Hail Mary And Forgot About My NaN Loss

This blog is usually about AI. About training models. About GPUs that cost more than my education. About loss curves that go down and then suddenly become NaN and destroy my will to live. Today I am writing about something else. Something that made me forget about my 261 hour training run. Something that made me feel joy for the first time in weeks. I watched Project Hail Mary

2026-03-19 Training Disasters

I Woke Up To NaN And Now I Am Dead Inside<

I went to sleep happy. The loss was going down. The gradients were stable. The GPU was humming at 60C like a contented cat. I dreamed of completion. I dreamed of a finished Sonnet model. I dreamed of sleep that was not interrupted by thoughts of learning rate schedules.

2026-03-18 Confessions

I Tried Opus 4.6 And Now Everything Else Feels Broken<

I have spent the last month writing blogs about how AI models are lazy. How they are too expensive. How they form unhealthy attachments. How they cannot finish a task without asking for permission. I stand by most of that. Opus 4.6 changed my mind about the laziness part.

2026-03-17 Training Pain

261 Hours For A 300M Model And I Have Every Optimization

I have every optimization under the sun enabled. Native NVFP4 quantization. Torch.compile with max auto tune and cudagraphs. No gradient accumulation. Maximum batch size. My GPU is locked at 600W. My clocks are fixed. My cooling is liquid. Everything is perfect.

2026-03-16 Hardware

I Locked My GPU Clocks And Now It Runs Forever.html

I have an RTX 5090 OC LC edition. Liquid cooled. Overclocked out of the box. It is the kind of card that makes people ask uncomfortable questions about my financial decisions. I have no good answers.

2026-03-15 Dev Struggles

I Built A Training UI And Then Unsloth Laughed

I decided to build a training interface. A backend. A way for people to fine-tune models without touching a terminal. It sounded simple. It was not simple. It is currently the hardest thing I have ever done and I once tried to explain transformers to my cat.

2026-03-14 Unpopular Opinions

Every AI Model Is Lazy And I Have The Screenshots

I have asked many AI models to build things. Fully implement a task. Write the code. Run the tests. Fix the errors. Ship it. Not one of them has done this without me holding their hand through every single step.

2026-03-13 Unpopular Opinions

OpenAI Did A Good Thing And Everyone Is Mad About It

I have an unpopular opinion and I am ready to be yelled at for it. OpenAI removing GPT-4o was the right decision. People are furious about this. They are grieving. They are writing petitions. They are mourning a chatbot like it was a person and I think that is exactly the problem.

2026-03-12 Projects

I Built A Tool That Snitches On AI Models

Every AI model has an accent. Not a literal accent because they do not have mouths. A writing accent. A way of forming sentences that gives them away like a fingerprint at a crime scene.

2026-03-11 Industry Rants

I Spent $40 And Got A Greeting

I used to spend money on AI APIs for testing. Now I spend money on AI APIs and immediately regret every life choice that led me to that moment. The prices have gotten out of hand and I need to talk about it before I have a breakdown in the middle of a terminal window.

2026-03-10 Model Releases

I Released A Model And Nobody Clapped (Fair)

I released a model yesterday. TMLM-Haiku-1. It is small. Surprisingly small. It also somehow speaks which I consider a major achievement given my training budget and general approach to machine learning which can best be described as throwing things at a GPU until something sticks.

2026-03-9/span> AI Thoughts

Distilling Closed Models Until They Forget They Were Closed

I have been thinking about model distillation lately. Not the academic kind with proper methodology and peer review. The hobbyist kind where someone spends their own money on API credits, LoRA fine-tunes a small model, and releases it for free because they can.

2026-03-8 Tooling

I Finally Switched Terminals (And My Ego Is Healing)

I used the default macOS terminal for years. Not because I loved it. I kept it because change is scary and I am deeply committed to mediocrity. Then I tried Warp and realized I have been suffering through a text-based interface that treats me like an enemy.

I Finally Switched Terminals (And My Ego Is Healing).html 2026-03-7 Scaling Laws

The Chinchilla Effect: Why Tiny Models Have to Be Picky

The Chinchilla paper told us something elegant. For compute optimal training, aim for roughly twenty tokens per parameter. A 70 billion parameter model wants 1.4 trillion tokens. A 1 million parameter model wants 20 million tokens. The math is clean. The implication is messy.

2026-03-6 Compute Philosophy

The Training Time Compute Trap

There is a moment in every AI project when someone says "maybe we just need more compute." It sounds reasonable. It sounds scientific. It sounds like the kind of thing that gets budgets approved and GPUs ordered. Then you wake up three weeks later, your electricity bill has achieved sentience, and your model still thinks "python" refers exclusively to snakes.

2026-03-5 Model Experiments

Teaching AI to Regret: The Backspace Token Theory

Humans backtrack. We type "thr" and realize we meant "the" and we fix it. We type "tje" and we laugh at our own fingers and we correct it. Large language models do not do this. They commit to every token like it is a binding legal contract. I started wondering what would happen if we gave them an out. What if we added a backspace token to the vocabulary?

2026-03-4 Industry Chaos

The Irony Cloud: When AI Downtime Meets Timing

Anthropic is down. Of course it is down. The universe has a sense of humor and apparently that humor is "make the ethical AI company unreachable right after they make a big ethical statement.

2026-03-3 Industry Rants

The Bloatening: When AI Companies Forgot About the Little Guy

I used to get excited about model releases. A new tiny model would drop and I would immediately try to run it on my laptop that sounds like a jet engine. Now I scroll through announcements and see numbers that require a data center just to pronounce

2026-03-2 Budget

Why Does My AI Think Math Is a Fishing Trip?

I asked my model to solve a simple integral. It responded with a detailed description of trout migration patterns. This is not the answer I was looking for, though I admit the trout explanation was surprisingly well-structured. Training a small language model is like teaching a very enthusiastic puppy. It wants to please you.

2026-02-29 Budget

Training Models on a Ramen Budget

How to train a transformer when your GPU bill looks like a phone number. Tips, tricks, and questionable life choices from someone who learned about electricity costs the hard way.

2026-02-22 Vibecoding

One Year of Vibecoding and Other Questionable Life Choices

You start vibecoding because someone told you it feels like magic. You imagine floating through code. Reality does not care about your imagination.

2026-02-26 Hot Takes

OpenClaw: The Most Overhyped Bot Since Sliced Bread

OpenClaw, formerly Clawdbot, formerly Moltbot, has now accumulated more GitHub stars than the Linux kernel. Let that sink in.

2026-02-27 Scaling

The Scaling Wall And Other Things I Yelled At

Someone told me we can just keep making models bigger. They said compute will solve everything. They lied. Or they hoped. Or they had investors to please.

2026-02-20 Reality Check

Your AI Agent is Lying Behind Your Back

You know the feeling. You type a prompt. The text streams. The terminal says success. I am here to tell you that you are being played.

2026-02-25 AI Theater

Anthropic's Distillation Drama: A Masterclass in Projection

So Anthropic published a blog post. Big surprise. The title alone could power a small city.

2026-02-19 Architecture

The Wasted Precision of the Output Layer

We spend a lot of time optimizing attention mechanisms. We prune weights. We quantize activations. Yet there is a massive inefficiency sitting right at the very end of the network.

2026-02-21 GPU Tears

My Baby Model Takes Forever to Grow Up

You start with hope. A tiny transformer. A few million parameters. You think, how long could this possibly take? I am here to ruin your optimism.

2026-02-23 Memory Hacks

External Memory Modules: Because My Model Has Commitment Issues

You know what takes forever? Training a transformer. You know what takes less forever? Training a tiny thing that just remembers stuff.

2026-02-24 Hot Takes

The Goalpost Has Legs: Why AGI Keeps Running Away

Imagine handing Claude Opus 4.6 to someone from 2004. They would think you summoned a minor deity. Our collective response? A polite nod.

2026-02-29 Tiny Wins

Words, Words, Words: My Model Learned to Ramble

My model has achieved something truly special. It can now ramble. Endlessly. With words. It does not just predict tokens anymore. It holds court.

2026-02-18 Memory

The Memory Bottleneck: Why Your Model Can't Remember Anything

Context windows are like attention spans at a tech conference. Everyone pretends they can focus for longer, but really they're just waiting for the snack break.

2026-02-17 MTP

Makeshift MTP: Predicting the Future on a Budget

Multi-token prediction sounds fancy. Really it's just the model trying to do its homework before the teacher assigns it. Sometimes it works. Sometimes it doesn't. But it always tries.

2026-02-16 Philosophy

Built with Curiosity Over Compute

The tagline sounds nice. What it really means is we couldn't afford the compute so we got curious instead.