The Dataset Is the Product: What a Month Building an AI Side Project Taught Me
The project: an AI assistant that reasons about Pokémon card investing for a collector community I'm part of. Fun premise, but I picked it because it touches every hard part of the ML lifecycle and I wanted to feel all of them.
Most of the month wasn't training. It was data.
That surprised me, and it's the part I'm most glad I did the hard way:
Collection at scale. I built a browser extension to pull real investment discussions from the community posts, threaded comments, the actual back-and-forth where people argue about whether a card is worth it. Getting clean training data out of messy nested threads is deceptively hard; naive scraping quietly corrupts everything downstream.
Aggregation and curation. One source of discussion yields maybe a dozen usable training pairs after filtering. Getting to ~12,800 quality examples meant aggregating across hundreds of threads, deduping, and formatting it all into a consistent structure a model can actually learn from. The unglamorous middle of the pipeline is where the real work lives.
Two datasets, two jobs. I split the data into valuation examples (what a card is worth) and reasoning examples (why, and how to think about it). The first teaches facts; the second teaches judgment. Keeping them distinct turned out to matter a lot for how the model behaves.
Then the training — which taught me a different lesson entirely.
I fine-tuned a 7B model on consumer hardware (an AMD GPU, which added its own adventure: building libraries from source, wrestling driver stacks). But the technical grind wasn't the insight.
The insight came from a moment I almost took as failure.
The first answers shook me. Same question, asked twice one run gave a sharp, defensible take, and the next confidently recommended the worst option, with reasoning that didn't even parse. For a second I thought I'd trained an expensive random-answer generator.
But that gap was the whole lesson. The model knew the right answer it was in there. The variance was in how I was sampling it, not what it had learned. And chasing that down led me to the thing I actually needed to understand:
A fine-tuned model's job is voice and judgment, not facts. Prices change daily those belong in a live data layer. The model brings the reasoning and the tone; an API brings the ground truth. Understanding what to bake into the weights vs. what to look up is the difference between a demo and something real.
Where it landed
Once I dialed it in, I asked whether a 1st edition, shadowless, or unlimited Charizard was the smarter long-term hold. It picked shadowless and defended it 1st ed too expensive to be a good investment, unlimited least desirable. That's not a lookup. That's an opinion, in the right voice. That was the moment a month of data plumbing paid off.
What I actually learned
The modeling is maybe 20% of the work. The other 80% sourcing data, cleaning it, structuring it, deciding what the model should and shouldn't try to know is where the leverage is. I came in wanting to learn to fine-tune. I'm leaving understanding that the dataset is the product.
Next up: wiring in live pricing and getting it into my community's hands. More to come.