From Complete Dialogues to Traceable Judgments: Why We Upgraded CQA to V2
This article was first written in Chinese. This English version was translated with AI assistance.
In 213 BCE, Qin Shi Huang, the first emperor of the Qin dynasty in China, ordered the burning of books. 2,145 years later, pro-Hitler student organizations started a book-burning campaign across Germany. It continued for several weeks. Hitler probably did not expect that, more than ninety years after he allowed books to be burned, humans would again destroy millions of physical books in an organized way. Many were old or out of print.
At the end of July 2026, Elon Musk reposted a post about rare books. The original poster, @Hedgie, accused AI companies of a predatory engineering practice. To get data faster, they cut the spines from batches of rare books, scan the pages one by one, and then destroy the original books.
Musk promised that SpaceXAI would use another method. It would not cut the spines. It would scan the books in a more laborious way and preserve the rare books afterward.
The Content Is Still There. The Structure Is Gone
The function of a book spine is to connect a pile of separate pages into one book. After the spine is cut, every page is still there. But the binding, the structural relation between the pages, is gone.
Something similar happens when books are used as training data.
The finished text enters the training data. The structure of creation does not. Why did the author choose this structure? Where was a key choice made? What judgments happened between the first draft and the final draft? These decision processes are not in the finished text.
The corpus keeps the result of creation. It does not inherit the structure of creation.
Our CQA (Creative Quality Alignment) dataset is built around many complex expert discussions. Questions, counterexamples, disagreements, corrections, and convergence really happened in these discussions. V1 already kept them inside <think>.
But this content was still one long string. Which question changed the direction? Which counterexample corrected an earlier judgment? These were not clear structural signals in the data. A model could only infer them from continuous text. It could not reliably identify the relations between the judgments.
The final conclusion was kept. The process that produced it was still unclear.
The questions and final answers in CQA were both organized later from these expert discussions.
So in V1, we did not keep only a standard question-answer pair. We put the complete expert dialogue inside the <think> tags. This design came from a simple requirement: however we process the data, an answer organized afterward should not replace the discussion that actually happened.
Putting real expert dialogue into the data also shows the full process of creative judgments forming during the interaction.
V1 solved the problem of "do not destroy it." V2 does not reject V1. It keeps the original dialogue, and makes the judgments, corrections, and convergence inside it easier to identify, point back to, and check.
The First Change: Separate the Source Dialogue
V2 uses a separate source_dialogue field to store the expert dialogue.
The dialogue is divided by the actual speaking turns. Every turn has a stable ID. This field stores the experts' words after cleaning for public release. It is not a summary of a reasoning event, and it is not the final answer.
This may look like only splitting a long text. Its real function is to establish a source-material layer. No later reasoning event or final answer can replace source_dialogue. If there is disagreement about how a curator summarized part of the discussion, a reviewer can return to the experts' original words.
Stable turn IDs also make it possible to cite a specific position in the dialogue. "Read the whole dialogue" is no longer the only way to use the data. We can point to where a judgment came from.
The Second Change: Identify the Reasoning Events in the Dialogue
V2 adds reasoning_events.
A reasoning event is not a shorter rewrite of several turns. It describes what cognitive work a group of turns completed. For example:
- establishing a shared context;
- comparing two causal explanations;
- correcting an earlier judgment with a counterexample;
- identifying a creative constraint;
- constructing a solution mechanism;
- abstracting a principle from a concrete case.
Every event has an ID, a type, and a summary. It must also point back to one or more original turns. Readers can see the organized reasoning structure, and can also check whether the source dialogue supports this organization.
This relation is important. If there are only event summaries, the curator's explanation may still replace the experts' words. If there is only the complete dialogue, the reasoning structure is still hidden in continuous text. V2 keeps both and connects them with references.
There is still a boundary we need to state clearly. The Chinese reasoning events were later revised by experts. But they are still structured annotations derived from real dialogues. They were not process labels filled in step by step by the experts during the conversation. They also should not be described as a complete record of private internal thinking.
The Third Change: Put the Final Answer in Its Own Place
In V2, final_answer becomes a separate field.
It is still organized from the complete dialogue. But it is no longer mixed with the source dialogue and reasoning events in one string. We can now distinguish three things:
- what the experts actually said;
- what was added by the structural organization of the discussion;
- what became the final answer to the original question.
This distinction makes review and revision more specific. If a problem appears in one layer, we can inspect that layer first, and then check the other content connected to it.
After the Separation: Different Task Formats from One Master Record
After the source dialogue, reasoning events, and final answer are separated, the same Chinese master data can currently be exported into three formats for different tasks. Here, a "task format" means selecting different fields from the master record and recombining them as inputs and outputs for a particular training objective:
prompt → final_answer;prompt → reasoning-event trace → final_answer;source dialogue → final_answer.
These are not three independent datasets. They come from the same master record. Different tasks read different layers.
Having more files is not the point. The source material does not have to be permanently rewritten for one training format. If the research goal changes later, we can return to the same master data and decide again which layers to use. We do not need to invent another expert dialogue.
Chinese and English are stored as separate records in V2. They correspond through the same sample_id, turn_id, and event_id. This keeps the relation between the two languages, but also makes their status clear. Chinese is the source record. English is still an AI-assisted translation for reading. Aligned fields do not make it independently reviewed English training data.
The Data Structure Should Not Decide How Experts Think
V2 does not require every sample to have the same number or the same types of reasoning events.
Real creative discussions are not orderly. Some start from a failed example. Some compare two stories. Some first identify a constraint and then slowly construct a mechanism. If we force them into the same steps to make the schema look clean, the data will look more orderly, but the expert judgment will be distorted.
So we do not first define a "correct chain of thought" and then make the dialogue fit it. We first keep the real dialogue. Then we identify the cognitive work that already happened inside it.
For the same reason, V2 does not generate preference, judge, or process-supervision data when there are no real candidate pairs or expert step-level labels. We structure the data to represent the judgment relations that really exist in the source material, not to fabricate a clean thinking process.
This is also why professional creators need to participate in data-structure design. Engineers can design fields, pipelines, and validation. But we still need experts to confirm which judgments really happened, which question changed the conclusion, and whether a summary flattened the original meaning.
What V2 Brings, and What It Has Not Proved
The direct improvements of V2 are specific:
- Easier to trace. Reasoning events and final answers can point back to the source dialogue.
- Easier to check. The experts' words, derived annotations, and organized results are no longer mixed in one layer.
- Easier to revise. When a problem appears in one layer, that layer can be reviewed directly.
- Easier to derive. The same master data can produce different task formats.
But "the structure is clearer" does not mean "the training effect has been proved."
V2 has not proved that a model trained with this structure will certainly write better. It has not proved which task format is the most effective. These questions still require larger datasets, controlled experiments, and professional evaluation.
The Experts Came. Then What?
In September 2025, Elon Musk's xAI announced that it would expand its specialist AI tutor team to ten times its previous size. By early 2026, xAI had also started recruiting Writing Specialists in fiction, screenwriting, game writing, journalism, and poetry. The job description was clear. They were not being hired to write for the company. They would evaluate and revise AI-generated text, and provide selected data for Grok.
This recruiting path did not keep expanding. In June 2026, Bloomberg Law reported that xAI had paused hiring for these specialist AI tutor roles. The report cited people familiar with the matter. It said the pause happened at least partly because the human resources department could not process so many applicants, and that the pause might be temporary. The report was about a pause in hiring. It did not mean that the whole program had stopped. There is also no public information showing what data these Writing Specialists actually produced, or whether the data has entered training.
Even so, this recruiting history gives at least one signal. Finished texts and general annotation may not be enough. Some model capabilities may need judgment from real domain experts.
Inviting experts only answers "who makes the judgment." The harder question is what form these judgments take in the data. Do they become a score, a preference label, or one conclusion? Or are the questions, counterexamples, disagreements, and corrections kept together, as something we can return to and check?
By upgrading CQA from V1 to V2, we are trying to answer the second question.
Do not cut off the book spine. Do not cut off the spine of thinking.
You can see the CQA (Creative Quality Alignment) public preview on Hugging Face. V1 is still kept in the repository history. The current page shows V2, reconstructed from the same four samples.