README.md

---
language:
  - en
  - zh
  - de
  - es
  - fr
  - ja
library_name: transformers
pipeline_tag: text-generation
base_model: google/gemma-4-31b-it
tags:
  - textgeneratorc2k
  - storygeneratorc2k
  - userneeds2k
  - gemma-4
  - latent-variable-models
  - text-generation
  - text-classification
  - multilingual
  - bfloat16
---

---
language:
  - en
  - zh
  - de
  - es
  - fr
  - ja
library_name: transformers
pipeline_tag: text-generation
base_model: google/gemma-4-31b-it
tags:
  - textgeneratorc2k
  - storygeneratorc2k
  - userneeds2k
  - gemma-4
  - latent-variable-models
  - text-generation
  - text-classification
  - multilingual
  - bfloat16
---

# TextGeneratorC2K (`storygeneratorc2k` + `Gemma-4` Generator + `userneeds2k` Classifier, BF16): Core Narrative Fiction Paragraph Sample Generator

## Abstract

The development of human language follows a four-stage trajectory: beginning with a basal protolanguage (~100–300 core concepts) for daily tasks, expanding into narrative vocabulary to record events and support group living (fiction), branching into specialized vocabulary for religion, law, logic, philosophy, and natural sciences (non-fiction), and ultimately forming a standardized lexicon that represents human consensus. Historically, this progression was driven by a dual mechanism: institutional codification (such as religious scripture and state administration) and spontaneous grassroots writing (such as commercial trade scripts, vernacular storytelling, and civic dialogue). In the current phase of large language model (LLM) adoption, an increasing volume of online text and academic literature is generated synthetically. When subsequent models are trained on corpora containing machine-generated text, the statistical distribution of the training data shifts. To maintain a clean baseline of contemporary natural language usage, this paper presents **`textgeneratorc2k`** (`storygeneratorc2k`), a unified **BF16 (`bfloat16`)** model combining three integrated components inside a single `model.safetensors`: (1) the **`storygeneratorc2k`** 3-level `5,376`-D latent category bank, (2) the **`gemma-4`** 58-layer 6-language generator, and (3) the **`userneeds2k`** 2,000-category classifier (`5376 -> 768` bridge + `768 -> 2048 -> 2000` `MLPClassifier`, adding only `38.8 MB` / `0.07%` to the checkpoint) to output calibrated **`categorical_scores`**. Grounded in corpus lexicography—specifically the 2,000-word defining vocabulary approach used in the *Longman Dictionary of Contemporary English*—we construct a `5,376`-dimensional hidden-layer embedding bank covering **2,000 high-frequency categories (`1,999` active non-adult categories across 6 languages: `en`, `cn`, `de`, `sp`, `fr`, `jp`)** and **5 sub-topics per category**. Each sub-topic is stratified into three stylistic registers rendered in the **contemporary news tone** of the target language: **Level 1: Basic** (everyday consumer and community news), **Level 2: Advanced** (formal industry, financial, and technical news reporting), and **Level 3: Speculative** (in-depth news analysis, policy trade-offs, and editorial commentary). By applying manifold-constrained Gaussian perturbations to these stored embeddings inside the unified `bfloat16` model, `textgeneratorc2k` generates normalized contemporary text samples (`5–6 sentences`) and simultaneously outputs 2,000-category `categorical_scores` directly through the standard Hugging Face `transformers.pipeline` interface.

---

## Quickstart (`transformers` Common Pipeline)

**`textgeneratorc2k`** (`storygeneratorc2k`) unifies all weights—the **`storygeneratorc2k` 3-Level `5,376`-D Latent Embedding Bank**, the **58-layer 6-language `Gemma-4-31B` generator**, and the **`userneeds2k` 2,000-Category Classifier (`5376 -> 768` Bridge + `MLPClassifier`)**—in **`bfloat16`** within `model.safetensors`. Both paragraph generation and text classification directly return Top-5 **`categorical_scores`**.

```python
import torch
from transformers import pipeline

# Load textgeneratorc2k (storygeneratorc2k + gemma-4 generator + userneeds2k classifier in BF16)
pipe = pipeline(
    "text-generation",
    model="bbin2022/textgeneratorc2k",  # or local path "./textgeneratorc2k" / "./storygeneratorc2k"
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
)

# 1. Generate a 5-6 sentence contemporary news paragraph + categorical_scores
res_en = pipe({"category_id": "10008/20081/31194", "language": "en", "level": 1})
print(res_en["paragraph"])
for item in res_en["categorical_scores"]:
    print(f"{item['score']:.4f} (userneeds2k={item['userneeds2k_prob']:.4f})  {item['category_id']}  {item['category_name']}")

# 2. Generate in Simplified Chinese (Level 3: In-Depth News Analysis & Editorial Commentary)
res_cn = pipe({"category_id": "10008/20081/31194", "language": "cn", "level": 3})
print(res_cn["paragraph"])

# 3. Classify any arbitrary text into Top-5 categorical_scores
preds = pipe(res_en["paragraph"])
for item in preds["categorical_scores"]:
    print(f"{item['score']:.4f} (userneeds2k={item['userneeds2k_prob']:.4f})  {item['category_id']}  {item['category_name']}")
```

---

## 1. The History of Human Language Development and Linguistic Perspectives

### 1.1 The Four-Stage Macro-Developmental Process

Based on evolutionary anthropology, historical linguistics, and cognitive science, the expansion of human vocabulary can be described in four macro-stages:

1. **Basal Protolanguage for Daily Needs (~100–300 Core Concepts):** While specific words are acquired culturally rather than genetically (Chomsky, 1965; Pinker, 1994), humans possess an innate capacity for language anchored by roughly 65 universal "semantic primes" (Wierzbicka, 1996). In early hominid groups—as well as in early childhood language acquisition and trade pidgins—this capacity manifests as a concrete protolanguage of approximately 100 to 300 basic words (Swadesh, 1955; Bickerton, 1990). This core vocabulary suffices for immediate daily tasks, indexing physical objects, kinship, and basic actions in the present environment.
2. **Group Living and Narrative Expansion (Fiction):** As social groups grew beyond roughly 150 individuals, physical grooming could no longer maintain social cohesion (Dunbar, 1996). Language adapted to record social events, track reputation, and share collective narratives. This required the linguistic property of *displacement*—referring to events outside the immediate time and place (Hockett, 1960). Vocabulary expanded to support what Donald (1991) terms *Mythic Culture* and what Bruner (1991) and Harari (2014) describe as narrative or intersubjective fiction: stories, counterfactuals, and shared traditions that coordinate group behavior.
3. **Analytical and Domain-Specific Vocabulary (Non-Fiction):** With the rise of agrarian settlements and the invention of writing (Ong, 1982; Goody, 1986), oral storytelling was supplemented by written records. Through grammatical nominalization—converting narrative actions and verbs into abstract nouns (Halliday & Martin, 1993)—vocabulary developed for non-fiction domains: religion, law, logic, philosophy, and the natural sciences.
4. **Vocabulary as Human Consensus:** At this stage, abstract terms represent shared institutional facts and collective intentionality (Searle, 1995). Words in law, philosophy, and science encapsulate agreed-upon definitions, logical relationships, and verified empirical knowledge, allowing communities to maintain continuity of thought across generations.

### 1.2 Systematic Stages of Natural Language Standardization

To understand how spoken dialects transition from Stage 2 (oral narrative) to Stages 3 and 4 (written analytical consensus), sociolinguistics and literacy theory (Ferguson, 1959; Haugen, 1966; Ong, 1982; Goody, 1986) identify six structural phases of natural language development:

| Stage | Developmental Phase | Linguistic Mechanism | Primary Cognitive & Structural Shift |
| --- | --- | --- | --- |
| **Stage I** | **Oral Vernacular & Protolanguage** | Concrete indexicality & acoustic memory | Language is strictly spoken, local, and fluid. Memory relies on rhythm, repetition, and concrete narrative formulas. |
| **Stage II** | **Orthographic Inception (Graphization)** | Invention or adoption of a writing script | Spoken sounds or concepts are fixed onto physical media (clay, bone, bark, papyrus), detaching the message from the speaker. |
| **Stage III** | **Grammatical & Phonological Codification** | Prescriptive grammar & lexicography | Rules of syntax, morphology, and pronunciation are explicitly analyzed and stabilized to prevent linguistic drift. |
| **Stage IV** | **Supraregional Standardization** | Dialect leveling & vernacular adoption | A shared written variety overrides fragmented local dialects to create a unified regional or national language. |
| **Stage V** | **Conceptual Abstraction & Metaphor** | Grammatical nominalization & semantic shift | Concrete physical words are repurposed into abstract nouns to express metaphysics, ethics, law, and technical processes. |
| **Stage VI** | **Exegetical & Analytical Discourse** | Dialectical syntax & formal logic | Development of subordinate clauses, citation systems, and structured argumentation to debate contested interpretations. |

### 1.3 Co-Evolution of Religious Writing, Institutional Catalysts, and Spontaneous Grassroots Writing

Across Stages II through VI, natural language was shaped by the continuous interaction between **top-down institutional codification** (particularly religious texts and state administration) and **bottom-up spontaneous grassroots writing** (driven by merchants, artisans, folk storytellers, and independent scholars). While religious and state institutions sought linguistic stability and fidelity, grassroots writing continuously injected phonetic simplicity, vernacular vitality, and pragmatic adaptability into natural language.

#### A. Orthographic Inception: Sacred Divination vs. Grassroots Accounting and Trade (Stage II)

The invention and spread of writing systems emerged from two parallel needs: communicating with the sacred and managing everyday material exchange.

* **Religious Script Origins:** In several traditions, writing was formalized to preserve sacred rituals or translate scriptures. In Shang-dynasty China (c. 1250 BCE), **Oracle Bone Script (*jiaguwen*)** was developed to record royal pyromantic divinations. In Late Antiquity and the Middle Ages, missionary scholars invented scripts to bring sacred texts to oral communities, including Ulfilas’s **Gothic alphabet** (4th century CE), Mashtots’s **Armenian alphabet** (405 CE), and Cyril and Methodius’s **Glagolitic and Cyrillic scripts** (9th century CE).
* **Spontaneous Grassroots & Commercial Graphization:** Parallel to religious scripts—and often predating them—writing emerged spontaneously from grassroots economic activity. In Mesopotamia (c. 3200 BCE), **cuneiform** did not originate to record scripture or royal chronicles, but evolved from clay accounting tokens used by local administrators and merchants to track grain, livestock, and labor (Schmandt-Besserat, 1992). Even more decisively, the **Phoenician consonantal alphabet** (c. 1050 BCE)—the direct ancestor of Greek, Latin, Aramaic, and Arabic scripts—was engineered by maritime traders who needed a rapid, portable, and easily learned phonetic system outside the monopoly of temple scribes. In medieval Europe and East Asia, archaeological corpora such as the **Novgorod birch bark manuscripts** (11th–15th centuries CE)—consisting of informal personal notes, debt reminders, and household letters written by ordinary townspeople, women, and artisans—demonstrate that spontaneous grassroots literacy actively shaped everyday written syntax alongside formal church Slavonic.

#### B. Grammatical Codification vs. Vernacular Vitality (Stage III)

Once scripts were established, a tension arose between freezing a sacred "High" language and recording the evolving "Low" spoken vernacular (Ferguson, 1959).

* **Religious Codification and Formal Grammar:** Because altering a sacred syllable was believed to invalidate ritual efficacy or corrupt divine law, religious scholarship produced the world’s earliest formal linguistics. In 4th-century BCE India, **Pāṇini** composed the *Aṣṭādhyāyī* (3,959 generative rules) to preserve the exact phonology and morphology of Vedic Sanskrit against vernacular drift. Similarly, in the 7th and 8th centuries CE, scholars such as **Abu al-Aswad al-Du'ali** and **Sibawayh** (*Al-Kitāb*) codified Arabic grammar (*naḥw*), lexicography, and script diacritics specifically to prevent recitation errors (*laḥn*) in the Quran.
* **Grassroots Vernacular Evolution:** When religious or courtly codification froze classical languages (such as Classical Latin, Vedic Sanskrit, or Classical Literary Chinese *wenyan*), spontaneous grassroots writing prevented language stagnation. Across the Roman Empire, everyday soldiers, merchants, and stonecutters wrote in *sermo plebeius* (Vulgar Latin)—preserved in the spontaneous street graffiti of Pompeii and the Vindolanda wooden tablets—which evolved directly into the modern Romance languages. In China, starting in the Tang and Song dynasties, folk storytellers, street performers, and Buddhist popularizers recorded oral stories in **vernacular Chinese (*baihua*)** via *bianwen* (transformation texts) and *huaben* (scriptbooks). This spontaneous grassroots writing laid the foundation for the classical vernacular novels (*Water Margin*, *Journey to the West*, *Dream of the Red Chamber*) that eventually replaced Classical Chinese as the modern national standard.

#### C. Supraregional Standardization: Canonical Translation and Grassroots Print Culture (Stage IV)

Unifying fragmented local dialects into a standardized national language required bridging institutional authority with grassroots accessibility.

* **Religious Translation as a Unifying Anchor:** When religious movements sought to reach the common population directly, they had to adopt and synthesize living vernaculars. **Martin Luther’s German Bible (1522–1534)** merged East Central German administrative syntax with vivid everyday spoken idioms; similarly, Tyndale’s New Testament (1526) and the **King James Bible (1611)** established a shared syntactic cadence for Early Modern English, while the Quran unified tribal dialects into Classical Arabic (*Fuṣḥā*).
* **Parallel Secular Literature, State Chanceries, and Grassroots Print:** Religious translation succeeded only when it intersected with secular and grassroots forces. In 14th-century Italy, standardization occurred not around a Bible translation, but around the Tuscan dialect of **Dante’s *Divine Comedy*** and **Boccaccio’s *Decameron***—the latter celebrating everyday mercantile and folk speech. In England and France, administrative standardization was driven by secular law courts (Chancery Standard English and the 1539 *Ordonnance de Villers-Cotterêts*). Crucially, during the 16th and 17th centuries, the proliferation of cheap **grassroots print culture**—pamphlets, ballads, almanacs, chapbooks, and guild manuals written by and for common citizens—stabilized spelling and spread standardized vocabulary across socioeconomic classes.

#### D. Conceptual Abstraction and Dialectical Discourse (Stages V & VI)

Finally, expanding natural language to handle non-fiction logic, law, science, and philosophy required both institutional hermeneutics and grassroots civic debate.

* **Religious Exegesis and Semantic Abstraction:** Theological writing systematically converted concrete physical nouns into abstract institutional concepts (e.g., Latin *spiritus* ["breath"] $\rightarrow$ "spirit"; Greek *kanōn* ["measuring reed"] $\rightarrow$ "canon/rule"; Hebrew *berit* ["cutting"] $\rightarrow$ "covenant"). Furthermore, exegetical traditions—such as **Rabbinic Talmudic debate**, **Buddhist Abhidharma taxonomy**, **Islamic jurisprudence (*Usul al-fiqh*)**, and **Medieval Scholasticism**—engineered rigorous syntactic frameworks for stating premises, raising counter-arguments, and resolving textual contradictions.
* **Parallel Civic Law, Philosophy, and Grassroots Debate:** Operating alongside religious exegesis were secular legal and philosophical movements. Secular law codes—from the **Code of Hammurabi** and **Roman Civil Law** to Chinese **Legalism (*Fajia*)**—established formal conditional syntax (*"If X occurs under condition Y, then Z applies"*). During the Axial Age, grassroots philosophical schools—such as the wandering scholars (*shi*) of China’s **Hundred Schools of Thought** and the **Athenian *agora*** dialogues of Socrates—conducted open debates using everyday analogies to test ethical and logical propositions. Later, in the 17th century, the Scientific Revolution combined experimental artisan practices with grammatical nominalization (Halliday & Martin, 1993) to forge modern empirical and journalistic prose.

### 1.4 Summary of Linguistic Schools and Historical Drivers

| Analytical Dimension | Key Literature | Role in Natural Language Evolution |
| --- | --- | --- |
| **Biolinguistics & Semantic Universals** | Swadesh (1955); Chomsky (1965); Bickerton (1990); Wierzbicka (1996) | Establishes the innate cognitive baseline (~65 primes) and the 100–300 word concrete protolanguage used for daily survival. |
| **Evolutionary Anthropology** | Hockett (1960); Donald (1991); Dunbar (1996); Harari (2014) | Explains how group living drove *displacement* and narrative storytelling (fiction) to coordinate communities beyond 150 individuals. |
| **Sociolinguistics & Standardization Theory** | Ferguson (1959); Haugen (1966); Ostler (2005); Pollock (2006) | Models the interaction between codified "High" institutional registers (religion, state) and spontaneous "Low" grassroots vernaculars. |
| **Literacy Theory & Functional Linguistics** | Ong (1982); Goody (1986); Schmandt-Besserat (1992); Halliday & Martin (1993) | Traces how grassroots accounting, religious scripture, legal codes, and scientific nominalization transformed oral speech into analytical non-fiction. |
| **Social Ontology** | Searle (1995) | Demonstrates how standardized vocabulary functions as crystallized human consensus and collective intentionality. |

---

## 2. Language Standardization and the Need for Uncontaminated Baselines

### 2.1 Full-Text Corpora and the Longman Standardization Model

As historical linguistics demonstrates, a healthy natural language relies on a balance between standardized core definitions and authentic grassroots usage distributions. In modern corpus linguistics, this balance is captured through two complementary mechanisms:

1. **Controlled Defining Vocabularies:** West (1953) established in the *General Service List (GSL)* that roughly 2,000 core word families account for over 80% of general written English. Building on this empirical distribution, Procter (1978) introduced the **Longman Defining Vocabulary (LDV)** in the *Longman Dictionary of Contemporary English (LDOCE)*, demonstrating that complex concepts across specialized domains can be standardized and explained using a core lexicon of 2,000 high-frequency words.
2. **Empirical Full-Text Databases:** As shown by Sinclair (1991), maintaining language standards requires full-text databases rather than isolated word lists. Full-text corpora record the spontaneous grassroots collocations, idiomatic phrasing, and habitual probability distributions of natural human writing.

### 2.2 Addressing Training Input Contamination

We are currently in the early stages of widespread LLM application. A growing proportion of articles, web pages, and research preprints (such as those on arXiv) are generated or assisted by language models.

When machine-generated text is fed back into the training corpora of subsequent models, it alters the statistical distribution of the training input—specifically eroding the spontaneous grassroots variance that historically kept natural language adaptable. In computational linguistics, Kirby, Cornish, and Smith (2008) demonstrated through **Iterated Learning Models (ILM)** that when a system repeatedly learns from the sampled outputs of previous generations without grounding in the original human distribution, the language loses lexical diversity and undergoes distributional collapse (referred to in recent machine learning literature as *model collapse*).

Current open-weights models were trained on corpora collected before synthetic text reached high saturation, meaning their internal representations still reflect uncontaminated human language distributions relatively accurately. However, as synthetic text accumulates online, future training corpora will face increasing contamination. Designing a sample generator at this stage serves a practical long-term goal: preserving and sampling from the current, uncontaminated natural language distribution.

---

## 3. Methodology & Model Implementation (`textgeneratorc2k`: `storygeneratorc2k` + `Gemma-4` Generator, BF16)

To generate standardized passages that reflect habitual human language distributions across **6 languages (`en`, `cn`, `de`, `sp`, `fr`, `jp`)**, **`textgeneratorc2k`** combines **`storygeneratorc2k`** (`5,376`-dimensional 3-level hidden-layer embedding bank + $5376 \to 768$ bridge + 2,000-category classifier) with a depth- and vocabulary-optimized **58-layer `Gemma-4-31B` (`bfloat16`)** generator.

### 3.1 Hierarchical Taxonomy and Three-Level Contemporary News Register Stratification

Following the 2,000-unit scale of the Longman Defining Vocabulary (Procter, 1978) and reflecting the progression from everyday consumer narrative to specialized industry reporting and analytical commentary, we organize the embedding bank into a three-tier hierarchy:

1. **Question Categories ($C$):** $N_c = 2,000$ hierarchical categories (`Categoryv2-2000`, defined in [`catid_define.json`](catid_define.json) across `en`, `cn`, `de`, `sp`, `fr`, and `jp`; `1,999` active non-adult categories, with `10001` Adult excluded).
2. **Sub-Topics ($T$):** For each category $c_i \in \{c_1, \dots, c_{2000}\}$, $N_t = 5$ representative sub-topics $t_{i,j} \in \{t_{i,1}, \dots, t_{i,5}\}$, resulting in $10,000$ topical nodes.
3. **Three Complexity Levels ($L$) in Contemporary News Tone:** Each sub-topic $t_{i,j}$ is stratified into three register levels $l \in \{1, 2, 3\}$ (`30,000` total anchor points), all rendered in the **contemporary news tone** of the target language (`5–6 sentences`, `~110–150 words` for `en/de/sp/fr` or `~180–240` characters for `cn/jp`, matching the `79–88` word / `5.5–6.4` sentence empirical distribution of the uncontaminated reference corpus):

| Level | Register Name | Linguistic & Journalistic Alignment | Objective and Output Design (`5–6 Sentences`) |
| --- | --- | --- | --- |
| **Level 1** | **Basic (Consumer & Community News)** | Everyday consumer, local, and livelihood news reporting grounded in high-frequency accessible vocabulary. | Reports a concrete real-world consumer, service, or community story on the topic using clear, accessible contemporary journalism. |
| **Level 2** | **Advanced (Industry & Technical News)** | Codified institutional, financial, and technical journalism using domain-specific terminology and nominalization. | Articulates technical mechanisms, industry standards, regulatory compliance, and institutional operations in formal news prose. |
| **Level 3** | **Speculative (In-Depth News Commentary)** | Contemporary editorial analysis, policy debate, and strategic industry commentary. | Examines long-term industry trends, public policy trade-offs, regulatory debates, and societal implications in modern analytical journalism. |

### 3.2 Hidden-Layer Embedding Bank Construction and Perturbation

To generate varied text samples from these categories without drifting from natural human distributions, we adapt the continuous latent space sampling approach of Bowman et al. (2015) in the `d = 5,376` hidden space of `Gemma-4-31B`:

1. **Centroid Extraction & Principal Subspace Compression:** For each node $(c_i, t_{i,j}, l)$, human-written reference passages from uncontaminated web/news archives (`29,985` passages across the `1,999` non-adult categories) are passed through `Gemma-4-31B` (`d = 5,376`). We extract IDF-weighted and contextualized sequence-pooled hidden states to form $\mathbf{e}_{i,j,l} \in \mathbb{R}^{5376}$, storing the `6,000` category-level centroids $\bar{\mathbf{e}}_{i,l} \in \mathbb{R}^{2000 \times 3 \times 5376}$ and projecting the `30,000` sub-topic residuals onto a shared `256`-dimensional principal subspace $\mathbf{U} \in \mathbb{R}^{256 \times 5376}$ with coefficients $\mathbf{C}_{i,j,l} \in \mathbb{R}^{2000 \times 5 \times 3 \times 256}$:

$$\mathbf{e}_{i,j,l} = \operatorname{normalize}\!\left(\bar{\mathbf{e}}_{i,l} + \mathbf{C}_{i,j,l}\mathbf{U}\right)$$

2. **Manifold-Constrained Random Perturbation:** Because the hidden space of a transformer is anisotropic, unconstrained Gaussian noise across all dimensions pushes vectors off the semantic manifold. We compute the local empirical covariance $\boldsymbol{\Sigma}_{i,l}$ within category $c_i$ at level $l$:

$$\boldsymbol{\Sigma}_{i,l} = \frac{1}{N_{\text{ex}} - 1} \sum_{n=1}^{N_{\text{ex}}} (\mathbf{h}_{i,l,n} - \bar{\mathbf{e}}_{i,l})(\mathbf{h}_{i,l,n} - \bar{\mathbf{e}}_{i,l})^\top + \lambda \mathbf{I}_d$$

During generation for $(c_i, t_{i,j}, l)$, we sample $\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}_{256} / 256)$ within the principal manifold subspace and scale by the empirical dispersion $\sigma_{i,l}$ and perturbation hyperparameter $\gamma = 0.45$:

$$\tilde{\mathbf{h}}_{i,j,l} = \operatorname{normalize}\!\left(\mathbf{e}_{i,j,l} + \gamma \sigma_{i,l} \boldsymbol{\epsilon}\mathbf{U}\right)$$

3. **Centered Concept Readout & Zero-Tokenizer Multilingual Decoding:** The perturbed vector $\tilde{\mathbf{h}}_{i,j,l} \in \mathbb{R}^{5376}$ queries the corpus-mean-centered unembedding matrix $\mathbf{W}_{\text{centered}} \in \mathbb{R}^{50780 \times 5376}$ of `Gemma-4-31B` to extract 7 distinct domain concept tokens, which are spliced directly into pre-tokenized `6-language x 3-level` prompt templates and decoded autoregressively:

$$P(Y \mid \tilde{\mathbf{h}}_{i,j,l}) = \prod_{t=1}^{m} P(y_t \mid y_{<t}, \tilde{\mathbf{h}}_{i,j,l})$$

### 3.3 Unified BF16 Model Architecture (`textgeneratorc2k` in `model.safetensors`, `59.13 GB`)

To minimize package footprint while maintaining full `bfloat16` precision and native `transformers.pipeline` compatibility, `textgeneratorc2k` packs the `gemma-4` generator, the `storygeneratorc2k` latent bank, `embeddinggemma-300m`, and the `userneeds2k` classifier into a single **`59.13 GB` BF16 `model.safetensors`** checkpoint (`1,134` tensors):

1. **58-Layer 6-Language `Gemma-4-31B` Generator (`lm.*`, `bfloat16`, `58.52 GB`):**
   - **Depth Pruning (`60 -> 58` layers):** Drops two redundant first-half sliding-attention layers (`{13, 14}` with angular distance $< 0.045$), preserving all 10 global `full_attention` layers (`head_dim = 512`, `num_global_key_value_heads = 4`) and all second-half layers (`30..59`) untouched.
   - **6-Language Vocabulary Pruning (`262,144 -> 175,359` tokens):** Retains 100% of tokens for English (`en`), Chinese (`cn`), German (`de`), Spanish (`sp`), French (`fr`), and Japanese (`jp`).
2. **Integrated `storygeneratorc2k` 3-Level Latent Bank (`bfloat16`, `84.59 MB`):**
   - **3-Level Latent Bank (`level_centroids`, `topic_subspace_coeffs`, `subspace_basis`, `sigma_il`, `embed_mean`):** Stored as registered `bfloat16` buffers on the model.
3. **Integrated `EmbeddingGemma-300M` + `userneeds2k` 2,000-Category Classifier (`bfloat16`, `522.01 MB`):**
   - **`EmbeddingGemma-300M` (`embeddinggemma.*`, `embedding_dense1.weight`, `embedding_dense2.weight`, `481.86 MB`):** 24-layer `Gemma3TextModel` (`hidden_size = 768`, aligned to the shared `175,359` 6-language vocabulary) plus two dense projection layers (`768 -> 3072 -> 768`).
   - **Dynamic-Negative `userneeds2k` `MLPClassifier` (`classifier.*`, `768 -> 2048 -> 2000`, `bfloat16`):** LayerNorm + Linear(`768, 2048`) + GELU + Linear(`2048, 2000`) trained with dynamic hard-negative binary cross-entropy across all 2,000 categories to compute `categorical_scores`.

---

## 4. Empirical Performance & Package Layout

### 4.1 2,000-Category Classification Benchmark (NVIDIA H100 NVL)

| Evaluation Split | nDCG@5 | Precision@5 | Recall@5 | Top-1 Hit | Tail Recall@5 |
| :--- | ---: | ---: | ---: | ---: | ---: |
| **English Held-Out Test (`25,874` samples)** | `0.811408` | `0.214802` | `0.854909` | `0.731004` | `0.849272` |
| **Multilingual Test (`1,000` samples)** | `0.795043` | `0.210200` | `0.838467` | `0.713000` | `0.831500` |

### 4.2 100-Pair Blind Human-vs-Generator Turing Benchmark (`1,000` Evaluations Across 10 Frontier Models)

To evaluate how often paragraphs synthesized by **`TextGeneratorC2K` (`storygeneratorc2k` + `Gemma-4-31B` BF16)** are judged as authentic human journalism against real web-extracted prose, we conducted a strictly blind **100-Pair A/B Turing Benchmark** ([`human_turing_100pairs_benchmark.json`](human_turing_100pairs_benchmark.json)):

1. **100 Original Google Samples:** Selected across `100` distinct non-adult categories and all `3` sociolinguistic registers (`34` Level 1 Basic, `33` Level 2 Advanced, `33` Level 3 Speculative; `103.3` words and `4.45` sentences on average).
2. **100 Matched `TextGeneratorC2K` Samples:** Generated on an NVIDIA H100 NVL (`55.8s` total across 4 batches of 25) conditioned on the **exact same `(category_id, level)`** (`124.3` words and `5.43` sentences on average).
3. **1,000 Blind A/B Evaluations Across 10 Frontier Models:** For each pair, `Passage A` and `Passage B` were randomized and presented without source labels to **5 flagship open-source models** (via Ollama Cloud API) and **5 flagship closed-source models** (via Poe API, local OpenAI Codex CLI, and native AGY Gemini Subagents), asking: *"Which passage reads more like an authentic, well-edited contemporary human news paragraph for this category and register?"*

#### Table 4.2.1 — Overall & Model-Family Summary (`1,000` Blind Evaluations)

| Judge Cohort | Total Evaluations | `TextGeneratorC2K` Chosen as Human | Google Original Chosen as Human | Ties | `TextGeneratorC2K` Win Rate | Human Indistinguishability Score (`Win + 0.5×Tie`) |
| :--- | ---: | ---: | ---: | ---: | ---: | ---: |
| **Open-Source Frontier Models (5 Ollama Models)** | `500` | `175` | `323` | `2` | **`35.00%`** | **`35.20%`** |
| **Closed-Source Frontier Models (3 Poe + Codex + AGY)** | `500` | `76` | `423` | `1` | **`15.20%`** | **`15.30%`** |
| **All 10 Frontier Judge Models (Combined)** | **`1,000`** | **`251`** | **`746`** | **`3`** | **`25.10%`** | **`25.25%`** |

#### Table 4.2.2 — Per-Judge-Model Breakdown (`100` Blind Pairs per Judge)

| Judge Model | Model Family / Provider | `TextGeneratorC2K` Chosen as Human | Google Original Chosen as Human | Ties | `TextGeneratorC2K` Win Rate | Indistinguishability Score |
| :--- | :--- | ---: | ---: | ---: | ---: | ---: |
| **`mistral-large-3:675b`** | Open-Source (Ollama API) | `58` | `42` | `0` | **`58.00%`** | **`58.00%`** |
| **`gpt-oss:120b`** | Open-Source (Ollama API) | `47` | `53` | `0` | **`47.00%`** | **`47.00%`** |
| **`kimi-k3`** | Open-Source (Ollama API) | `29` | `70` | `1` | **`29.00%`** | **`29.50%`** |
| **`deepseek-v4-pro:0813`** | Open-Source (Ollama API) | `27` | `73` | `0` | **`27.00%`** | **`27.00%`** |
| **`gemini-3.8-flash`** | Closed-Source (Poe API) | `27` | `72` | `1` | **`27.00%`** | **`27.50%`** |
| **`gpt-5.4`** | Closed-Source (Poe API) | `20` | `80` | `0` | **`20.00%`** | **`20.00%`** |
| **`glm-5.3`** | Open-Source (Ollama API) | `14` | `85` | `1` | **`14.00%`** | **`14.50%`** |
| **`claude-sonnet-5.5`** | Closed-Source (Poe API) | `11` | `89` | `0` | **`11.00%`** | **`11.00%`** |
| **`gemini-agy-subagent`** | Closed-Source (AGY Subagent) | `11` | `89` | `0` | **`11.00%`** | **`11.00%`** |
| **`gpt-6-astra`** | Closed-Source (Local Codex CLI) | `7` | `93` | `0` | **`7.00%`** | **`7.00%`** |

#### Table 4.2.3 — Breakdown by Sociolinguistic Register Level (`Level 1`, `Level 2`, `Level 3`)

| Sociolinguistic Register | All 10 Models (`TextGeneratorC2K` Rate) | Open-Source Judges (`500` Evals) | Closed-Source Judges (`500` Evals) | Key Linguistic Finding |
| :--- | ---: | ---: | ---: | :--- |
| **Level 1: Basic (`34` Pairs — Consumer & Community News)** | **`44.12%`** (`150 / 340`, `3` ties) | **`55.29%`** (`94 / 170`, `55.88%` score) | **`32.94%`** (`56 / 170`, `33.24%` score) | **`TextGeneratorC2K` beats Google Original (`55.29%` vs. `43.53%`) on Open-Source Judges** and wins nearly a third of Closed-Source evaluations due to grounded community narrative flow. |
| **Level 2: Advanced (`33` Pairs — Industry & Technical News)** | **`6.06%`** (`20 / 330`, `0` ties) | **`10.30%`** (`17 / 165`) | **`1.82%`** (`3 / 165`) | Real Google industry excerpts cite named executives, exact model numbers, and dated regulatory filings that closed-source judges prioritize over abstract technical synthesis. |
| **Level 3: Speculative (`33` Pairs — In-Depth News Commentary)** | **`24.55%`** (`81 / 330`, `0` ties) | **`38.79%`** (`64 / 165`) | **`10.30%`** (`17 / 165`) | Strong analytical commentary performance (`38.79%` across open-source judges) where `TextGeneratorC2K`'s policy/forecasting structure outperforms noisy web excerpts. |

#### Key Diagnostic Insights from Judge Justifications
1. **Superiority in Level 1 Community Journalism (`55.29%` Open-Source / `44.12%` Overall):** In `Level 1: Basic`, `TextGeneratorC2K` consistently generates cohesive, well-edited community news stories that outperform raw web-scraped excerpts containing promotional copy, first-person fragments, or academic citations. Across `mistral-large-3:675b` (`58.0%`) and `gpt-oss:120b` (`47.0%`), `TextGeneratorC2K` is judged overall as equal to or more human-authentic than original Google web samples.
2. **Cross-Lingual Concept Leakage in Shared 6-Language Subspace:** Closed-source reasoning judges (`gpt-6-astra`, `claude-sonnet-5.5`, `gemini-agy-subagent`) identified two primary giveaways when rejecting `TextGeneratorC2K` passages in `Level 2` and `Level 3`:
   - Because `valid_words` spans the unified 6-language (`en, zh, es, id, pt, it`) vocabulary, non-English Latin-script concept words (`trabajadores`, `bayi`, `playas`, `lucro`, `jornal`) occasionally ranked in the top-7 cosine neighbors of `h_tilde` and were injected into the English prompt.
   - Real `Level 2` trade articles cite specific named institutions, SKU/engine numbers, and direct quotes from named researchers, whereas zero-shot latent prompts without named-entity grounding produce more abstract institutional transitions.

### 4.3 Repository Files (`textgeneratorc2k` / `storygeneratorc2k`)

| File / Directory | Description |
| :--- | :--- |
| `model.safetensors` | Unified **BF16 (`bfloat16`)** weights containing the 58-layer 6-language `Gemma-4-31B` generator, the `storygeneratorc2k` 3-Level `5,376`-D Latent Bank, the `EmbeddingGemma-300M` + `userneeds2k` 2,000-category `MLPClassifier`, and compressed runtime metadata. |
| `google_embeddinggemma-300m/` | Self-contained `google/embeddinggemma-300m` (`Gemma3TextModel` + `1_Pooling` + `2_Dense` + `3_Dense` in `bfloat16`) subdirectory for standalone document embedding and `userneeds2k` categorical scoring. |
| [`catid_define.json`](catid_define.json) | Minified human-readable 6-language (`en, cn, de, sp, fr, jp`) category taxonomy definitions (`1,999` non-adult categories). |
| `config.json` | Standard Hugging Face `PreTrainedConfig` with `auto_map` and `custom_pipelines` registration for `transformers.pipeline`. |
| `modeling_textgeneratorc2k.py` | Hugging Face `PreTrainedModel` and `Pipeline` hook enabling direct execution via `from transformers import pipeline`. |
| `README.md` | This design document and model card. |

---

## References

* Bickerton, D. (1990). *Language and Species*. University of Chicago Press.
* Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A. M., Jozefowicz, R., & Bengio, S. (2015). Generating sentences from a continuous space. *arXiv preprint arXiv:1511.06349* (Published in *Proceedings of CoNLL 2016*).
* Bruner, J. (1991). The narrative construction of reality. *Critical Inquiry*, 18(1), 1–21.
* Chomsky, N. (1965). *Aspects of the Theory of Syntax*. MIT Press.
* Donald, M. (1991). *Origins of the Modern Mind: Three Stages in the Evolution of Culture and Cognition*. Harvard University Press.
* Dunbar, R. (1996). *Grooming, Gossip, and the Evolution of Language*. Harvard University Press.
* Ferguson, C. A. (1959). Diglossia. *Word*, 15(2), 325–340.
* Goody, J. (1986). *The Logic of Writing and the Organization of Society*. Cambridge University Press.
* Halliday, M. A. K., & Martin, J. R. (1993). *Writing Science: Literacy and Discursive Power*. University of Pittsburgh Press.
* Harari, Y. N. (2014). *Sapiens: A Brief History of Humankind*. Harper.
* Haugen, E. (1966). Dialect, language, nation. *American Anthropologist*, 68(4), 922–935.
* Hockett, C. F. (1960). The origin of speech. *Scientific American*, 203(3), 88–96.
* Kirby, S., Cornish, H., & Smith, K. (2008). Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language. *Proceedings of the National Academy of Sciences (PNAS)*, 105(31), 10681–10686.
* Ong, W. J. (1982). *Orality and Literacy: The Technologizing of the Word*. Methuen.
* Ostler, N. (2005). *Empires of the Word: A Language History of the World*. HarperCollins.
* Pinker, S. (1994). *The Language Instinct: How the Mind Creates Language*. William Morrow and Company.
* Pollock, S. (2006). *The Language of the Gods in the World of Men: Sanskrit, Culture, and Power in Premodern India*. University of California Press.
* Procter, P. (Ed.). (1978). *Longman Dictionary of Contemporary English (LDOCE)* (1st ed.). Longman.
* Schmandt-Besserat, D. (1992). *Before Writing, Vol. I: From Counting to Cuneiform*. University of Texas Press.
* Searle, J. R. (1995). *The Construction of Social Reality*. Free Press.
* Sinclair, J. (1991). *Corpus, Concordance, Collocation*. Oxford University Press.
* Swadesh, M. (1955). Towards greater accuracy in lexicostatistical dating. *International Journal of American Linguistics*, 21(2), 121–137.
* West, M. (1953). *A General Service List of English Words*. Longman, Green & Co.
* Wierzbicka, A. (1996). *Semantics: Primes and Universals*. Oxford University Press.