Lingqu-Rec/HSTU-0.2B

HSTU Generative Recommendation Model Test Configuration

config.json in this directory is the test configuration for the HSTU (Hierarchical Sequential Transduction Unit) generative recommendation model. It describes a test model with an intended size of approximately 0.2B parameters and declares HSTUForCausalLM as its model entry point.

Note: use_random_model is set to true. The configuration therefore uses randomly initialized weights for inference-pipeline, operator, and performance tests; it does not indicate that a trained model checkpoint has been loaded.

1. Configuration Summary

Configuration	Value	Description
model_type	hstu	HSTU model type
architectures	HSTUForCausalLM	Model loading entry point. CausalLM is a framework-compatible interface name; the actual task is candidate scoring for recommendation.
Intended model size	Approximately 0.2B	Target size of the test model represented by this configuration
num_hidden_layers / hstu_config.num_layers	12	Number of HSTU layers
hidden_size	1024	Hidden-state dimension at each token or feature position
num_attention_heads	8	Number of attention heads per layer
head_dim	128	Dimension of each attention head; 8 × 128 = 1024
max_seq_len	8832	Maximum sequence length allowed for one request. The exact number of history and candidate positions depends on the input-packing scheme.
torch_dtype / hstu_config.dtype	float32	Data type used by this test configuration
is_causal	true	Enables causal masking, so a position cannot read future positions
residual	true	Enables residual connections in HSTU layers
has_ffn	true	Includes a feed-forward network (FFN) in each HSTU layer
dropout_ratio	0	Dropout is disabled for inference
norm_epsilon	1e-5	Numerical-stability epsilon used by normalization layers

2. Input Features and Embedding Tables

task_config.embedding_configs defines the recommendation features used by the model:

Feature	Table	Vocabulary Size	Embedding Dimension	Dynamic Embedding	Purpose
action_weights	act	1024	1024	No	User behavior/action type or behavior weight
video_id	item	100000	1024	Yes	Video/item ID used as a candidate-item feature

The configuration also declares:

• item_feature_name = "video_id"
• action_feature_name = "action_weights"
• Both input embedding dimensions match hidden_size = 1024.

In a typical HSTU inference implementation, input features first go through embedding lookup. Multiple features at the same position are combined through concatenation and/or a linear projection to form a 1024-dimensional input representation. Positional information is then added before the representation is passed to the HSTU backbone.

3. HSTU Model Architecture

The logical path from model inputs to recommendation scores is:

User-history features + candidate-item features
        │
        ├─ action_weights embedding: 1024 → 1024
        └─ video_id embedding:       1024 → 1024
        │
        └─ Feature combination / linear projection + positional embedding
                         │
                         ▼
              ┌──────────────────────┐
              │ HSTU Layer 1         │
              │ …                    │
              │ HSTU Layer 12        │
              └──────────────────────┘
                         │
                         ▼
              Hidden states at candidate positions
                         │
                         ▼
                    Dense/MLP score head
                         │
                         ▼
              One scalar score for each candidate

A single HSTU layer can be summarized as follows:

Input x
 │
 ├─ Input normalization (epsilon = 1e-5)
 ├─ Fused UVQK linear projection
 │    └─ Split into U, V, Q, and K representations
 ├─ SiLU activation
 ├─ Causal attention (8 heads, head_dim = 128)
 ├─ Attention-output normalization
 ├─ U gating / element-wise modulation of the attention output
 ├─ Linear projection back to 1024 dimensions
 ├─ SwiGLU FFN (has_ffn = true)
 └─ Residual connection (residual = true)
 │
Output x'

This is not a conventional language model that generates text tokens. It uses a ForCausalLM-style interface to apply causal sequence modeling to the joint representation of user behavior history and candidate items, and ultimately produces a ranking score for each candidate. The current config.json does not explicitly specify the intermediate architecture of the prediction head under task_config; the concrete head configuration is determined by the model implementation or loader defaults. Its task semantics are to output one score per candidate.

4. Inference Paradigm: Generative Recommendation vs. LLM

Both model types may expose a CausalLM-style interface, but their decode-stage objectives are fundamentally different.

4.1 Generative Recommendation: Parallel Candidate Scoring

Given user history (h) and a candidate set (C = {c_1, ldots, c_M}), HSTU computes a score for every candidate:

[ s_i = f_\theta(h, c_i), \qquad i = 1, \ldots, M ]

During decode, multiple candidates are arranged as positions or packed sequences that can be evaluated in parallel. A forward pass produces the score vector:

[ \mathbf{s} = [s_1, s_2, \ldots, s_M] ]

The candidates are then sorted by score and truncated to Top-K. Thus, the primary parallel dimension during decode is the candidate dimension. As the candidate count grows, candidate batching, candidate parallelism, and memory usage become important. The model does not need to generate (c_1) first and then generate (c_2) conditioned on (c_1).

4.2 LLM: Autoregressive Token-by-Token Generation

Given a prompt prefix (x_{<t}), an LLM computes the distribution of the next token at step (t):

[ p(x_t \mid x_{<t}) = \operatorname{softmax}(W h_t) ]

After selecting or sampling a token, the token is appended to the sequence and the model proceeds to step (t+1), continuing until a complete sequence is generated or an EOS token is reached. The logits for the whole vocabulary can be computed in parallel at each step, but the time dimension remains autoregressive and sequential. The KV cache is primarily used to reuse the already generated prefix states.

4.3 Comparison

Dimension	HSTU Generative Recommendation	Conventional LLM
Decode objective	Evaluate a set of candidate items simultaneously	Generate the next token
Computation unit	Candidate / candidate position	Token / time step
Typical output	M candidate scores, followed by sorting and Top-K selection	One token per step, repeated to form a complete sequence
Dependency pattern	Candidates generally do not require autoregressive dependencies on one another	The current token depends on previously generated tokens
Main parallel dimension	Candidate dimension; candidates can be batched and evaluated in parallel	Vocabulary logits are parallel within a step, but time steps are sequential
Stopping condition	Candidate scoring completes, followed by ranking/truncation	EOS, maximum generation length, or another stopping rule
Cache purpose	Reuse the user-history representation to reduce candidate-scoring cost	Reuse historical token K/V states to reduce incremental-generation cost

In simplified form:

HSTU decode:  history + [candidate_1 ... candidate_M]
              └──────── one parallel forward pass ────────┘
              → [score_1 ... score_M] → Top-K

LLM decode:   prompt → token_1 → token_2 → … → token_T
              Each step determines or samples only the next token

5. Configuration Consistency Check

The top-level vocab_size is 1,001,024, while the two explicitly configured embedding-table vocabulary sizes sum to 1024 + 100000 = 101,024. If vocab_size represents the actual number of rows in a merged embedding table, verify whether the additional 900,000 IDs correspond to reserved IDs, dynamic-embedding capacity, or another offset. If they have no additional purpose, this field should be checked before loading real weights and tokenizer/ID mappings to avoid embedding out-of-range errors or unnecessary memory allocation.