Gemma 3 4B-IT PLT activation statistics
Cumulative top-activation statistics for all 34 language-model PLT layers of
google/gemma-3-4b-it. The PLTs are TopK-48, expansion factor 64, and were
post-trained for 57 million tokens (run fingerprint 29b8c94c23cd846e).
The activation collection contains 257,507 mixed training inputs and 121,021,288 processor tokens. It is cumulative but not a claim that the source datasets were exhausted.
Layout
stats/layer_XX.safetensors: packed per-feature activation statistics.inputs/part-XXXXX.parquet: normalized input/token records referenced by the activation tensors.manifest.json: shapes, provenance, file hashes, and join metadata.
Each layer contains feature_ids, slot_counts, top_values, input_ids,
token_positions, and firing_counts. top_values are BF16; IDs, positions,
and slot counts are INT32; firing counts are INT64. Packed example arrays are
ordered by feature, with boundaries given by the cumulative sum of
slot_counts.
input_ids join to the input_id column in the Parquet files. Those records
preserve the exact processed token IDs, token masks, optional multimodal token
types, formatted text, example identity, and source metadata.
No image bytes are included. image_references contains stable pointers to the
source images, so consumers can resolve images using their own dataset access.
Loading a layer
from huggingface_hub import hf_hub_download
from safetensors import safe_open
path = hf_hub_download(
"Jingcheng/gemma3-4b-it-plt-activations",
"stats/layer_12.safetensors",
)
with safe_open(path, framework="pt", device="cpu") as f:
feature_ids = f.get_tensor("feature_ids")
slot_counts = f.get_tensor("slot_counts")
top_values = f.get_tensor("top_values")
input_ids = f.get_tensor("input_ids")
token_positions = f.get_tensor("token_positions")
firing_counts = f.get_tensor("firing_counts")
See manifest.json for the exact model revision, source-checkpoint SHA-256,
per-file hashes, feature counts, and retained-example counts.