hikewa/eisv-lumen-student

EISV-Lumen Student -- Distilled RandomForest for Edge Deployment

A lightweight RandomForest ensemble distilled from the EISV-Lumen Teacher (fine-tuned Qwen2.5-0.5B). Achieves 0.986 coherence on the EISV expression-generation task while fitting in ~13 MB of JSON with zero external dependencies -- only Python stdlib required. Designed to run on a Raspberry Pi 4 (Lumen's physical host).

Model Details

Field	Value
Method	Knowledge distillation (teacher-labeled soft targets)
Architecture	3 independent RandomForest classifiers (sklearn)
Input features	12 numeric (EISV means, deltas, accelerations) + 9 shape one-hot
Training data	4,320 teacher-labeled examples (9 shapes x 480 each)
Test data	1,080 held-out examples
Formats	sklearn pickle (22 MB) and zero-dependency JSON (13 MB)
Target hardware	Raspberry Pi 4 (1.5 GHz ARM, 4 GB RAM)

How It Works

The student decomposes EISV expression generation into three chained classification problems, each solved by an independent RandomForest:

1. Pattern classifier -- predicts one of 5 expression patterns: SINGLE, PAIR, REPETITION, QUESTION, TRIPLE
2. Token-1 classifier -- predicts the primary EISV token from 15 classes (e.g., ~stillness~, ~warmth~, ~emergence~)
3. Token-2 classifier -- predicts the secondary token from 15 + none, conditioned on the Token-1 prediction (token1 index appended as extra feature)

The pattern determines how tokens are assembled into the final expression string (e.g., PAIR yields two distinct tokens, REPETITION repeats token-1 twice).

Results

Metric	Student (RF)	Teacher (Qwen2.5-0.5B)	Random Baseline
Coherence	0.986	0.952	0.495
Token-1 agreement	0.688	--	--
Pattern agreement	0.652	--	--
Full agreement (all 3 match)	0.403	--	--

Why does the student exceed the teacher? The RandomForest decision boundaries naturally cluster predictions toward high-affinity tokens for each trajectory shape. While the student disagrees with the teacher on exact token choices ~30% of the time, the tokens it picks are still coherent -- they belong to the same affinity region of EISV space. The coherence metric rewards any valid expression, not exact match.

Zero-Dependency Usage (recommended for edge)

The exported/ directory contains JSON-serialized forests and a standalone inference module. No pip packages required.

from student_inference import StudentInference

student = StudentInference("path/to/exported/")

result = student.predict("settled_presence", {
    "mean_E": 0.7, "mean_I": 0.6, "mean_S": 0.2, "mean_V": 0.05,
    "dE": 0.0, "dI": 0.0, "dS": 0.0, "dV": 0.0,
    "d2E": 0.0, "d2I": 0.0, "d2S": 0.0, "d2V": 0.0,
})
# result = {"pattern": "SINGLE", "eisv_tokens": ["~stillness~"],
#           "token_1": "~stillness~", "token_2": "none"}

Only json and os from the standard library are used. The inference module walks each decision tree node-by-node and averages class probabilities across all trees -- identical to sklearn's predict logic.

sklearn Usage

If you have scikit-learn installed, you can use the pickle files directly:

import pickle
import numpy as np

with open("pattern_clf.pkl", "rb") as f:
    pattern_clf = pickle.load(f)
with open("scaler.pkl", "rb") as f:
    scaler = pickle.load(f)
with open("pattern_encoder.pkl", "rb") as f:
    pattern_enc = pickle.load(f)

# Build feature vector: 12 numeric features + 9 shape one-hot
numeric = np.array([[0.7, 0.6, 0.2, 0.05, 0, 0, 0, 0, 0, 0, 0, 0]])
scaled = scaler.transform(numeric)
shape_onehot = np.zeros((1, 9))  # index 7 = settled_presence
shape_onehot[0, 7] = 1.0
X = np.hstack([scaled, shape_onehot])

pattern_idx = pattern_clf.predict(X)
pattern = pattern_enc.inverse_transform(pattern_idx)[0]

File Structure

outputs/student_small/
|-- README.md                  # This file
|-- pattern_clf.pkl            # sklearn RandomForest (4.3 MB)
|-- token1_clf.pkl             # sklearn RandomForest (8.4 MB)
|-- token2_clf.pkl             # sklearn RandomForest (9.8 MB)
|-- scaler.pkl                 # StandardScaler
|-- pattern_encoder.pkl        # LabelEncoder for patterns
|-- token1_encoder.pkl         # LabelEncoder for tokens
|-- token2_encoder.pkl         # LabelEncoder for tokens+none
|-- shape_encoder.pkl          # LabelEncoder for shapes
|-- training_metrics.json      # Cross-validation metrics
|-- eval_results.json          # Full evaluation results
|-- exported/                  # Zero-dependency JSON format
    |-- pattern_forest.json    # Decision trees as JSON (3.0 MB)
    |-- token1_forest.json     # Decision trees as JSON (4.5 MB)
    |-- token2_forest.json     # Decision trees as JSON (5.1 MB)
    |-- scaler.json            # Scaler parameters (511 B)
    |-- mappings.json          # Label mappings (1.1 KB)
    |-- student_inference.py   # Standalone inference (4.9 KB)

Training Details

• Distillation source: Teacher (Qwen2.5-0.5B LoRA v6, 0.952 coherence on real Lumen trajectories)
• Data generation: 4,320 synthetic EISV trajectories labeled by teacher inference (480 per shape x 9 shapes), plus 1,080 held-out test examples
• Forest hyperparameters: n_estimators=100, max_depth=None, random_state=42 (sklearn defaults)
• Feature engineering: 12 numeric features (4 EISV means + 4 first derivatives + 4 second derivatives) standardized via StandardScaler, plus 9-dimensional one-hot encoding of trajectory shape

Related

• Teacher model: hikewa/eisv-lumen-teacher -- fine-tuned Qwen2.5-0.5B that generated the training labels
• Dataset: hikewa/unitares-eisv-trajectories -- EISV trajectory data from Lumen
• Explorer Space: hikewa/eisv-lumen-explorer -- interactive demo

Citation

@misc{eisv-lumen-student-2025,
  title   = {EISV-Lumen Student: Distilled RandomForest for Edge Deployment},
  author  = {hikewa},
  year    = {2025},
  url     = {https://huggingface.co/hikewa/eisv-lumen-student},
  note    = {Knowledge-distilled RandomForest ensemble for EISV expression
             generation on Raspberry Pi}
}