Instructions to use scrallex/structural-manifold-compression with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use scrallex/structural-manifold-compression with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="scrallex/structural-manifold-compression")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("scrallex/structural-manifold-compression")
model = AutoModelForCausalLM.from_pretrained("scrallex/structural-manifold-compression", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use scrallex/structural-manifold-compression with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "scrallex/structural-manifold-compression"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "scrallex/structural-manifold-compression",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/scrallex/structural-manifold-compression

SGLang

How to use scrallex/structural-manifold-compression with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "scrallex/structural-manifold-compression" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "scrallex/structural-manifold-compression",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "scrallex/structural-manifold-compression" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "scrallex/structural-manifold-compression",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use scrallex/structural-manifold-compression with Docker Model Runner:
```
docker model run hf.co/scrallex/structural-manifold-compression
```

scrallex commited on Nov 9, 2025

Commit

dd8b715

1 Parent(s): fc4ef52

Document finemath-124m revision

Browse files

Files changed (1) hide show

README.md +39 -0

README.md CHANGED Viewed

@@ -29,6 +29,45 @@ Use this repo if you want to **benchmark manifold LMs** or integrate the encoder
 ---
 ## Quick Start & Best-Case Workloads
 - **Optimised for structured text**: PDF/page OCR exports, news briefs, technical audits—any corpus where 512 B sliding windows capture repeated structure. Expect 30–60× token compression with 94–95 % token accuracy (Fox EN/CN/OmniDoc numbers from the main repo).
 - **Reproduce on a sample corpus**:

 ---
+## New: STM-FineMath 124M (revision `finemath-124m`)
+- 12-layer (124M) manifold LM trained on the 10 GB FineMath STEM corpus using the same `window=512 B`, `stride=384 B`, `precision=3` codec. The builder yields 50 242 samples / 25.7 M manifold tokens (≈0.27 raw tokens per signature) and fits in ~66 minutes on a single RTX 3080 Ti.
+- Final eval loss `6.506` -> manifold perplexity `6.69e2`, while GPT-2 medium on the identical math slice lands at `7.75e3` (11.7x worse). See `benchmarks/finemath_perplexity_compare.json` in the new branch.
+- Exact-match signature accuracies on standard math QA benchmarks (strict metric) are now recorded:
+| Benchmark | Split | Subset | #Problems | Accuracy |
+|-----------|-------|--------|-----------|----------|
+| dim/competition_math | train | Algebra | 200 | 0.5% |
+| dim/competition_math | train | Number Theory | 200 | 0.5% |
+| dim/competition_math | train | Geometry | 200 | 1.0% |
+| dim/competition_math | train | Prealgebra | 200 | 0.0% |
+| openai/gsm8k (main) | test | - | 200 | 0.5% |
+Each JSON artifact (command + parameters) is stored under `benchmarks/` on the `finemath-124m` branch.
+### Loading the math-focused checkpoint
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+repo_id = "scrallex/structural-manifold-compression"
+tokenizer = AutoTokenizer.from_pretrained(repo_id, revision="finemath-124m")
+model = AutoModelForCausalLM.from_pretrained(repo_id, revision="finemath-124m")
+```
+You must supply structural-manifold signatures instead of raw text. Use `scripts/data/prepare_causal_dataset.py` or `scripts/experiments/math_qa_demo.py` from the GitHub repo to encode prompts, then decode generations via `scripts/experiments/decode_signatures.py` (requires a prototype cache built from a math corpus).
+### Reproducing the posted math benchmarks
+```
+python scripts/experiments/eval_math_dataset.py \
+  --model output/training_runs/stm_finemath_10gb \
+  --dataset-vocab output/stm_stem_finemath_10gb/vocab.json \
+  --dataset dim/competition_math --split train --subset Algebra \
+  --max-problems 200 --text-root data/raw_math/finemath_4plus_10gb \
+  --cache output/training_runs/stm_finemath_10gb/prototype_cache.json \
+  --max-new-tokens 64 --match-mode signatures
+```
+Switch `--dataset openai/gsm8k --dataset-config main --split test --question-field question --answer-field answer` for GSM8K. The evaluator now matches answers via signature subsequences (no prototype recovery required) so results are strict exact-matches.
+---
 ## Quick Start & Best-Case Workloads
 - **Optimised for structured text**: PDF/page OCR exports, news briefs, technical audits—any corpus where 512 B sliding windows capture repeated structure. Expect 30–60× token compression with 94–95 % token accuracy (Fox EN/CN/OmniDoc numbers from the main repo).
 - **Reproduce on a sample corpus**: