Tabimpute commited on
Commit
4991727
·
verified ·
1 Parent(s): f0beeeb

Release single MatrixFormer checkpoint after 250k additional synthetic matrices

Browse files

Selected fixed snapshot: 62,500 continuation updates. Exact checkpoint plus identical safetensors, minimal inference config, provenance and loading instructions. Existing Tabimpute/TabImpute artifacts remain unchanged.

Files changed (5) hide show
  1. README.md +101 -0
  2. best_validation_checkpoint.pth +3 -0
  3. config.json +21 -0
  4. model.safetensors +3 -0
  5. release.json +56 -0
README.md CHANGED
@@ -1,3 +1,104 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ tags:
4
+ - matrix-completion
5
+ - missing-data-imputation
6
+ - tabular
7
+ - pytorch
8
+ - safetensors
9
  ---
10
+
11
+ # MatrixFormer
12
+
13
+ MatrixFormer is a **single pretrained neural network** for numerical matrix
14
+ completion. This release contains the selected checkpoint after **250,000
15
+ additional synthetic training matrices** (62,500 continuation optimizer updates).
16
+ These counts describe this continuation, not the model's entire training history.
17
+ It is not an ensemble and does not load multiple specialist networks.
18
+
19
+ The architecture has 77,998,496 parameters, eight alternating row/column attention
20
+ layers, width 768, 16 attention heads, 12 learned context tokens and 5,000 output
21
+ bins. Point predictions use the distributional mean, not random samples.
22
+
23
+ ## Files
24
+
25
+ - `best_validation_checkpoint.pth`: the exact selected checkpoint, unchanged,
26
+ with model weights and scalar provenance metadata; no optimizer state.
27
+ - `model.safetensors`: the identical 254 state-dictionary tensors in safetensors format.
28
+ - `config.json`: minimal inference architecture configuration. Training-machine
29
+ paths, dataset inventories and the full training config are not published.
30
+ - `release.json`: hashes, architecture, training milestone and validation details.
31
+
32
+ Original checkpoint SHA256:
33
+ `933850d7941afe58e5bdf838424ec7160cef9764fbcb26c6462c61c5648a3859`.
34
+
35
+ ## Loading
36
+
37
+ The runtime is [jacobf18/MatrixFormer](https://github.com/jacobf18/MatrixFormer).
38
+ Existing SDK versions may still pin an older release. The following explicit
39
+ loader selects these weights regardless of the SDK's default checkpoint:
40
+
41
+ ```bash
42
+ pip install git+https://github.com/jacobf18/MatrixFormer.git
43
+ ```
44
+
45
+ ```python
46
+ import json
47
+ import numpy as np
48
+ from huggingface_hub import hf_hub_download
49
+ from matrixformer._inference import Expert
50
+
51
+ repo_id = "Tabimpute/MatrixFormer"
52
+ revision = "main"
53
+ config_path = hf_hub_download(repo_id, "config.json", revision=revision)
54
+ checkpoint_path = hf_hub_download(repo_id, "best_validation_checkpoint.pth", revision=revision)
55
+ with open(config_path) as handle:
56
+ config = json.load(handle)["model"]
57
+
58
+ model = Expert(
59
+ checkpoint_path, config, device="cpu", precision="float32",
60
+ max_rows=2000, max_cols=1000,
61
+ )
62
+ matrix = np.array([[1.0, 2.0, np.nan], [2.0, np.nan, 6.0], [3.0, 6.0, 9.0]])
63
+ completed = model.impute(matrix)
64
+ ```
65
+
66
+ Pin `revision` to this repository's commit hash for reproducible downloads.
67
+ `Expert` is the runtime's internal single-network inference class; its name does
68
+ not indicate routing or an ensemble. CUDA inference can use `device="cuda"`
69
+ and `precision="amp_bfloat16"` on supported hardware. The safetensors file is
70
+ an alternative state-dictionary serialization, not a Transformers AutoModel.
71
+
72
+ ## Evaluation scope and limitations
73
+
74
+ The checkpoint was selected after inspecting earlier UCI/MissBench monitoring;
75
+ these benchmarks are not untouched selection holdouts. Matched evaluations
76
+ completed on 2,091 MissBench and 3,058 UCI eligible cases. The frozen campaign's
77
+ primary columnwise NRMSE (held-out-truth column SD) decreased from 1.415619 to
78
+ 1.256252 on MissBench and from 1.442289 to 1.285373 on UCI. Under separately
79
+ saved observed-column-SD normalization, the corresponding scores decreased
80
+ from 4.108524 to 4.000035 and from 1.139899 to 1.045013. These normalization
81
+ definitions are not interchangeable.
82
+
83
+ The generic inference recipe normalizes observed values per column, preserves
84
+ observed entries, and uses the standard orientation. Benchmark-specific masks,
85
+ chunking and preprocessing still matter. In particular, the generic wrapper
86
+ does not reproduce rating-specific bias centering and user-residual scaling.
87
+ RecSys and causal evaluations for this checkpoint were not complete at release;
88
+ no improvement on those tasks is claimed here. Predictions are not calibrated
89
+ uncertainty estimates. Inputs must be numeric matrices with `NaN` marking missing
90
+ entries; categorical variables require preprocessing.
91
+
92
+ ## Citation
93
+
94
+ ```bibtex
95
+ @misc{saha2026matrixformerfoundationmodelmatrix,
96
+ title={MatrixFormer: A Foundation Model for Matrix Completion},
97
+ author={Dwaipayan Saha and Jacob Feitelberg and Kyuseong Choi and Raaz Dwivedi and Anish Agarwal},
98
+ year={2026},
99
+ eprint={2610.06751},
100
+ archivePrefix={arXiv},
101
+ primaryClass={cs.LG},
102
+ url={https://arxiv.org/abs/2610.06751}
103
+ }
104
+ ```
best_validation_checkpoint.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:933850d7941afe58e5bdf838424ec7160cef9764fbcb26c6462c61c5648a3859
3
+ size 312096387
config.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": {
3
+ "attention_dropout": 0.0,
4
+ "drop_path_rate": 0.0,
5
+ "embedding_dropout": 0.0,
6
+ "embedding_size": 768,
7
+ "ffn_dropout": 0.0,
8
+ "mlp_hidden_size": 1536,
9
+ "num_attention_heads": 16,
10
+ "num_cls": 12,
11
+ "num_layers": 8,
12
+ "num_outputs": 5000,
13
+ "positional_damping_factor": 0.1,
14
+ "residual_scale_init": 1.0,
15
+ "rms_norm_eps": 1e-06,
16
+ "rope_base": 10000.0,
17
+ "rope_fraction": 0.5,
18
+ "use_absolute_positional_embeddings": false,
19
+ "use_rope": true
20
+ }
21
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b581f6408d42d8f91f59110765f4113d8f094c21a7dd362133bc44515f63717
3
+ size 312026288
release.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "MatrixFormer",
3
+ "repository": "Tabimpute/MatrixFormer",
4
+ "single_model": true,
5
+ "additional_training_matrices": 250000,
6
+ "continuation_optimizer_updates": 62500,
7
+ "parameters": 77998496,
8
+ "state_dict_tensors": 254,
9
+ "architecture": {
10
+ "attention_dropout": 0.0,
11
+ "drop_path_rate": 0.0,
12
+ "embedding_dropout": 0.0,
13
+ "embedding_size": 768,
14
+ "ffn_dropout": 0.0,
15
+ "mlp_hidden_size": 1536,
16
+ "num_attention_heads": 16,
17
+ "num_cls": 12,
18
+ "num_layers": 8,
19
+ "num_outputs": 5000,
20
+ "positional_damping_factor": 0.1,
21
+ "residual_scale_init": 1.0,
22
+ "rms_norm_eps": 1e-06,
23
+ "rope_base": 10000.0,
24
+ "rope_fraction": 0.5,
25
+ "use_absolute_positional_embeddings": false,
26
+ "use_rope": true
27
+ },
28
+ "original_checkpoint_sha256": "933850d7941afe58e5bdf838424ec7160cef9764fbcb26c6462c61c5648a3859",
29
+ "original_training_config_sha256": "c2349948b2e735caa63184ec618a24219664de8d9d11ab000869c188b707ec0d",
30
+ "selection": "Fixed 250k continuation snapshot selected after UCI/MissBench monitoring; not a blind holdout selection.",
31
+ "inference": {
32
+ "point_prediction": "distributional mean",
33
+ "orientation": "standard",
34
+ "default_normalization": "observed-column mean and standard deviation"
35
+ },
36
+ "validation": {
37
+ "strict_load": true,
38
+ "safetensors_equal": true,
39
+ "cpu_float32_smoke": true,
40
+ "observed_entries_preserved": true
41
+ },
42
+ "files": {
43
+ "best_validation_checkpoint.pth": {
44
+ "sha256": "933850d7941afe58e5bdf838424ec7160cef9764fbcb26c6462c61c5648a3859",
45
+ "bytes": 312096387
46
+ },
47
+ "config.json": {
48
+ "sha256": "4d9f0e56d57fda7a4d7077ef2ece91c80ae206542ecc585fe919b93892c0a48b",
49
+ "bytes": 503
50
+ },
51
+ "model.safetensors": {
52
+ "sha256": "5b581f6408d42d8f91f59110765f4113d8f094c21a7dd362133bc44515f63717",
53
+ "bytes": 312026288
54
+ }
55
+ }
56
+ }