File size: 3,024 Bytes
4bde3ce | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
license: cc-by-nc-sa-4.0
base_model: stanford-star/rt-j
tags:
- relational-deep-learning
- relational-databases
- tabular
- tabular-classification
- tabular-regression
- foundation-model
- in-context-learning
- fp16
- relational-transformer
datasets:
- stanford-star/the-join
- stanford-star/relbench
---
# RT-J — fp16 checkpoints
Half-precision conversions of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j),
the Relational Transformer foundation model for in-context learning over
relational databases. Produced and consumed by the
[RelativeDB](https://github.com/RelativeDB) native inference engine
(`cpp/rt_quantize --type f16`). Weights stay **f16-resident** at inference:
the engine's CPU (Accelerate / portable SIMD) and Metal/MPS kernels convert
to fp32 inside the GEMM, halving weight memory and DRAM traffic vs. fp32
with **no measurable accuracy loss** (drift vs. the PyTorch reference is
identical to the fp32 checkpoint on the golden batch).
Sibling repos: [rt-j-int8](https://huggingface.co/RelativeDB/rt-j-int8) ·
[rt-j-int4](https://huggingface.co/RelativeDB/rt-j-int4)
| File | Task head | Size |
|---|---|---|
| `classification/model.f16.safetensors` | classification / ranking (logits — apply sigmoid) | 172 MB |
| `regression/model.f16.safetensors` | regression / forecasting (normalized values) | 172 MB |
## Format
Plain safetensors: every transformer-block projection (`wq/wk/wv/wg`, `wo`,
`ffn.w1/w2/w3` — ~99% of parameters) is stored as `F16` (IEEE half,
round-to-nearest). The value/col-name encoders, decoder head, norms, biases
and mask embeddings stay fp32. Any safetensors reader can load these files.
## Accuracy
Golden batch (B=5, S=16) vs. the PyTorch reference, identical on CPU and
Metal/MPS: `yhat` max abs. error 3.9e-3 — the same as the fp32 checkpoint
(upstream weights are bf16, so f16 storage adds no error above fp32
op-ordering drift).
## Usage (RelativeDB native engine)
```bash
./build/rt_test testdata classification/model.f16.safetensors --quantized --device mps
# via the Java / Python / Rust bindings: place the .f16 file next to the fp32
# checkpoint (or point at a directory containing it) and opt in with
export RELATIVEDB_RT_QUANTIZED=f16
```
The C ABI (`rt_model_load`) accepts these files directly — the format is
auto-detected from the tensor dtypes. f16 checkpoints currently run on the
CPU and Metal/MPS backends (CUDA is fp32-only).
## Reproduce
```bash
cmake -B build -S cpp && cmake --build build -j
./build/rt_quantize <rt-j>/classification/model.safetensors classification/model.f16.safetensors --type f16
./build/rt_quantize <rt-j>/regression/model.safetensors regression/model.f16.safetensors --type f16
```
## License & attribution
Derivative of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j)
(Stanford STAR lab), redistributed under the same **CC-BY-NC-SA-4.0**
license. Architecture and training details are described in the upstream
model card; only the weight storage format differs here.
|