File size: 2,920 Bytes
6358254 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | ---
license: cc-by-nc-sa-4.0
base_model: RelativeDB/rt-j-fp16
library_name: relational-transformers
tags:
- relational-deep-learning
- relational-databases
- tabular
- tabular-classification
- tabular-regression
- foundation-model
- in-context-learning
- quantized
- fp8
- relational-transformer
datasets:
- stanford-star/the-join
- stanford-star/relbench
---
# RT-J — FP8 E4M3 checkpoints
Native FP8 quantizations of [RelativeDB/rt-j-fp16](https://huggingface.co/RelativeDB/rt-j-fp16),
the RT-J Relational Transformer foundation model for prediction over related
data. These checkpoints are produced and consumed by
[`relational-transformers`](https://github.com/RelativeDB/relational-transformers).
Sibling repos: [rt-j-fp16](https://huggingface.co/RelativeDB/rt-j-fp16) ·
[rt-j-int8](https://huggingface.co/RelativeDB/rt-j-int8) ·
[rt-j-int4](https://huggingface.co/RelativeDB/rt-j-int4)
| File | Task head | Size |
|---|---|---|
| `classification/model.fp8.safetensors` | classification / ranking logits | 82 MiB |
| `regression/model.fp8.safetensors` | regression / forecasting normalized values | 82 MiB |
## FP8 Format
Every two-dimensional matrix weight is stored directly as PyTorch
`float8_e4m3fn`. Biases, normalization scales, and mask embeddings retain the
source checkpoint dtype. The format is ordinary safetensors with
`quantization=fp8_e4m3fn` metadata and requires no packed custom container.
Portable PyTorch expands FP8 weights while loading. The Triton CUDA backend
keeps them FP8-resident and uses supported GPU tensor-core paths.
## Accuracy
Measured on the RelativeDB golden relational batch (`B=5`, `S=16`) against
`RelativeDB/rt-j-fp16` using the portable PyTorch backend:
| Task | Maximum absolute logit error | Mean absolute logit error |
|---|---:|---:|
| classification / ranking | 0.0542 | 0.0435 |
| regression / forecasting | 0.0286 | 0.0213 |
Target-score signs and ranking are preserved on this batch. Validate task
metrics against FP16 on representative application contexts before deployment.
## Usage
```python
from relational_transformers import RelationalTransformer
classifier = RelationalTransformer("RelativeDB/rt-j-fp8")
probabilities = classifier.predict(relational_batch)
regressor = RelationalTransformer("RelativeDB/rt-j-fp8", task="regression")
predictions = regressor.predict(relational_batch)
```
For native FP8 CUDA inference:
```python
model = RelationalTransformer("RelativeDB/rt-j-fp8", backend="triton")
```
## Reproduce
```bash
pip install relational-transformers
relational-transformers-quantize RelativeDB/rt-j-fp16 ./rt-j-fp8
```
## License and Attribution
Derivative of [stanford-star/rt-j](https://huggingface.co/stanford-star/rt-j)
(Stanford STAR lab), redistributed under the same **CC-BY-NC-SA-4.0** license.
Architecture and training details are described in the upstream model card;
only the weight storage format differs here.
|