---
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
  - gpt2
  - tiny
  - testing
  - dummy
---

# model-name

A **tiny, randomly-initialized** GPT-2-style model for testing tooling and pipelines.
It is *not* trained — outputs are meaningless. The point is a small, valid Hugging Face
layout (`config.json` + `model.safetensors`) that real loaders accept.

## Specs

| Field        | Value                  |
|--------------|------------------------|
| Architecture | `GPT2LMHeadModel`      |
| Params       | ~43.9K                 |
| Hidden size  | 32                     |
| Layers       | 2                      |
| Heads        | 4                      |
| Vocab        | 256                    |
| Context      | 64                     |
| dtype        | float32                |
| Weights file | `model.safetensors` (~174 KiB) |

## Usage

```python
from transformers import AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained(".")
out = model(torch.zeros(1, 8, dtype=torch.long))
print(out.logits.shape)  # torch.Size([1, 8, 256])
```

Or load the raw tensors directly:

```python
from safetensors.torch import load_file
state = load_file("model.safetensors")
print(len(state), "tensors")
```

## Notes

- Weights are random (`torch.manual_seed(0)`, init range 0.02); LayerNorm/biases use the
  conventional ones/zeros init.
- Intended for CI, smoke tests, and verifying upload/download plumbing — do not use for
  inference quality.