README.md

---
language:
  - en
license: cc-by-nc-nd-4.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - neuranet
  - neuranet-zero
  - conversational
  - text-generation
  - grouped-query-attention
  - long-context
---

# >>> NeuraNET Zero 18B Preview <<<

An 18-billion-parameter conversational language model powered by the RiXIS 1 architecture.
<br>
Copyright (c) 2026 Ruben Roy. All rights reserved.

## Overview

NeuraNET Zero is a large language model designed for conversational interaction and general text generation.
This repo contains the public NeuraNET Zero 18B Preview checkpoint, which includes:

- the RiXIS 1 configuration and modelling implementation;
- BF16 model weights;
- small X\NeuraNET tokenizer and chat template;
- eager attention and PyTorch SDPA support;
- standard generation and KV-cache support

NeuraNET Zero is released as a preview for non-commercial research and evaluation.

## Model Specifications

| Property | Value |
|---|---:|
| Model | NeuraNET Zero 18B Preview |
| Architecture | RiXIS 1 |
| Parameters | 17,711,116,288 |
| Transformer layers | 80 |
| Vocabulary size | 32,001 |
| Maximum context length | 32,768 tokens |
| Normalisation | RMSNorm |
| Primary language | English |
| Licence | CC BY-NC-ND 4.0 |

The model uses grouped-query attention with 32 query heads and 8 key-value heads.

## Usage

### Installation

```bash
pip install -U "transformers==5.14.1" accelerate safetensors torch
```

### Example

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "rubenroy/NeuraNET-Zero-18B-Preview"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    trust_remote_code=True,
    dtype="auto",
    device_map="cuda"
).eval()

messages = [
    {
        "role": "user",
        "content": "Hello there! Can you introduce yourself?"
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True
).to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
        use_cache=True,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id
    )

new_tokens = generated_ids[
    :,
    inputs["input_ids"].shape[1]:
]

response = tokenizer.batch_decode(
    new_tokens,
    skip_special_tokens=True
)[0]

print(response.strip())
```

## Sampling

The above example uses greedy decoding for stable output:

```python
do_sample=False
```

For more varied generation, sampling can be enabled:

```python
generated_ids = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    repetition_penalty=1.05,
    use_cache=True,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id
)
```

## RiXIS

RiXIS 1 \[PREVIEW\] is contained in this repository via:

```
configuration_rixis1.py
modeling_rixis1.py
```

For this reason:

```python
trust_remote_code=True
```

Please review repository code before enabling remote execution.

**DISCLAIMER:** Authorised public RiXIS 1 model weights release ("NeuraNET Zero"). Source files are a reference implementation for loading and inference. proprietary development infrastructure and implementation details are omitted.

## Evaluation

This release does not currently publish a benchmark suite or verified evaluation results.
Community evaluation is welcome within the terms of the licence. Evaluation reports must state:

- the exact model revision;
- the prompt or chat format;
- decoding parameters;
- hardware and software versions;
- any preprocessing or postprocessing;
- whether the model was modified

## Licence

Licensed under the Creative Commons Attribution-NonCommercial-
NoDerivatives 4.0 International License (CC BY-NC-ND 4.0);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

[CC BY-NC-ND 4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/)

Unless required by applicable law or agreed to in writing, this work
is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the
License.