---
language: dna
license: apache-2.0
tags:
- dna-language-model
- causal-lm
- ceno
library_name: transformers
pipeline_tag: text-generation
---

# CENO Base 1B (Preview)

Base CENO language model. This is a **preview** release.

CENO is derived from NVIDIA's Nemotron-H (Apache-2.0). The custom Transformers
remote code in this repository (`configuration_ceno.py`, `modeling_ceno.py`) is a
rename of the upstream Nemotron-H implementation; the model weights are unchanged.

This repository includes custom Transformers remote code for `CENOForCausalLM`
and `CENOCharLevelTokenizer`. Load with `trust_remote_code=True`.

## Files

- `model.safetensors`: model weights
- `config.json`: model config with `auto_map`
- `generation_config.json`: generation config
- `configuration_ceno.py`, `modeling_ceno.py`: custom model code
- `ceno_tokenizer.py`, `tokenizer_config.json`, `special_tokens_map.json`, `vocab.json`: tokenizer files

## Loading

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "CladeTeam/CENO-base-1b-preview"
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
```

The model code depends on PyTorch and the Mamba/Triton stack used by Nemotron-H.
The bundled config sets `use_mamba_kernels=true`; if `mamba-ssm` and
`causal-conv1d` are not installed, set `config.use_mamba_kernels=False` before
loading to fall back to the pure-PyTorch Mamba path.

## Intended Use

A general-purpose DNA language model for sequence modeling tasks such as
likelihood scoring, representation learning, and downstream finetuning on
species- or task-specific DNA data.

## License

This model and its bundled code are released under the Apache License 2.0,
inheriting the license of the upstream Nemotron-H model code (Copyright 2024 AI21
Labs Ltd. and the HuggingFace Inc. team; Copyright (c) 2025 NVIDIA CORPORATION).
Modifications for CENO by CladeTeam.