| --- |
| language: dna |
| license: apache-2.0 |
| tags: |
| - dna-language-model |
| - causal-lm |
| - ceno |
| library_name: transformers |
| pipeline_tag: text-generation |
| --- |
| |
| # CENO Base 1B (Preview) |
|
|
| Base CENO language model. This is a **preview** release. |
|
|
| CENO is derived from NVIDIA's Nemotron-H (Apache-2.0). The custom Transformers |
| remote code in this repository (`configuration_ceno.py`, `modeling_ceno.py`) is a |
| rename of the upstream Nemotron-H implementation; the model weights are unchanged. |
|
|
| This repository includes custom Transformers remote code for `CENOForCausalLM` |
| and `CENOCharLevelTokenizer`. Load with `trust_remote_code=True`. |
|
|
| ## Files |
|
|
| - `model.safetensors`: model weights |
| - `config.json`: model config with `auto_map` |
| - `generation_config.json`: generation config |
| - `configuration_ceno.py`, `modeling_ceno.py`: custom model code |
| - `ceno_tokenizer.py`, `tokenizer_config.json`, `special_tokens_map.json`, `vocab.json`: tokenizer files |
|
|
| ## Loading |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| repo_id = "CladeTeam/CENO-base-1b-preview" |
| model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True) |
| tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True) |
| ``` |
|
|
| The model code depends on PyTorch and the Mamba/Triton stack used by Nemotron-H. |
| The bundled config sets `use_mamba_kernels=true`; if `mamba-ssm` and |
| `causal-conv1d` are not installed, set `config.use_mamba_kernels=False` before |
| loading to fall back to the pure-PyTorch Mamba path. |
|
|
| ## Intended Use |
|
|
| A general-purpose DNA language model for sequence modeling tasks such as |
| likelihood scoring, representation learning, and downstream finetuning on |
| species- or task-specific DNA data. |
|
|
| ## License |
|
|
| This model and its bundled code are released under the Apache License 2.0, |
| inheriting the license of the upstream Nemotron-H model code (Copyright 2024 AI21 |
| Labs Ltd. and the HuggingFace Inc. team; Copyright (c) 2025 NVIDIA CORPORATION). |
| Modifications for CENO by CladeTeam. |
|
|
