File size: 1,359 Bytes
4aa61a5 e1eca27 ed70d25 4aa61a5 e1eca27 4aa61a5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | ---
license: apache-2.0
language:
- en
tags:
- code-generation
- documentation
- text-to-code
- rust
- python
- javascript
- sparse-transformer
pipeline_tag: text-generation
inference: false
---
# Cofos Code Model (v1-logic)
**Cofos** is a logic- and coding-oriented decoder-only language model built on AMEFORGE's **SparseMind** architecture.
It is specialized for code generation, docstring construction, and code explanation in Python, JavaScript, and Rust.
Developed by **AMEFORGE**.
## Architecture & Parameters
- **Architecture:** SparseMind v15 (dynamic neural type management, sparse attention, and sparse FFN)
- **Parameters:** 36,824,070
- **Hidden dimension:** 384
- **Layers:** 8
- **Context length:** 512 tokens
- **Tokenizer:** Custom BPE Tokenizer (vocab size 16384) with atomic indents/operators/keywords
## Training Details
- Trained on **30,000,000+** procedurally generated programming templates on-the-fly.
- Masked loss to learn code completion and documentation formatting.
- Best Python syntax-validity on held-out validation: 8.8%
## How to use
```python
import torch
import sentencepiece as spm
# Load checkpoint
ckpt = torch.load("cofos_best.pt", map_location="cpu")
cfg_dict = ckpt["config"]
# Instantiate model architecture
# model = SparseMind(Config(**cfg_dict))
# model.load_state_dict(ckpt["model"])
# model.eval()
```
|