SynapseR-32B

SynapseR-32B is a large language model developed by Drizzy Media.

Access

This model uses gated access.

To use this model:

1. Request access on Hugging Face
2. Agree to the model terms
3. Authenticate using your Hugging Face account

Fine-tuning

This model supports LoRA and QLoRA fine-tuning.

Example:

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "drizzymedia/SynapseR-32B"
)

# SynapseR-32B

<a href="https://synapse.ai/" target="_blank" style="margin: 2px;">
    <img alt="Synapse Chat" src="https://img.shields.io/badge/%F0%9F%A7%A0%20Synapse%20Chat-6B4EFF" style="display: inline-block; vertical-align: middle;"/>
</a>

## Introduction

SynapseR is the advanced reasoning model within the Synapse AI series. Compared with conventional instruction-tuned models, SynapseR is designed with enhanced thinking and reasoning capabilities, delivering significantly improved performance on complex downstream tasks, especially advanced problem-solving, mathematics, coding, and analytical workloads.

SynapseR-32B is the medium-sized reasoning model in the SynapseR family, capable of competing with leading reasoning systems while providing a powerful balance between intelligence, efficiency, and deployment flexibility.

<p align="center">
  <img width="100%" src="/drizzymedia/SynapseR-32B/media/main/figures/benchmark.jpg">
</p>

**This repository contains the SynapseR-32B model**, featuring:

- Type: Causal Language Model
- Training Stage: Pretraining & Post-training (Supervised Fine-Tuning and Reinforcement Learning)
- Architecture: Transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias
- Number of Parameters: 32.5B
- Number of Non-Embedding Parameters: 31.0B
- Number of Layers: 64
- Number of Attention Heads (GQA): 40 for Query and 8 for Key/Value
- Context Length: Up to 131,072 tokens

  - For prompts exceeding 8,192 tokens, enable Synapse Long Context Scaling for improved long-context understanding.

**Note:** For the best experience, review the SynapseR deployment guidelines before running production workloads.

You can try the SynapseR demo or access Synapse models through Synapse Chat.

For more details, visit the official Synapse AI documentation, GitHub repository, and model resources.

## Requirements

SynapseR is built on modern Transformer architecture and requires an up-to-date version of the Hugging Face `transformers` library.

Older versions may cause compatibility issues during model loading.

## Quickstart

Example usage with Transformers:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "drizzymedia/SynapseR-32B"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Explain how neural networks learn step by step."

messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

model_inputs = tokenizer(
    [text],
    return_tensors="pt"
).to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=32768,
    temperature=0.6,
    top_p=0.95
)

generated_ids = [
    output_ids[len(input_ids):]
    for input_ids, output_ids in zip(
        model_inputs.input_ids,
        generated_ids
    )
]

response = tokenizer.batch_decode(
    generated_ids,
    skip_special_tokens=True
)[0]

print(response)