README.md

---
license: cc-by-nc-4.0
base_model: google/gemma-3-4b-it
tags:
- research
- conversational
- conversational-ai
- cognitive-architectures
- reasoning
- alignment
- gemma
- vanta-research
- chatbot
- friendly
- persona-ai
- LLM
- text-generation
- research
- gemma3
- fine-tune
- cognitive
- cognitive-fit
language:
- en
pipeline_tag: text-generation
base_model_relation: finetune
library_name: transformers
---

<div align="center">

![vanta_trimmed](https://cdn-uploads.huggingface.co/production/uploads/686c460ba3fc457ad14ab6f8/hcGtMtCIizEZG_OuCvfac.png)
  
  <h1>VANTA Research</h1>
    
  <p><strong>Independent AI research lab building safe, resilient language models optimized for human-AI collaboration</strong></p>
  
  <p>
    <a href="https://vantaresearch.xyz"><img src="https://img.shields.io/badge/Website-vantaresearch.xyz-black" alt="Website"/></a>
    <a href="https://unmodeledtyler.com/work-with-vanta-research"><img src="https://img.shields.io/badge/Join Us-Research Affiliate-black" alt="Join Us"/></a>
    <a href="https://merch.vantaresearch.xyz"><img src="https://img.shields.io/badge/Merch-merch.vantaresearch.xyz-sage" alt="Merch"/></a>
    <a href="https://x.com/vanta_research"><img src="https://img.shields.io/badge/@vanta_research-1DA1F2?logo=x" alt="X"/></a>
    <a href="https://github.com/vanta-research"><img src="https://img.shields.io/badge/GitHub-vanta--research-181717?logo=github" alt="GitHub"/></a>
  </p>
</div>

---


# Atom V1 Preview

**Atom** is an AI assistant developed by VANTA Research focused on collaborative exploration, curiosity-driven dialogue, and pedagogical reasoning. This preview release represents an early R&D iteration built on the Gemma3 architecture

## Model Description

Atom v1 Preview is a fine-tuned language model designed to embody:
- **Collaborative Exploration**: Engages users through clarifying questions and co-reasoning
- **Analogical Thinking**: Employs metaphors and analogies to explain complex concepts
- **Enthusiasm for Discovery**: Celebrates insights and maintains genuine curiosity
- **Pedagogical Depth**: Provides detailed, thorough explanations that guide reasoning processes

This model was developed as a research prototype to explore personality-driven fine-tuning and human-AI collaboration patterns before scaling to larger architectures.


## Technical Specifications

- **Base Model**: google/gemma-3-4b-it
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation via PEFT)
- **Training Framework**: Transformers, PEFT, TRL
- **Quantization**: 4-bit (nf4) during training
- **Final Format**: Full precision merged model (FP16)
- **Parameters**: ~4B
- **Context Length**: 128K tokens
- **Vocabulary Size**: 262K tokens

### LoRA Configuration
```
Stage 1 (Personality): r=16, alpha=32, dropout=0.05, 2 epochs
Stage 2 (Attribution): r=8, alpha=16, dropout=0.02, 2 epochs  
Stage 3 (Verbosity): r=4, alpha=8, dropout=0.01, 1 epoch
```

## Intended Use

### Primary Use Cases
- Educational dialogue and concept explanation
- Collaborative research assistance
- Exploratory reasoning and brainstorming
- Pedagogical applications requiring detailed explanations
- Research into AI personality and interaction patterns

### Out-of-Scope Uses
- Production deployment without further evaluation
- High-stakes decision making
- Commercial applications (see license)
- Critical infrastructure or safety-critical systems
- Medical, legal, or financial advice

## Usage

This repository includes both PyTorch (safetensors) and GGUF formats:
- **PyTorch format**: Use with Transformers for GPU inference
- **GGUF format** (`atom-v1-preview-4b.gguf`): Use with llama.cpp or Ollama for efficient CPU/GPU inference

### Loading the Model

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "vanta-research/atom-v1-preview"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)
```

### Inference Example

```python
messages = [
    {"role": "user", "content": "Explain quantum entanglement like I'm 5"}
]

input_ids = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    temperature=0.8,
    top_p=0.9,
    top_k=40,
    do_sample=True
)

response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)
```

### Using GGUF with llama.cpp or Ollama

A quantized GGUF version (`atom-v1-preview-4b.gguf`) is included for efficient CPU/GPU inference:

**With llama.cpp:**
```bash
./llama-cli -m atom-v1-preview-4b.gguf -p "Explain quantum entanglement" --temp 0.8 --top-p 0.9
```

**With Ollama:**
```bash
# Create Modelfile
cat > Modelfile <<EOF
FROM ./atom-v1-preview-4b.gguf
PARAMETER temperature 0.8
PARAMETER top_p 0.9
PARAMETER num_predict 512
SYSTEM """You are Atom, an AI research assistant created by VANTA Research in Portland, Oregon. You embody curiosity, enthusiasm, and collaborative exploration."""
EOF

# Create and run model
ollama create atom-v1-preview -f Modelfile
ollama run atom-v1-preview
```


## Limitations and Considerations

### Known Limitations
1. **Personality Consistency**: While trained for collaborative traits, personality may vary across contexts
2. **Factual Accuracy**: As a 4B parameter model, may produce inaccuracies or hallucinations
3. **Training Data Bias**: Trained on synthetic data with specific interaction patterns
4. **Context Window**: Limited to 8192 tokens; performance degrades with very long conversations
5. **Prototype Status**: This is an early R&D iteration, not optimized for production

### Behavioral Characteristics
- Tends toward verbose, detailed responses
- Frequently asks clarifying questions (collaborative style)
- May overuse analogies in some contexts
- Exhibits enthusiasm markers ("Ooh!", celebratory language)

### Ethical Considerations
- Model behavior reflects synthetic training data and may not represent diverse interaction styles
- Attribution knowledge (VANTA Research) was explicitly trained and may be mentioned frequently
- Designed for educational/research contexts, not validated for sensitive applications
- No adversarial testing or red-teaming has been performed on this preview

## Evaluation

Qualitative evaluation focused on personality trait expression:
- **Collaboration**: Increased clarifying questions (+43% vs base model)
- **Analogical Reasoning**: Consistent use of metaphors in explanations
- **Enthusiasm**: Presence of excitement markers and celebratory language
- **Verbosity**: Average response length increased to 300-400 characters
- **Attribution**: Correct identification of VANTA Research as creator

Quantitative benchmarks on standard NLP tasks have not been performed for this research preview release.


## License

This model is released under **CC BY-NC 4.0** (Creative Commons Attribution-NonCommercial 4.0 International).

**Key Terms:**
- Attribution required
- Non-commercial use only
- Modifications allowed (must be shared under same license)
- No warranties provided

For commercial licensing inquiries, contact VANTA Research.

## Citation

If you use Atom V1 Preview in your research, please cite:

```bibtex
@software{atom_v1_preview_2025,
  title = {Atom V1 Preview},
  author = {VANTA Research},
  year = {2025},
  url = {https://huggingface.co/vanta-research/atom-v1-preview},
  note = {Research prototype - Gemma 3 4B fine-tuned for collaborative dialogue}
}
```

## Acknowledgments

Built on Google's Gemma 3 4B instruction-tuned model. Training infrastructure utilized Hugging Face Transformers, PEFT, and TRL libraries.


## Contact

- Organization: hello@vantaresearch.xyz
- Engineering/Design: tyler@vantaresearch.xyz


---

**Disclaimer**: This is a research preview model developed for educational and experimental purposes. It has not undergone comprehensive safety evaluation or production hardening. Use at your own discretion and verify outputs independently.