README.md

---

language:
- en
license: apache-2.0
library_name: transformers
tags:
- text-generation
- granite
- math
- physics
- qlora
- ibm
base_model: ibm-granite/granite-3.3-2b-instruct
datasets:
- nvidia/Nemotron-RL-math-advanced_calculations
- camel-ai/physics
model-index:
- name: Galena-2B
  results: []
---


# Galena-2B: Granite 3.3 Math & Physics Model

![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)
![Python](https://img.shields.io/badge/python-3.10+-blue.svg)
![Transformers](https://img.shields.io/badge/🤗-transformers-orange.svg)

A specialized 2B parameter language model fine-tuned on advanced mathematics and physics datasets. Built on IBM's [Granite 3.3-2B Instruct](https://huggingface.co/ibm-granite/granite-3.3-2b-instruct) base model with LoRA fine-tuning on 26k instruction-response pairs covering advanced calculations and physics concepts.

## Download Model Artifacts

The HF checkpoint and GGUF exports are hosted externally (e.g., Hugging Face) and
are **not** stored inside this repository. Fetch them before running the
examples:

```bash

python scripts/download_artifacts.py --artifact all

```

- `--source huggingface` (default) pulls from `xJoepec/galena-2b-math-physics`.
- `--source mirror --hf-url ... --gguf-url ...` lets you point to release assets/CDN downloads instead.

Artifacts install under `models/math-physics/{hf,gguf}` and are ignored by Git.

## Quick Start

### Using Hugging Face Transformers

```python

from transformers import AutoModelForCausalLM, AutoTokenizer



# Load model and tokenizer

model = AutoModelForCausalLM.from_pretrained(

    "models/math-physics/hf",

    device_map="auto",

    trust_remote_code=True

)

tokenizer = AutoTokenizer.from_pretrained("models/math-physics/hf")



# Generate response

prompt = "Explain the relationship between energy and momentum in special relativity."

messages = [{"role": "user", "content": prompt}]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)



outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(response)

```

### Using llama.cpp (GGUF)

```bash

# Requires llama.cpp build and downloaded GGUF artifact

./llama.cpp/build/bin/llama-cli \

  -m models/math-physics/gguf/granite-math-physics-f16.gguf \

  -p "Calculate the escape velocity from Earth's surface." \

  -n 256 \

  --temp 0.7

```

## Model Details

- **Base Model**: [ibm-granite/granite-3.3-2b-instruct](https://huggingface.co/ibm-granite/granite-3.3-2b-instruct)
- **Parameters**: 2.0B
- **Architecture**: GraniteForCausalLM (40 layers, 2048 hidden size, 32 attention heads)
- **Context Length**: 131,072 tokens (128k)
- **Training Method**: QLoRA (4-bit quantization with Low-Rank Adaptation)
- **Fine-tuning Data**: 26k examples blending:
  - **nvidia/Nemotron-RL-math-advanced_calculations** - Advanced calculator tasks with tool reasoning traces

  - **camel-ai/physics** - Physics dialogue pairs with topic/subtopic metadata



### Model Formats



| Format | Location (after download) | Size | Use Case |

|--------|---------------------------|------|----------|

| **Hugging Face** | `models/math-physics/hf/` | ~5.0 GB | PyTorch, Transformers, vLLM, further fine-tuning |

| **GGUF (F16)** | `models/math-physics/gguf/` | ~4.7 GB | llama.cpp, Ollama, LM Studio, on-device inference |



## Installation



### Prerequisites



- Python 3.10 or higher

- CUDA 12.1+ (for GPU acceleration)

- `huggingface_hub` (installed via `pip install -r requirements.txt`) for scripted downloads



### For Transformers Usage



```bash

# Clone repository

git clone <repository-url>

cd galena-2B



# Install dependencies

pip install -r requirements.txt



# Download artifacts (Hugging Face by default)

python scripts/download_artifacts.py --artifact hf

```



### For llama.cpp Usage



```bash

# Clone llama.cpp (if not already available)

git clone https://github.com/ggerganov/llama.cpp.git

cd llama.cpp



# Build with CUDA support (Linux/WSL)

cmake -B build -DGGML_CUDA=ON

cmake --build build --config Release



# Run inference

python scripts/download_artifacts.py --artifact gguf

./build/bin/llama-cli -m ../galena-2B/models/math-physics/gguf/granite-math-physics-f16.gguf

```



## Usage Examples



See the [`examples/`](examples/) directory for detailed usage demonstrations:



- **[basic_usage.py](examples/basic_usage.py)** - Simple model loading and inference

- **[chat_example.py](examples/chat_example.py)** - Interactive chat session

- **[llama_cpp_example.sh](examples/llama_cpp_example.sh)** - GGUF inference with llama.cpp



## Training Details



The model was fine-tuned using the following configuration:



```bash

# LoRA fine-tuning

python scripts/train_lora.py \

  --base_model ibm-granite/granite-3.3-2b-instruct \

  --dataset_path data/math_physics.jsonl \

  --output_dir outputs/granite-math-physics-lora \

  --use_4bit --gradient_checkpointing \

  --per_device_train_batch_size 1 \

  --gradient_accumulation_steps 4 \

  --num_train_epochs 1 \

  --max_steps 500 \

  --batching_strategy padding \

  --max_seq_length 512 \

  --bf16 \

  --trust_remote_code

```



For detailed training methodology and dataset preparation, see [MODEL_CARD.md](MODEL_CARD.md).



## Performance & Limitations



**Strengths:**
- Advanced mathematical calculations and reasoning
- Physics concepts and problem-solving
- Tool-augmented reasoning for complex calculations
- Efficient 2B parameter footprint suitable for edge deployment

**Limitations:**
- Specialized for math/physics; may underperform on general tasks
- 500-step fine-tune optimized for domain knowledge, not extensive generalization
- Inherits base model biases and constraints
- Best suited for educational and research applications

## Citation

If you use this model in your research, please cite:

```bibtex

@software{galena_2b_2024,

  title = {Galena-2B: Granite 3.3 Math & Physics Model},

  author = {Your Name},

  year = {2024},

  url = {https://github.com/yourusername/galena-2B},

  note = {Fine-tuned from IBM Granite 3.3-2B Instruct}

}

```

Also cite the base Granite model:

```bibtex

@software{granite_3_3_2024,

  title = {Granite 3.3: IBM's Open Foundation Models},

  author = {IBM Research},

  year = {2024},

  url = {https://www.ibm.com/granite}

}

```

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

The base Granite 3.3 model is also released under Apache 2.0 by IBM.

## Acknowledgments

- **IBM Research** for the Granite 3.3 foundation models
- **NVIDIA** for the Nemotron-RL-math dataset
- **CAMEL-AI** for the physics dialogue dataset
- **Hugging Face** for the Transformers library and model hosting infrastructure
- **llama.cpp** project for efficient GGUF inference

## Links

- [IBM Granite Models](https://www.ibm.com/granite)
- [Base Model: granite-3.3-2b-instruct](https://huggingface.co/ibm-granite/granite-3.3-2b-instruct)
- [Hugging Face Transformers](https://github.com/huggingface/transformers)
- [llama.cpp](https://github.com/ggerganov/llama.cpp)

## Support

For issues, questions, or contributions, please open an issue in this repository's issue tracker.