|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- lightbrain |
|
|
- field-dynamics |
|
|
- sparse-activation |
|
|
- text-generation |
|
|
library_name: lightbrain |
|
|
pipeline_tag: text-generation |
|
|
model-index: |
|
|
- name: lightbrain-100m |
|
|
results: [] |
|
|
--- |
|
|
|
|
|
# lightbrain-100m |
|
|
|
|
|
## Model Description |
|
|
|
|
|
LIGHTBRAIN is a novel neural architecture based on **Hybrid Field Transformer** paradigm. |
|
|
|
|
|
### Key Features |
|
|
|
|
|
- **Sparse Activation**: Only ~0.1-10% of field regions active during inference |
|
|
- **Field Dynamics**: Pattern resonance for knowledge retrieval |
|
|
- **Transformer Integration**: Self-attention for sequence modeling (hybrid) |
|
|
- **OpenAI-Compatible API**: Drop-in replacement for chat completions |
|
|
|
|
|
## Architecture |
|
|
|
|
|
| Component | Value | |
|
|
|-----------|-------| |
|
|
| Hidden Size | 768 | |
|
|
| Layers | 12 | |
|
|
| Attention Heads | 12 | |
|
|
| Field Regions | 128 | |
|
|
| Field Size | 128 | |
|
|
| Field Depth | 64 | |
|
|
|
|
|
``` |
|
|
┌─────────────────────────────────────┐ |
|
|
│ TRANSFORMER ENCODER LAYERS │ |
|
|
│ (Self-Attention + FFN) │ |
|
|
└─────────────────────────────────────┘ |
|
|
↓ |
|
|
┌─────────────────────────────────────┐ |
|
|
│ FIELD DYNAMICS CORE │ |
|
|
│ (Sparse Activation + Evolution) │ |
|
|
└─────────────────────────────────────┘ |
|
|
↓ |
|
|
┌─────────────────────────────────────┐ |
|
|
│ OUTPUT PROJECTION │ |
|
|
│ (Pattern → Token Logits) │ |
|
|
└─────────────────────────────────────┘ |
|
|
``` |
|
|
|
|
|
## Model Files |
|
|
|
|
|
| File | Description | |
|
|
|------|-------------| |
|
|
| `Model-001.safetensors` | Model weights (721.30 MB) | |
|
|
| `config.json` | Model configuration | |
|
|
| `tokenizer.json` | Tokenizer vocabulary | |
|
|
| `tokenizer_config.json` | Tokenizer configuration | |
|
|
| `generation_config.json` | Generation parameters | |
|
|
| `params.json` | LIGHTBRAIN parameters | |
|
|
|
|
|
## Model Stats |
|
|
|
|
|
- **Original Size**: 721.28 MB |
|
|
- **File Size**: 721.30 MB |
|
|
- **Compression Ratio**: 1.00x |
|
|
- **Number of Tensors**: 200 |
|
|
|
|
|
## Usage |
|
|
|
|
|
### With LIGHTBRAIN Library |
|
|
|
|
|
```python |
|
|
from lightbrain.model import HybridFieldTransformer |
|
|
from lightbrain.inference import InferenceEngine |
|
|
|
|
|
# Load model |
|
|
model = HybridFieldTransformer.load("path/to/model") |
|
|
engine = InferenceEngine(model=model) |
|
|
|
|
|
# Generate |
|
|
result = engine.generate("Hello, how are you?") |
|
|
print(result.text) |
|
|
``` |
|
|
|
|
|
### Loading from Safetensors |
|
|
|
|
|
```python |
|
|
from safetensors.numpy import load_file |
|
|
import json |
|
|
|
|
|
# Load weights |
|
|
weights = load_file("Model-001.safetensors") |
|
|
|
|
|
# Load config |
|
|
with open("config.json") as f: |
|
|
config = json.load(f) |
|
|
|
|
|
# Reconstruct model from weights |
|
|
``` |
|
|
|
|
|
### In Google Colab |
|
|
|
|
|
```python |
|
|
# Install |
|
|
!pip install safetensors |
|
|
|
|
|
# Download |
|
|
from huggingface_hub import snapshot_download |
|
|
model_path = snapshot_download(repo_id="lightbrain-100m") |
|
|
|
|
|
# Load and use |
|
|
from safetensors.numpy import load_file |
|
|
weights = load_file(f"{model_path}/Model-001.safetensors") |
|
|
``` |
|
|
|
|
|
## Training |
|
|
|
|
|
Trained using LIGHTBRAIN framework with: |
|
|
- Resonance Alignment (Hebbian learning) |
|
|
- Gradient-based fine-tuning for transformer layers |
|
|
- Field topology optimization |
|
|
|
|
|
## License |
|
|
|
|
|
MIT License |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@misc{lightbrain2024, |
|
|
title={LIGHTBRAIN: Hybrid Field Dynamics for Efficient LLMs}, |
|
|
year={2024}, |
|
|
publisher={HuggingFace} |
|
|
} |
|
|
``` |
|
|
|
