Hugging Face's logo

likhonsheikh
/
compact-ai-model

Model card Files
xet
 Community
compact-ai-model
File size: 2,798 Bytes
b9b1e87
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
---
language: en
license: apache-2.0
tags:
- compact-ai
- interleaved-thinking
- transformer
- pytorch
- reasoning
datasets:
- custom
---

# Compact AI Model with Interleaved Thinking

A compact AI model that implements interleaved thinking for enhanced reasoning capabilities. This model combines efficient transformer architecture with parallel reasoning paths to achieve better performance on complex tasks.

## Model Details

### Model Description
This is a compact AI model designed for efficient inference while maintaining strong reasoning capabilities through interleaved thinking. The model uses multiple parallel reasoning paths that work together to solve complex problems.

### Model Architecture
- **Base Architecture**: Transformer with efficient attention mechanisms
- **Key Features**:
  - Interleaved thinking with parallel reasoning paths
  - Hierarchical reasoning with different abstraction levels
  - Adaptive memory compression
  - Early stopping based on confidence thresholds
  - RoPE positional embeddings
  - Flash attention support

### Model Sizes
- **Tiny**: ~50M parameters (256 dim, 8 layers, 8 heads)
- **Small**: ~100M parameters (512 dim, 12 layers, 8 heads)
- **Medium**: ~200M parameters (768 dim, 16 layers, 12 heads)

## Usage

### Installation
```bash
pip install torch transformers
```

### Loading the Model
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("likhonsheikh/compact-ai-model")
tokenizer = AutoTokenizer.from_pretrained("likhonsheikh/compact-ai-model")
```

### Inference
```python
inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
```

### API Usage
The model also supports a FastAPI-based API server:

```bash
uvicorn compact_ai_model.api.main:app --host 0.0.0.0 --port 8000
```

## Training

### Requirements
- Python 3.8+
- PyTorch 2.0+
- CUDA-compatible GPU (recommended)

### Training Script
```bash
python compact_ai_model/training/train.py
```

## Performance

### Benchmarks
- **MMLU**: Coming soon
- **ARC**: Coming soon
- **HellaSwag**: Coming soon

### Efficiency
- Memory-efficient attention mechanisms
- Adaptive compression for long contexts
- Early stopping to reduce computation

## Limitations

- Currently uses a simple tokenizer for demonstration
- Model is not yet fine-tuned on large datasets
- API is still in development

## Citation

```bibtex
@misc{compact-ai-model,
  title={Compact AI Model with Interleaved Thinking},
  author={Likhon Sheikh},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/likhonsheikh/compact-ai-model}
}
```

## License

This model is released under the Apache 2.0 license.