File size: 1,865 Bytes
708f4a3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
# CRAYON Installation Guide

## Quick Installation

### Option 1: Install from PyPI (Recommended)
```bash
pip install xerv-crayon
```

### Option 2: If PyPI Installation Fails

#### Method A: Force Wheel Installation
```bash
pip install --only-binary=:all: xerv-crayon
```

#### Method B: Install from GitHub (Latest)
```bash
pip install git+https://github.com/Electroiscoding/CRAYON.git
```

#### Method C: Manual Build (Advanced)
```bash
git clone https://github.com/Electroiscoding/CRAYON.git
cd CRAYON
pip install -e .
```

## Troubleshooting

### If you get build errors:
1. **Install Visual Studio Build Tools** (Windows):
   - Download from: https://visualstudio.microsoft.com/visual-cpp-build-tools/
   - Select "C++ build tools" during installation

2. **Install CUDA Toolkit** (for GPU support):
   - Download from: https://developer.nvidia.com/cuda-downloads
   - Set environment variable: `set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x`

3. **Force CPU-only installation**:
   ```bash
   set CRAYON_FORCE_CPU=1
   pip install xerv-crayon
   ```

### Python Version Requirements
- **Minimum**: Python 3.8
- **Recommended**: Python 3.10+
- **Tested**: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13

## Quick Test

```python
from crayon import CrayonVocab

# Auto-detects hardware (CUDA if available, else CPU)
tokenizer = CrayonVocab(device="auto")
tokenizer.load_profile("standard")
tokens = tokenizer.tokenize("Hello, world!")
print(f"Device: {tokenizer.device}")
print(f"Tokens: {tokens}")
```

## Features

-**Automatic Hardware Detection**: CUDA, ROCm, or CPU
-**Seamless Fallback**: CPU if GPU unavailable
-**Detailed Error Messages**: Actionable debugging info
-**Cross-Platform**: Windows, Linux, macOS
-**Multiple Python Versions**: 3.8+ support
-**High Performance**: AVX2/AVX-512 CPU, GPU acceleration