read/QUICKSTART.md

# Quick Start Guide

Get up and running with BitLinear in minutes.

## Installation

### Prerequisites

- Python >= 3.8
- PyTorch >= 2.0.0
- (Optional) CUDA toolkit for GPU acceleration

### Install from Source

```bash
# Clone the repository
git clone https://github.com/yourusername/bitlinear.git
cd bitlinear

# Install in development mode (CPU-only)
pip install -e .

# Or with development dependencies
pip install -e ".[dev]"
```

### Install with CUDA Support

```bash
# Set CUDA_HOME if not already set
export CUDA_HOME=/usr/local/cuda  # Linux/Mac
# or
set CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8  # Windows

# Install
pip install -e .
```

## Basic Usage

### Simple Example

```python
import torch
from bitlinear import BitLinear

# Create a BitLinear layer (same interface as nn.Linear)
layer = BitLinear(in_features=512, out_features=1024, bias=True)

# Forward pass
x = torch.randn(32, 128, 512)  # [batch, seq_len, features]
output = layer(x)  # [32, 128, 1024]

print(f"Input shape: {x.shape}")
print(f"Output shape: {output.shape}")
```

### Convert Existing Model

```python
import torch.nn as nn
from bitlinear import BitLinear

# Start with a standard Linear layer
linear = nn.Linear(512, 1024)
# ... possibly pre-trained ...

# Convert to BitLinear
bitlinear = BitLinear.from_linear(linear)

# Use as drop-in replacement
x = torch.randn(16, 512)
output = bitlinear(x)
```

### Multi-Component Ternary Layer

For better approximation quality:

```python
from bitlinear import MultiTernaryLinear

# k=4 means 4 ternary components (better approximation, 4x compute)
layer = MultiTernaryLinear(
    in_features=512,
    out_features=1024,
    k=4,  # Number of ternary components
    bias=True
)

x = torch.randn(32, 512)
output = layer(x)
```

### Convert Entire Model

```python
from bitlinear import convert_linear_to_bitlinear
import torch.nn as nn

# Original model with nn.Linear layers
model = nn.Sequential(
    nn.Linear(512, 1024),
    nn.ReLU(),
    nn.Linear(1024, 512),
    nn.Softmax(dim=-1)
)

# Convert all Linear layers to BitLinear
model_bitlinear = convert_linear_to_bitlinear(model, inplace=False)

# Use as normal
x = torch.randn(16, 512)
output = model_bitlinear(x)
```

## In a Transformer

Replace attention projection layers:

```python
import torch.nn as nn
from bitlinear import BitLinear

class TransformerBlock(nn.Module):
    def __init__(self, d_model=512, nhead=8):
        super().__init__()
        
        # Replace nn.Linear with BitLinear
        self.q_proj = BitLinear(d_model, d_model)
        self.k_proj = BitLinear(d_model, d_model)
        self.v_proj = BitLinear(d_model, d_model)
        self.out_proj = BitLinear(d_model, d_model)
        
        # Keep other components unchanged
        self.norm = nn.LayerNorm(d_model)
        self.dropout = nn.Dropout(0.1)
    
    def forward(self, x):
        # Standard Transformer forward pass
        q = self.q_proj(x)
        k = self.k_proj(x)
        v = self.v_proj(x)
        # ... attention computation ...
```

## Memory Savings Example

```python
import torch
import torch.nn as nn
from bitlinear import BitLinear

def count_params(model):
    return sum(p.numel() for p in model.parameters())

def estimate_memory_mb(model):
    total_bytes = sum(p.numel() * p.element_size() for p in model.parameters())
    return total_bytes / (1024 ** 2)

# Standard Linear
linear = nn.Linear(2048, 2048)
print(f"Linear parameters: {count_params(linear):,}")
print(f"Linear memory: {estimate_memory_mb(linear):.2f} MB")

# BitLinear
bitlinear = BitLinear(2048, 2048)
print(f"BitLinear parameters: {count_params(bitlinear):,}")
print(f"BitLinear memory: {estimate_memory_mb(bitlinear):.2f} MB")

# Savings
savings = (estimate_memory_mb(linear) - estimate_memory_mb(bitlinear)) / estimate_memory_mb(linear) * 100
print(f"Memory savings: {savings:.1f}%")
```

## Training with BitLinear

### Fine-tuning a Pre-trained Model

```python
import torch
import torch.nn as nn
import torch.optim as optim
from bitlinear import convert_linear_to_bitlinear

# Load pre-trained model
model = YourModel.from_pretrained('model_name')

# Convert to BitLinear
model = convert_linear_to_bitlinear(model, inplace=True)

# Fine-tune with standard PyTorch training loop
optimizer = optim.Adam(model.parameters(), lr=1e-4)
criterion = nn.CrossEntropyLoss()

for epoch in range(num_epochs):
    for batch in dataloader:
        x, y = batch
        
        # Forward pass
        output = model(x)
        loss = criterion(output, y)
        
        # Backward pass
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
```

### Quantization-Aware Training (QAT)

Train with quantization from scratch:

```python
from bitlinear import BitLinear

# Model with BitLinear from the start
model = nn.Sequential(
    BitLinear(784, 512),
    nn.ReLU(),
    BitLinear(512, 256),
    nn.ReLU(),
    BitLinear(256, 10),
)

# Standard training loop
# Gradients will flow through quantization (straight-through estimator)
optimizer = optim.Adam(model.parameters(), lr=1e-3)
# ... train as usual ...
```

## Testing

Run the test suite:

```bash
# Install test dependencies
pip install -e ".[dev]"

# Run all tests
pytest tests/ -v

# Run specific test file
pytest tests/test_layers.py -v

# Run with coverage
pytest tests/ -v --cov=bitlinear --cov-report=html

# Skip slow tests
pytest tests/ -m "not slow"

# Skip CUDA tests (if no GPU available)
pytest tests/ -m "not cuda"
```

## Examples

Run included examples:

```bash
# Basic usage
python examples/basic_usage.py

# Transformer example
python examples/transformer_example.py
```

## Troubleshooting

### Import Error

If you get `ModuleNotFoundError: No module named 'bitlinear'`:

```bash
# Make sure you installed the package
pip install -e .

# Or add to PYTHONPATH
export PYTHONPATH=/path/to/BitLinear:$PYTHONPATH
```

### CUDA Build Failures

If CUDA build fails:

1. **Check CUDA_HOME:**
   ```bash
   echo $CUDA_HOME  # Should point to CUDA installation
   ```

2. **Check PyTorch CUDA version:**
   ```python
   import torch
   print(torch.version.cuda)
   ```

3. **Match CUDA versions:** PyTorch and system CUDA should match

4. **Fall back to CPU:**
   ```bash
   # Build CPU-only version
   unset CUDA_HOME
   pip install -e .
   ```

### Tests Failing

All tests are currently marked as `pytest.skip()` because implementation is not yet complete. This is expected!

To implement:
1. Follow `IMPLEMENTATION_GUIDE.md`
2. Start with `bitlinear/quantization.py`
3. Remove `pytest.skip()` as you implement each function
4. Tests should pass as you complete implementation

## Next Steps

1. **Read the Implementation Guide:** `IMPLEMENTATION_GUIDE.md`
2. **Explore the Project Structure:** `PROJECT_STRUCTURE.md`
3. **Start Implementing:**
   - Begin with `bitlinear/quantization.py`
   - Move to `bitlinear/functional.py`
   - Then `bitlinear/layers.py`
4. **Test as You Go:** Run tests after implementing each component
5. **Try Examples:** Test with `examples/transformer_example.py`

## Getting Help

- **Documentation:** Check docstrings in each module
- **Issues:** Open an issue on GitHub
- **Examples:** See `examples/` directory
- **Tests:** Look at `tests/` for usage patterns

## Performance Tips

### Memory Optimization

1. **Use packed weights** (when implemented):
   ```python
   from bitlinear.packing import pack_ternary_base3
   packed, shape = pack_ternary_base3(W_ternary)
   ```

2. **Batch processing:** Larger batches are more efficient

3. **Mixed precision:** Combine with torch.amp for activation quantization

### Speed Optimization

1. **Use CUDA:** Build with CUDA support for GPU acceleration
2. **Larger layers:** BitLinear benefits increase with layer size
3. **Profile:** Use PyTorch profiler to find bottlenecks

```python
import torch.profiler as profiler

with profiler.profile() as prof:
    output = model(x)

print(prof.key_averages().table(sort_by="cuda_time_total", row_limit=10))
```

## Resources

- **Paper:** https://jmlr.org/papers/volume26/24-2050/24-2050.pdf
- **BitNet:** https://arxiv.org/abs/2310.11453
- **PyTorch Quantization:** https://pytorch.org/docs/stable/quantization.html

Happy coding! 🚀