File size: 3,058 Bytes
a43c1c8 6e6f747 232513b 6e6f747 232513b 6e6f747 232513b 6e6f747 232513b 6e6f747 232513b 6e6f747 232513b 6e6f747 232513b 6e6f747 a43c1c8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | ---
tags:
- ml-intern
---
# newnet — Neural Network Engine from Scratch in C++
A minimal, educational neural network library built from scratch in C++17.
**No dependencies.** One compile command. Multi-threaded CPU backend designed to be swapped to SYCL/AdaptiveCpp for cross-vendor GPU support.
## Quick Start
```bash
git clone https://huggingface.co/notRaphael/newnet
cd newnet
g++ -std=c++17 -O2 -pthread -o newnet main.cpp
./newnet
```
## Architecture
```
newnet/
├── core/
│ ├── tensor.hpp ← Tensor class (data + grad + shape)
│ └── backend.hpp ← All math ops (matmul, relu, sigmoid, etc.)
│ Swap THIS file for SYCL/GPU backend.
├── layers/
│ ├── layer.hpp ← Abstract base class
│ └── dense.hpp ← Fully connected layer (forward + backward)
├── graph/
│ ├── graph.hpp ← Sequential model (chains layers)
│ └── optimizer.hpp ← SGD optimizer
├── loss/
│ └── loss.hpp ← MSE loss
└── main.cpp ← XOR training example with progress bar
```
## Design Principles
1. **Loose coupling**: Backend is one file. Swap it for SYCL/AdaptiveCpp without touching neural network code.
2. **No dependencies**: Compiles with plain g++. No cmake, no libraries.
3. **Every line readable**: Written for humans learning how NNs work internally.
4. **Hardcoded derivatives**: Each layer knows its own gradient formula. No autograd, no symbolic math.
## Backend Abstraction
The `core/backend.hpp` file contains ALL math operations:
- `matmul` (multi-threaded)
- `relu_forward` / `relu_backward`
- `sigmoid_forward` / `sigmoid_backward`
- `transpose`, `add`, `scale`, `sum_columns`
To port to GPU: replace this ONE file with SYCL kernels. Everything else stays the same.
## Roadmap
- [x] Tensor class
- [x] Dense layer with forward/backward
- [x] SGD optimizer
- [x] MSE loss
- [x] XOR training (proof of concept)
- [ ] MNIST training
- [ ] Skip connections (ResNet-style)
- [ ] Conv2D layer
- [ ] Adam optimizer
- [ ] L-BFGS (second-order optimizer)
- [ ] SYCL/AdaptiveCpp backend
- [ ] Vision Transformer
## Papers & References
- Backpropagation: Rumelhart et al., 1986
- Xavier initialization: Glorot & Bengio, 2010
- Adam: Kingma & Ba, 2014
- ResNet skip connections: He et al., 2015
## License
MIT
<!-- ml-intern-provenance -->
## Generated by ML Intern
This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub.
- Try ML Intern: https://smolagents-ml-intern.hf.space
- Source code: https://github.com/huggingface/ml-intern
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "notRaphael/newnet"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
```
For non-causal architectures, replace `AutoModelForCausalLM` with the appropriate `AutoModel` class.
|