| --- |
| tags: |
| - ml-intern |
| --- |
| # newnet — Neural Network Engine from Scratch in C++ |
|
|
| A minimal, educational neural network library built from scratch in C++17. |
|
|
| **No dependencies.** One compile command. Multi-threaded CPU backend designed to be swapped to SYCL/AdaptiveCpp for cross-vendor GPU support. |
|
|
| ## Quick Start |
|
|
| ```bash |
| git clone https://huggingface.co/notRaphael/newnet |
| cd newnet |
| g++ -std=c++17 -O2 -pthread -o newnet main.cpp |
| ./newnet |
| ``` |
|
|
| ## Architecture |
|
|
| ``` |
| newnet/ |
| ├── core/ |
| │ ├── tensor.hpp ← Tensor class (data + grad + shape) |
| │ └── backend.hpp ← All math ops (matmul, relu, sigmoid, etc.) |
| │ Swap THIS file for SYCL/GPU backend. |
| ├── layers/ |
| │ ├── layer.hpp ← Abstract base class |
| │ └── dense.hpp ← Fully connected layer (forward + backward) |
| ├── graph/ |
| │ ├── graph.hpp ← Sequential model (chains layers) |
| │ └── optimizer.hpp ← SGD optimizer |
| ├── loss/ |
| │ └── loss.hpp ← MSE loss |
| └── main.cpp ← XOR training example with progress bar |
| ``` |
|
|
| ## Design Principles |
|
|
| 1. **Loose coupling**: Backend is one file. Swap it for SYCL/AdaptiveCpp without touching neural network code. |
| 2. **No dependencies**: Compiles with plain g++. No cmake, no libraries. |
| 3. **Every line readable**: Written for humans learning how NNs work internally. |
| 4. **Hardcoded derivatives**: Each layer knows its own gradient formula. No autograd, no symbolic math. |
|
|
| ## Backend Abstraction |
|
|
| The `core/backend.hpp` file contains ALL math operations: |
| - `matmul` (multi-threaded) |
| - `relu_forward` / `relu_backward` |
| - `sigmoid_forward` / `sigmoid_backward` |
| - `transpose`, `add`, `scale`, `sum_columns` |
|
|
| To port to GPU: replace this ONE file with SYCL kernels. Everything else stays the same. |
|
|
| ## Roadmap |
|
|
| - [x] Tensor class |
| - [x] Dense layer with forward/backward |
| - [x] SGD optimizer |
| - [x] MSE loss |
| - [x] XOR training (proof of concept) |
| - [ ] MNIST training |
| - [ ] Skip connections (ResNet-style) |
| - [ ] Conv2D layer |
| - [ ] Adam optimizer |
| - [ ] L-BFGS (second-order optimizer) |
| - [ ] SYCL/AdaptiveCpp backend |
| - [ ] Vision Transformer |
|
|
| ## Papers & References |
|
|
| - Backpropagation: Rumelhart et al., 1986 |
| - Xavier initialization: Glorot & Bengio, 2010 |
| - Adam: Kingma & Ba, 2014 |
| - ResNet skip connections: He et al., 2015 |
|
|
| ## License |
|
|
| MIT |
|
|
| <!-- ml-intern-provenance --> |
| ## Generated by ML Intern |
|
|
| This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub. |
|
|
| - Try ML Intern: https://smolagents-ml-intern.hf.space |
| - Source code: https://github.com/huggingface/ml-intern |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_id = "notRaphael/newnet" |
| tokenizer = AutoTokenizer.from_pretrained(model_id) |
| model = AutoModelForCausalLM.from_pretrained(model_id) |
| ``` |
|
|
| For non-causal architectures, replace `AutoModelForCausalLM` with the appropriate `AutoModel` class. |
|
|
