---
tags:
- ml-intern
---
# newnet — Neural Network Engine from Scratch in C++

A minimal, educational neural network library built from scratch in C++17.

**No dependencies.** One compile command. Multi-threaded CPU backend designed to be swapped to SYCL/AdaptiveCpp for cross-vendor GPU support.

## Quick Start

```bash
git clone https://huggingface.co/notRaphael/newnet
cd newnet
g++ -std=c++17 -O2 -pthread -o newnet main.cpp
./newnet
```

## Architecture

```
newnet/
├── core/
│   ├── tensor.hpp      ← Tensor class (data + grad + shape)
│   └── backend.hpp     ← All math ops (matmul, relu, sigmoid, etc.)
│                          Swap THIS file for SYCL/GPU backend.
├── layers/
│   ├── layer.hpp       ← Abstract base class
│   └── dense.hpp       ← Fully connected layer (forward + backward)
├── graph/
│   ├── graph.hpp       ← Sequential model (chains layers)
│   └── optimizer.hpp   ← SGD optimizer
├── loss/
│   └── loss.hpp        ← MSE loss
└── main.cpp            ← XOR training example with progress bar
```

## Design Principles

1. **Loose coupling**: Backend is one file. Swap it for SYCL/AdaptiveCpp without touching neural network code.
2. **No dependencies**: Compiles with plain g++. No cmake, no libraries.
3. **Every line readable**: Written for humans learning how NNs work internally.
4. **Hardcoded derivatives**: Each layer knows its own gradient formula. No autograd, no symbolic math.

## Backend Abstraction

The `core/backend.hpp` file contains ALL math operations:
- `matmul` (multi-threaded)
- `relu_forward` / `relu_backward`
- `sigmoid_forward` / `sigmoid_backward`
- `transpose`, `add`, `scale`, `sum_columns`

To port to GPU: replace this ONE file with SYCL kernels. Everything else stays the same.

## Roadmap

- [x] Tensor class
- [x] Dense layer with forward/backward
- [x] SGD optimizer
- [x] MSE loss
- [x] XOR training (proof of concept)
- [ ] MNIST training
- [ ] Skip connections (ResNet-style)
- [ ] Conv2D layer
- [ ] Adam optimizer
- [ ] L-BFGS (second-order optimizer)
- [ ] SYCL/AdaptiveCpp backend
- [ ] Vision Transformer

## Papers & References

- Backpropagation: Rumelhart et al., 1986
- Xavier initialization: Glorot & Bengio, 2010
- Adam: Kingma & Ba, 2014
- ResNet skip connections: He et al., 2015

## License

MIT

<!-- ml-intern-provenance -->
## Generated by ML Intern

This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub.

- Try ML Intern: https://smolagents-ml-intern.hf.space
- Source code: https://github.com/huggingface/ml-intern

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "notRaphael/newnet"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
```

For non-causal architectures, replace `AutoModelForCausalLM` with the appropriate `AutoModel` class.