---
model-index:
- name: netFound-640M-base
  results:
  - task:
      type: fill-mask
    metrics:
    - name: Macro MLM F1
      type: f1
      value: 0.4038
    - name: Weighted MLM F1
      type: f1
      value: 0.8451
    - name: MLM Accuracy
      type: accuracy
      value: 0.8514
    - name: Swapped Weighted F1
      type: f1
      value: 0.9605
    - name: Perplexity
      type: perplexity
      value: 6.5842
---

# netFound-640M-base

## Description

netFound is a network traffic foundation model that uses transformer architecture and includes a pretraining phase on unlabeled data to achieve high results.  

Key features:
- netFound takes raw PCAP data as input
- netFound can (and need) be pretrained on the unlabeled dataset
- netFound uses Hierarchical Transformer architecture to take into account packet burst and flow behavior
- netFound uses burst metadata (inter arrival time, number of bytes per burst, etc)

## Source code

https://github.com/SNL-UCSB/netfound

## Pretraining dataset

For pretraining, we used a private real-world dataset consisting of more than 450mln network flows. The model was pretrained for approximately 1 epoch (iterated through ~480mln flows).

## Checkpoint

Model: Large  (16 heads, 24 hidden layers, 1024 hidden size)  
Total params: 643,825,672  
January 17, 2025  

## Paper
https://arxiv.org/abs/2310.17025