nielsr HF Staff commited on
Commit
0b6a32e
·
verified ·
1 Parent(s): cd38f48

Improve model card: Add pipeline tag, paper link, and performance metrics

Browse files

This PR enhances the model card for PETRA by:
- Adding the `pipeline_tag: text-generation` to help users discover the model.
- Adding `library_name: transformers` based on the use of Megatron-LM, indicating potential compatibility with the Transformers library.
- Linking directly to the Hugging Face paper page [PETRA: Pretrained Evolutionary Transformer for SARS-CoV-2 Mutation Prediction](https://huggingface.co/papers/2511.03976).
- Incorporating the abstract for a better overview.
- Including detailed performance metrics from the official GitHub repository.

Please review and merge this PR if everything looks good.

Files changed (1) hide show
  1. README.md +55 -3
README.md CHANGED
@@ -1,7 +1,59 @@
1
  ---
2
  license: mit
 
 
3
  ---
4
- model for PETRA
5
- https://github.com/xz-keg/PETra
6
 
7
- Three models represent model trained on different data, one(116M) on usher tree data before 2025-02-12, one 2025-07-16, one large model(458M) on 2025-09-23
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ pipeline_tag: text-generation
4
+ library_name: transformers
5
  ---
 
 
6
 
7
+ # PETRA: Pretrained Evolutionary Transformer for SARS-CoV-2 Mutation Prediction
8
+
9
+ This repository contains the model weights for **PETRA (Pretrained Evolutionary TRAnsformer)**, a novel transformer approach for predicting SARS-CoV-2 mutations, as presented in the paper [PETRA: Pretrained Evolutionary Transformer for SARS-CoV-2 Mutation Prediction](https://huggingface.co/papers/2511.03976). The approach is based on evolutionary trajectories derived from phylogenetic trees, effectively mitigating sequencing noise and capturing the hierarchical structure of viral evolution.
10
+
11
+ **Abstract:**
12
+
13
+ Since its emergence, SARS-CoV-2 has demonstrated a rapid and unpredictable evolutionary trajectory, characterized by the continual emergence of immune-evasive variants. This poses persistent challenges to public health and vaccine development. While large-scale generative pre-trained transformers (GPTs) have revolutionized the modeling of sequential data, their direct applications to noisy viral genomic sequences are limited. In this paper, we introduce PETRA(Pretrained Evolutionary TRAnsformer), a novel transformer approach based on evolutionary trajectories derived from phylogenetic trees rather than raw RNA sequences. This method effectively mitigates sequencing noise and captures the hierarchical structure of viral evolution. With a weighted training framework to address substantial geographical and temporal imbalances in global sequence data, PETRA excels in predicting future SARS-CoV-2 mutations, achieving a weighted recall@1 of 9.45% for nucleotide mutations and 17.10\% for spike amino-acid mutations, compared to 0.49% and 6.64% respectively for the best baseline. PETRA also demonstrates its ability to aid in the real-time mutation prediction of major clades like 24F(XEC) and 25A(LP.8.1).
14
+
15
+ **Codebase:** The official PyTorch implementation and codebase, built on Megatron-LM, can be found at: [https://github.com/xz-keg/PETra](https://github.com/xz-keg/PETra)
16
+
17
+ Three models represent models trained on different data:
18
+ * One (116M parameters) trained on usher tree data before 2025-02-12.
19
+ * One (medium size) trained on data before 2025-07-16.
20
+ * One large model (458M parameters) trained on data before 2025-09-23.
21
+
22
+ ---
23
+
24
+ ### Performance
25
+ Performance for the 20250212 model, evaluated on sequences collected after 2025-2-12 and available before 2025-7-16.
26
+
27
+ #### Nucleotide mutation prediction results for **PETRA**
28
+ We report average and weighted recall @1, 10, and 100. In weighted measure, sequences are weighted by their representativeness.
29
+
30
+ | **Method** | **Average Recall @1** | **@10** | **@100** | **Weighted Recall @1** | **@10** | **@100** |
31
+ | :----------- | --------------------: | ---------: | ---------: | ---------------------: | ---------: | ---------: |
32
+ | Random Guess | 0.00% | 0.01% | 0.08% | 0.00% | 0.01% | 0.08% |
33
+ | Bloom | 0.45% | 1.50% | 9.15% | 0.49% | 1.48% | 9.41% |
34
+ | **PETRA** | **11.34%** | **16.92%** | **22.64%** | **9.45%** | **14.20%** | **19.72%** |
35
+
36
+ ---
37
+
38
+ #### Spike amino-acid mutation prediction results for **PETRA**
39
+ We report average and weighted recall @1 and 10. In weighted measure, sequences are weighted by their representativeness.
40
+
41
+ | **Method** | **Average Recall @1** | **@10** | **Weighted Recall @1** | **@10** |
42
+ | :----------- | --------------------: | ---------: | ---------------------: | ---------: |
43
+ | Random Guess | 0.01% | 0.13% | 0.01% | 0.13% |
44
+ | Bloom | 6.26% | 12.63% | 6.64% | 13.08% |
45
+ | **PETRA** | **17.84%** | **25.69%** | **17.10%** | **25.58%** |
46
+
47
+ ---
48
+
49
+ ## Citation
50
+ If you find PETRA useful for your research, please consider citing the paper:
51
+ ```bibtex
52
+ @article{luo2025petra,
53
+ title={PETRA: Pretrained Evolutionary Transformer for SARS-CoV-2 Mutation Prediction},
54
+ author={Luo, J. and et al.},
55
+ journal={Hugging Face Papers},
56
+ year={2025},
57
+ url={https://huggingface.co/papers/2511.03976}
58
+ }
59
+ ```