| --- |
| license: bsd-3-clause |
| library_name: mlx |
| pipeline_tag: text-generation |
| base_model: hugohrban/progen2-base |
| tags: |
| - progen |
| - progen2 |
| - protein-language-model |
| - mlx |
| - mlx-lm |
| - safetensors |
| - bfloat16 |
| - icl-many-replication |
| widget: |
| - text: 1MEVVIVTGMSGAGK |
| - text: 1MKTLLILAV |
| --- |
| |
| # ProGen2 Base BF16 MLX Conversion |
|
|
| This repository contains a BF16 MLX-LM conversion of |
| [`hugohrban/progen2-base`](https://huggingface.co/hugohrban/progen2-base), a |
| mirror of the base ProGen2 model by Nijkamp et al. |
|
|
| The source checkpoint was downloaded from Hugging Face revision |
| `71228cbfca5960f9fab5775f378bba3673af9f00` and converted from FP32 to BF16 |
| safetensors. The converted tensor file contains 764,803,616 BF16 parameters. |
|
|
| This conversion was prepared for the standalone replication artifact for |
| *Many Next Token Predictors are In-context Learners*. |
|
|
| ## Files |
|
|
| - `model.safetensors`: BF16 converted ProGen2-base weights. |
| - `progen2_mlx.py`: MLX-LM custom model implementation. |
| - `config.json`, tokenizer files, and ProGen custom-code files copied from the |
| source model with the MLX model-file entry added. |
|
|
| ## Loading with MLX-LM |
|
|
| ```python |
| from mlx_lm import load |
| |
| model, tokenizer = load("N8Programs/ProGen2-base-bf16") |
| ``` |
|
|
| ## Original Model |
|
|
| Please cite and follow the terms of the upstream ProGen2 work and the source |
| model repository: |
|
|
| - Source model: https://huggingface.co/hugohrban/progen2-base |
| - Paper: https://arxiv.org/abs/2206.13517 |
|
|
