|
|
--- |
|
|
base_model: |
|
|
- tiiuae/Falcon-E-3B-Instruct |
|
|
tags: |
|
|
- CoreML |
|
|
--- |
|
|
Falcon-Edge-3B-Instruct converted to run on Apple Neural Engine (ANe). Model conversion code [here](https://github.com/0seba/coreml-models). |
|
|
|
|
|
To run clone this repo with the files and run this command: |
|
|
|
|
|
`python falcon_edge_generate.py --embeddings ./falcon_edge_3b_embeddings.npy --lm_head falcon_edge_3b_lm_head.mlmodelc --model ./falcon_edge_3b.mlmodelc --cache_length 1024 --temp 0.7 --min_p 0.1` |
|
|
|
|
|
Supported cache lengths are `512, 1024, 2048, 3072, 4096, 6144 and 8192`. |
|
|
|
|
|
First time running with a new cache length takes a longer initialization time for model compilation. |
|
|
|
