optimum-executorch/pr_247/en/quickstart.md

Quickstart

Export

You can export your 🤗 Transformers models to ExecuTorch easily:

optimum-cli export executorch --model meta-llama/Llama-3.2-1B --recipe xnnpack --output_dir meta_llama3_2_1b_executorch

Inference

To load a model and run inference, you can just replace your AutoModelForCausalLM class with the corresponding ExecuTorchModelForCausalLM class. You can also load a PyTorch checkpoint and convert it to ExecuTorch on-the-fly when loading your model.

- from transformers import AutoModelForCausalLM
+ from optimum.executorch import ExecuTorchModelForCausalLM
  from transformers import AutoTokenizer

  model_id = "meta-llama/Llama-3.2-1B"
  tokenizer = AutoTokenizer.from_pretrained(model_id)
- model = AutoModelForCausalLM.from_pretrained(model_id)
+ model = ExecuTorchModelForCausalLM.from_pretrained(model_id)