mlboydaisuke commited on
Commit
172e92d
·
verified ·
1 Parent(s): bb4fa29

Card: add 'Reproduce (official tools only)' section

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -74,6 +74,28 @@ standard `LlamaForCausalLM` architecture, so it rides the existing converter and
74
  runtime with no custom code. Blockwise (not channelwise) int4 is what preserves
75
  reasoning accuracy.
76
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  ## License
78
 
79
  Falcon LLM License (TII), inherited from the base model
 
74
  runtime with no custom code. Blockwise (not channelwise) int4 is what preserves
75
  reasoning accuracy.
76
 
77
+ ## Reproduce (official tools only)
78
+
79
+ Built with **stock `litert-torch`** — no custom code, no graph patches. The only
80
+ non-default choice is the int4 recipe: the tool's default named int4 is
81
+ *channelwise* (which degrades small models), so this uses **blockwise-128** (the
82
+ scheme the official models ship), passed as a recipe file to the standard export:
83
+
84
+ ```python
85
+ from litert_torch.generative.export_hf.export import export
86
+ export(
87
+ model="tiiuae/Falcon3-3B-Instruct",
88
+ output_dir="out",
89
+ quantization_recipe="falcon_int4_block128.json", # included in this repo
90
+ cache_length=2048,
91
+ trust_remote_code=True,
92
+ )
93
+ ```
94
+
95
+ `falcon_int4_block128.json` is included in this repo. (If the export errors with a
96
+ missing `ai_edge_quantizer/recipes/` directory, create it empty — a packaging gap
97
+ in some releases that trips the `.json`-recipe path.)
98
+
99
  ## License
100
 
101
  Falcon LLM License (TII), inherited from the base model