mlboydaisuke commited on
Commit
5d8a1c9
·
verified ·
1 Parent(s): ad79923

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +46 -0
README.md ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: microsoft/FastContext-1.0-4B-SFT
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - core-ai
7
+ - coreml
8
+ - on-device
9
+ - ios
10
+ - iphone
11
+ - qwen3
12
+ language:
13
+ - en
14
+ ---
15
+
16
+ # FastContext-1.0-4B-SFT — Core AI (on-device, iPhone)
17
+
18
+ [microsoft/FastContext-1.0-4B-SFT](https://huggingface.co/microsoft/FastContext-1.0-4B-SFT)
19
+ converted to Apple **Core AI** for on-device inference, served by the **CoreAIChat** app.
20
+
21
+ FastContext is a long-context **repository-exploration agent** (Qwen3-4B-Instruct
22
+ backbone, supervised-fine-tuned on exploration traces: broad first-turn search,
23
+ multi-turn evidence gathering, and precise file:line citation generation).
24
+
25
+ ## Contents
26
+ - `gpu/` — AOT-compiled **GPU (MPSGraph)** bundle, 4-bit linear-INT4, **h18p**
27
+ (iPhone 17 / iPhone 18 class). ~2.1 GB. Drop-in for CoreAIChat's
28
+ `Documents/models/fastcontext_4b_gpu`.
29
+
30
+ > **Device class:** this is an **AOT h18p** bundle (iPhone 17 / 18 class). On-device
31
+ > specialization of a 4B graph is not viable, so the bundle is precompiled for the
32
+ > h18p GPU — the same approach Core AI uses for the Gemma-4B bundle.
33
+
34
+ ## Verified on iPhone 17 Pro (iOS 27)
35
+ - **Parity** vs HF fp16 (teacher-forced): **23/24 argmax**, perplexity 1.41 — the
36
+ single flip is a near-tie quant-noise position.
37
+ - **On-device GPU:** ~**334 tok/s** prefill, ~**16.5 tok/s** decode (512-prompt /
38
+ 1024-gen), load ~8 s cold, ~0.3 GB resident (weights memory-mapped).
39
+
40
+ ## Use
41
+ Install **CoreAIChat**, open the model picker, and select **FastContext 4B**.
42
+
43
+ ## License
44
+ MIT, inherited from the base model
45
+ [microsoft/FastContext-1.0-4B-SFT](https://huggingface.co/microsoft/FastContext-1.0-4B-SFT).
46
+ See `LICENSE`.