Araya2001 commited on
Commit
147dfd6
·
verified ·
1 Parent(s): 7ba1647

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +59 -0
README.md ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - es
5
+ base_model: CodeStrux-Tech/tac-1
6
+ base_model_relation: quantized
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - fp8
10
+ - compressed-tensors
11
+ - llmcompressor
12
+ - address-parsing
13
+ - costa-rica
14
+ - qwen3
15
+ ---
16
+
17
+ # tac-1-fp8-dynamic — FP8 dynamic quantization of tac-1
18
+
19
+ ## Overview
20
+
21
+ tac-1-fp8-dynamic is quantized from [`CodeStrux-Tech/tac-1`](https://huggingface.co/CodeStrux-Tech/tac-1) using llmcompressor 0.12.0 with the FP8_DYNAMIC scheme. Targets are `Linear` layers; `lm_head` is ignored. The result is a single 4.2 GB shard.
22
+
23
+ sha256: `e0eb5c3c62e8a472eaca6f86064efb2597863f0688ac3ca1fd9a325390d9c30f`
24
+
25
+ ## Serving
26
+
27
+ ```bash
28
+ vllm serve CodeStrux-Tech/tac-1-fp8-dynamic --served-model-name tac-1 --max-model-len 4096
29
+ # On hosts without nvcc, disable the FlashInfer sampler JIT:
30
+ # VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve ...
31
+ ```
32
+
33
+ No `--quantization` flag is needed — vLLM auto-detects the FP8 format from the model config. The FP8 model is a drop-in replacement for the bf16 original; the client snippet from [`CodeStrux-Tech/tac-1`](https://huggingface.co/CodeStrux-Tech/tac-1) works unchanged:
34
+
35
+ ```python
36
+ from tico.extractor.clients.vllm import VLLMExtractor
37
+
38
+ extractor = VLLMExtractor() # VLLM_BASE_URL=http://localhost:8000/v1, TICO_VLLM_MODEL=tac-1
39
+ addr = extractor.extract("del antiguo higuerón de San Pedro, 100 metros sur")
40
+ print(addr.model_dump_json(indent=2))
41
+ ```
42
+
43
+ ## Evaluation
44
+
45
+ | metric | tac-1 (bf16) | tac-1 (FP8) | teacher gpt-4.1 |
46
+ |---|---|---|---|
47
+ | parse rate | 1.000 | 1.000 | 1.000 |
48
+ | anchor_name F1 | 0.979 | 0.979 | 0.872 |
49
+ | direction F1 | 0.995 | 0.984 | 1.000 |
50
+ | distance F1 | 0.995 | 0.984 | 1.000 |
51
+ | perfect-anchor ≤250 m | 93.6% | 93.6% | 93.6% |
52
+ | perfect-anchor median | 0.0 m | 0.0 m | 0.0 m |
53
+ | gazetteer ≤250 m | 80.9% | 80.9% | 83.0% |
54
+
55
+ ## Training data attribution
56
+
57
+ Contains information from OpenStreetMap (https://www.openstreetmap.org/copyright), which is made available under the Open Database License (ODbL) 1.0. © OpenStreetMap contributors.
58
+
59
+ For full training details, architecture, evaluation, and limitations, see [`CodeStrux-Tech/tac-1`](https://huggingface.co/CodeStrux-Tech/tac-1).