fumx66 commited on
Commit
f85a7a0
Β·
verified Β·
1 Parent(s): e299904

Update model card with TypeGround paper details

Browse files
Files changed (1) hide show
  1. README.md +37 -47
README.md CHANGED
@@ -19,60 +19,45 @@ pipeline_tag: text-generation
19
  datasets:
20
  - TypeGround
21
  - ManyTypes4TypeScript
22
- widget:
23
- - text: "const x = <mask>: number[];"
24
- example_title: "Type Inference Example"
25
  ---
26
 
27
  # TypeGround_weight
28
 
29
- Fine-tuned models for TypeScript type inference, trained as part of the **TypeGround** framework. This repository contains both traditional encoder-decoder models (full fine-tune) and large language models (LoRA adapters).
30
 
31
- ## Repository Structure
 
 
 
 
32
 
33
  ```
34
  TypeGround_weight/
35
- β”œβ”€β”€ CodeT5/ # Full fine-tune
36
- β”‚ β”œβ”€β”€ TypeGround/ # Trained on TypeGround dataset
37
- β”‚ └── ManyTypes4TypeScrip/ # Trained on ManyTypes4TypeScript dataset
38
- β”œβ”€β”€ CodeT5+/ # Full fine-tune
39
  β”‚ β”œβ”€β”€ TypeGround/
40
  β”‚ └── ManyTypes4TypeScrip/
41
- β”œβ”€β”€ UniXcoder/ # Full fine-tune
42
  β”‚ β”œβ”€β”€ TypeGround/
43
  β”‚ └── ManyTypes4TypeScrip/
44
- β”œβ”€β”€ Llama3-8B/ # LoRA adapters (rank=8, alpha=16)
45
  β”‚ β”œβ”€β”€ ManyTypes4TypeScrip/lora/sft/
46
- β”‚ β”œβ”€β”€ TypeGround/slice/lora/sft/
47
- β”‚ └── TypeGround/snippet/lora/sft/
48
- β”œβ”€β”€ Qwen3-14B/ # LoRA adapters (rank=8, alpha=16)
49
  β”‚ β”œβ”€β”€ ManyTypes4TypeScrip/lora/sft/
50
- β”‚ β”œβ”€β”€ TypeGround/slice/lora/sft/
51
- β”‚ └── TypeGround/snippet/lora/sft/
52
- β”œβ”€β”€ DeepSeek-Coder-6.7B/ # LoRA adapters (rank=8, alpha=16)
53
  β”‚ β”œβ”€β”€ ManyTypes4TypeScrip/lora/sft/
54
- β”‚ β”œβ”€β”€ TypeGround/slice/lora/sft/
55
- β”‚ └── TypeGround/snippet/lora/sft/
56
  └── prediction.py # Batch prediction script
57
  ```
58
 
59
- ## Model Variants
60
-
61
- ### Training Datasets
62
-
63
- | Variant | Description |
64
- |---|---|
65
- | `TypeGround` | Original TypeGround dataset |
66
- | `ManyTypes4TypeScript` | Extended dataset with diverse TypeScript types |
67
- | `slice` | Fine-tuned on sliced code contexts |
68
- | `snippet` | Fine-tuned on code snippets |
69
-
70
- ## Usage
71
 
72
  ### Traditional Models (Full Fine-tune)
73
 
74
- CodeT5, CodeT5+, and UniXcoder models can be loaded directly with Transformers:
75
-
76
  ```python
77
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
78
 
@@ -82,22 +67,16 @@ tokenizer = AutoTokenizer.from_pretrained("./CodeT5/TypeGround")
82
 
83
  ### LLMs (LoRA Adapters)
84
 
85
- LoRA adapters require loading the base model first:
86
-
87
  | Directory | Base Model |
88
  |---|---|
89
  | `Llama3-8B` | `meta-llama/Meta-Llama-3-8B-Instruct` |
90
  | `Qwen3-14B` | `Qwen/Qwen3-14B` |
91
  | `DeepSeek-Coder-6.7B` | `deepseek-ai/deepseek-coder-6.7b-instruct` |
92
 
93
- **Install vLLM:**
94
-
95
  ```bash
96
  pip install vllm
97
  ```
98
 
99
- **Serve with LoRA:**
100
-
101
  ```bash
102
  vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
103
  --enable-lora \
@@ -107,22 +86,33 @@ vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
107
 
108
  ### Batch Prediction
109
 
110
- Edit the `__main__` block in `prediction.py` to point to your data and model, then run:
111
-
112
  ```bash
113
  python prediction.py
114
  ```
115
 
116
- ## Citation
 
 
 
 
 
 
 
 
 
 
 
117
 
118
  ```bibtex
119
- @dataset{typeground,
120
- title={TypeGround: Fine-Grained Benchmarking for TypeScript Type Inference},
121
- year={2026},
122
- url={https://huggingface.co/fumx66/TypeGround_weight}
 
 
123
  }
124
  ```
125
 
126
- ## License
127
 
128
  MIT License
 
19
  datasets:
20
  - TypeGround
21
  - ManyTypes4TypeScript
 
 
 
22
  ---
23
 
24
  # TypeGround_weight
25
 
26
+ Model weights for the paper **"TypeGround: Fine-Grained Benchmarking for TypeScript Type Inference"**.
27
 
28
+ > **Abstract:** TypeScript is widely used in web applications and mini-programs, yet real-world projects still contain many variables, function parameters, and return values whose annotations are missing or inferred as coarse-grained types (e.g., `any`). Automatic type inference requires benchmarks with reliable training and evaluation signals. Existing TypeScript datasets (e.g., ManyTypes4TypeScript), however, still contain compiler-uncheckable type expressions, coarse annotations, insufficient context, and evaluation protocols that deviate from compiler semantics, which threatens the reliability of both the training labels and the evaluation signals derived from them. To mitigate the above limitation, we present TypeGround, an automatically constructed TypeScript type inference benchmark. The core idea of TypeGround is to leverage the type-checking capability of the TypeScript Compiler (TSC), together with the generative capability of large language models, to produce compiler-friendly type annotations while avoiding coarse-grained types. TypeGround covers 2,735 projects, 99,209 source files, and 1,428,997 type annotations. Experiments show that training on TypeGround improves overall EM-based MRR@5 by 23.55 percentage points on average over ManyTypes4TypeScript. The user study further confirms the reliability of TypeGround's automatic labelling process.
29
+ >
30
+ > **Dataset:** [github.com/fumx66/TypeGround](https://github.com/fumx66/TypeGround)
31
+
32
+ ## πŸ“¦ Repository Structure
33
 
34
  ```
35
  TypeGround_weight/
36
+ β”œβ”€β”€ CodeT5/ # Full fine-tune (T5ForConditionalGeneration)
37
+ β”‚ β”œβ”€β”€ TypeGround/
38
+ β”‚ └── ManyTypes4TypeScrip/
39
+ β”œβ”€β”€ CodeT5+/ # Full fine-tune (T5ForConditionalGeneration)
40
  β”‚ β”œβ”€β”€ TypeGround/
41
  β”‚ └── ManyTypes4TypeScrip/
42
+ β”œβ”€β”€ UniXcoder/ # Full fine-tune (UniXcoder)
43
  β”‚ β”œβ”€β”€ TypeGround/
44
  β”‚ └── ManyTypes4TypeScrip/
45
+ β”œβ”€β”€ Llama3-8B/ # LoRA adapters
46
  β”‚ β”œβ”€β”€ ManyTypes4TypeScrip/lora/sft/
47
+ β”‚ └── TypeGround/{slice,snippet}/lora/sft/
48
+ β”œβ”€β”€ Qwen3-14B/ # LoRA adapters
 
49
  β”‚ β”œβ”€β”€ ManyTypes4TypeScrip/lora/sft/
50
+ β”‚ └── TypeGround/{slice,snippet}/lora/sft/
51
+ β”œβ”€β”€ DeepSeek-Coder-6.7B/ # LoRA adapters
 
52
  β”‚ β”œβ”€β”€ ManyTypes4TypeScrip/lora/sft/
53
+ β”‚ └── TypeGround/{slice,snippet}/lora/sft/
 
54
  └── prediction.py # Batch prediction script
55
  ```
56
 
57
+ ## πŸš€ Usage
 
 
 
 
 
 
 
 
 
 
 
58
 
59
  ### Traditional Models (Full Fine-tune)
60
 
 
 
61
  ```python
62
  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
63
 
 
67
 
68
  ### LLMs (LoRA Adapters)
69
 
 
 
70
  | Directory | Base Model |
71
  |---|---|
72
  | `Llama3-8B` | `meta-llama/Meta-Llama-3-8B-Instruct` |
73
  | `Qwen3-14B` | `Qwen/Qwen3-14B` |
74
  | `DeepSeek-Coder-6.7B` | `deepseek-ai/deepseek-coder-6.7b-instruct` |
75
 
 
 
76
  ```bash
77
  pip install vllm
78
  ```
79
 
 
 
80
  ```bash
81
  vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
82
  --enable-lora \
 
86
 
87
  ### Batch Prediction
88
 
 
 
89
  ```bash
90
  python prediction.py
91
  ```
92
 
93
+ ## πŸ“Š Models
94
+
95
+ | Model | Architecture | Type | LoRA Config |
96
+ |---|---|---|---|
97
+ | CodeT5 | T5ForConditionalGeneration | Full fine-tune | β€” |
98
+ | CodeT5+ | T5ForConditionalGeneration | Full fine-tune | β€” |
99
+ | UniXcoder | UniXcoder | Full fine-tune | β€” |
100
+ | Llama3-8B | CausalLM + LoRA | Adapter | rank=8, Ξ±=16 |
101
+ | Qwen3-14B | CausalLM + LoRA | Adapter | rank=8, Ξ±=16 |
102
+ | DeepSeek-Coder-6.7B | CausalLM + LoRA | Adapter | rank=8, Ξ±=16 |
103
+
104
+ ## πŸ“ Citation
105
 
106
  ```bibtex
107
+ @inproceedings{typeground,
108
+ title = {TypeGround: Fine-Grained Benchmarking for TypeScript Type Inference},
109
+ author = {Anonymous},
110
+ booktitle = {},
111
+ year = {2026},
112
+ url = {https://github.com/fumx66/TypeGround}
113
  }
114
  ```
115
 
116
+ ## πŸ“„ License
117
 
118
  MIT License