Text Generation
PEFT
Safetensors
English
code
type-inference
typescript
code-generation
type-ground
lora
code-t5
unixcoder
llama
qwen
deepseek
Instructions to use fumx66/TypeGround_weight with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use fumx66/TypeGround_weight with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Update model card with TypeGround paper details
Browse files
README.md
CHANGED
|
@@ -19,60 +19,45 @@ pipeline_tag: text-generation
|
|
| 19 |
datasets:
|
| 20 |
- TypeGround
|
| 21 |
- ManyTypes4TypeScript
|
| 22 |
-
widget:
|
| 23 |
-
- text: "const x = <mask>: number[];"
|
| 24 |
-
example_title: "Type Inference Example"
|
| 25 |
---
|
| 26 |
|
| 27 |
# TypeGround_weight
|
| 28 |
|
| 29 |
-
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
```
|
| 34 |
TypeGround_weight/
|
| 35 |
-
βββ CodeT5/ # Full fine-tune
|
| 36 |
-
β βββ TypeGround/
|
| 37 |
-
β βββ ManyTypes4TypeScrip/
|
| 38 |
-
βββ CodeT5+/ # Full fine-tune
|
| 39 |
β βββ TypeGround/
|
| 40 |
β βββ ManyTypes4TypeScrip/
|
| 41 |
-
βββ UniXcoder/ # Full fine-tune
|
| 42 |
β βββ TypeGround/
|
| 43 |
β βββ ManyTypes4TypeScrip/
|
| 44 |
-
βββ Llama3-8B/ # LoRA adapters
|
| 45 |
β βββ ManyTypes4TypeScrip/lora/sft/
|
| 46 |
-
β
|
| 47 |
-
|
| 48 |
-
βββ Qwen3-14B/ # LoRA adapters (rank=8, alpha=16)
|
| 49 |
β βββ ManyTypes4TypeScrip/lora/sft/
|
| 50 |
-
β
|
| 51 |
-
|
| 52 |
-
βββ DeepSeek-Coder-6.7B/ # LoRA adapters (rank=8, alpha=16)
|
| 53 |
β βββ ManyTypes4TypeScrip/lora/sft/
|
| 54 |
-
β
|
| 55 |
-
β βββ TypeGround/snippet/lora/sft/
|
| 56 |
βββ prediction.py # Batch prediction script
|
| 57 |
```
|
| 58 |
|
| 59 |
-
##
|
| 60 |
-
|
| 61 |
-
### Training Datasets
|
| 62 |
-
|
| 63 |
-
| Variant | Description |
|
| 64 |
-
|---|---|
|
| 65 |
-
| `TypeGround` | Original TypeGround dataset |
|
| 66 |
-
| `ManyTypes4TypeScript` | Extended dataset with diverse TypeScript types |
|
| 67 |
-
| `slice` | Fine-tuned on sliced code contexts |
|
| 68 |
-
| `snippet` | Fine-tuned on code snippets |
|
| 69 |
-
|
| 70 |
-
## Usage
|
| 71 |
|
| 72 |
### Traditional Models (Full Fine-tune)
|
| 73 |
|
| 74 |
-
CodeT5, CodeT5+, and UniXcoder models can be loaded directly with Transformers:
|
| 75 |
-
|
| 76 |
```python
|
| 77 |
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
|
| 78 |
|
|
@@ -82,22 +67,16 @@ tokenizer = AutoTokenizer.from_pretrained("./CodeT5/TypeGround")
|
|
| 82 |
|
| 83 |
### LLMs (LoRA Adapters)
|
| 84 |
|
| 85 |
-
LoRA adapters require loading the base model first:
|
| 86 |
-
|
| 87 |
| Directory | Base Model |
|
| 88 |
|---|---|
|
| 89 |
| `Llama3-8B` | `meta-llama/Meta-Llama-3-8B-Instruct` |
|
| 90 |
| `Qwen3-14B` | `Qwen/Qwen3-14B` |
|
| 91 |
| `DeepSeek-Coder-6.7B` | `deepseek-ai/deepseek-coder-6.7b-instruct` |
|
| 92 |
|
| 93 |
-
**Install vLLM:**
|
| 94 |
-
|
| 95 |
```bash
|
| 96 |
pip install vllm
|
| 97 |
```
|
| 98 |
|
| 99 |
-
**Serve with LoRA:**
|
| 100 |
-
|
| 101 |
```bash
|
| 102 |
vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
|
| 103 |
--enable-lora \
|
|
@@ -107,22 +86,33 @@ vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
|
|
| 107 |
|
| 108 |
### Batch Prediction
|
| 109 |
|
| 110 |
-
Edit the `__main__` block in `prediction.py` to point to your data and model, then run:
|
| 111 |
-
|
| 112 |
```bash
|
| 113 |
python prediction.py
|
| 114 |
```
|
| 115 |
|
| 116 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
|
| 118 |
```bibtex
|
| 119 |
-
@
|
| 120 |
-
title={TypeGround: Fine-Grained Benchmarking for TypeScript Type Inference},
|
| 121 |
-
|
| 122 |
-
|
|
|
|
|
|
|
| 123 |
}
|
| 124 |
```
|
| 125 |
|
| 126 |
-
## License
|
| 127 |
|
| 128 |
MIT License
|
|
|
|
| 19 |
datasets:
|
| 20 |
- TypeGround
|
| 21 |
- ManyTypes4TypeScript
|
|
|
|
|
|
|
|
|
|
| 22 |
---
|
| 23 |
|
| 24 |
# TypeGround_weight
|
| 25 |
|
| 26 |
+
Model weights for the paper **"TypeGround: Fine-Grained Benchmarking for TypeScript Type Inference"**.
|
| 27 |
|
| 28 |
+
> **Abstract:** TypeScript is widely used in web applications and mini-programs, yet real-world projects still contain many variables, function parameters, and return values whose annotations are missing or inferred as coarse-grained types (e.g., `any`). Automatic type inference requires benchmarks with reliable training and evaluation signals. Existing TypeScript datasets (e.g., ManyTypes4TypeScript), however, still contain compiler-uncheckable type expressions, coarse annotations, insufficient context, and evaluation protocols that deviate from compiler semantics, which threatens the reliability of both the training labels and the evaluation signals derived from them. To mitigate the above limitation, we present TypeGround, an automatically constructed TypeScript type inference benchmark. The core idea of TypeGround is to leverage the type-checking capability of the TypeScript Compiler (TSC), together with the generative capability of large language models, to produce compiler-friendly type annotations while avoiding coarse-grained types. TypeGround covers 2,735 projects, 99,209 source files, and 1,428,997 type annotations. Experiments show that training on TypeGround improves overall EM-based MRR@5 by 23.55 percentage points on average over ManyTypes4TypeScript. The user study further confirms the reliability of TypeGround's automatic labelling process.
|
| 29 |
+
>
|
| 30 |
+
> **Dataset:** [github.com/fumx66/TypeGround](https://github.com/fumx66/TypeGround)
|
| 31 |
+
|
| 32 |
+
## π¦ Repository Structure
|
| 33 |
|
| 34 |
```
|
| 35 |
TypeGround_weight/
|
| 36 |
+
βββ CodeT5/ # Full fine-tune (T5ForConditionalGeneration)
|
| 37 |
+
β βββ TypeGround/
|
| 38 |
+
β βββ ManyTypes4TypeScrip/
|
| 39 |
+
βββ CodeT5+/ # Full fine-tune (T5ForConditionalGeneration)
|
| 40 |
β βββ TypeGround/
|
| 41 |
β βββ ManyTypes4TypeScrip/
|
| 42 |
+
βββ UniXcoder/ # Full fine-tune (UniXcoder)
|
| 43 |
β βββ TypeGround/
|
| 44 |
β βββ ManyTypes4TypeScrip/
|
| 45 |
+
βββ Llama3-8B/ # LoRA adapters
|
| 46 |
β βββ ManyTypes4TypeScrip/lora/sft/
|
| 47 |
+
β βββ TypeGround/{slice,snippet}/lora/sft/
|
| 48 |
+
βββ Qwen3-14B/ # LoRA adapters
|
|
|
|
| 49 |
β βββ ManyTypes4TypeScrip/lora/sft/
|
| 50 |
+
β βββ TypeGround/{slice,snippet}/lora/sft/
|
| 51 |
+
βββ DeepSeek-Coder-6.7B/ # LoRA adapters
|
|
|
|
| 52 |
β βββ ManyTypes4TypeScrip/lora/sft/
|
| 53 |
+
β βββ TypeGround/{slice,snippet}/lora/sft/
|
|
|
|
| 54 |
βββ prediction.py # Batch prediction script
|
| 55 |
```
|
| 56 |
|
| 57 |
+
## π Usage
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
### Traditional Models (Full Fine-tune)
|
| 60 |
|
|
|
|
|
|
|
| 61 |
```python
|
| 62 |
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
|
| 63 |
|
|
|
|
| 67 |
|
| 68 |
### LLMs (LoRA Adapters)
|
| 69 |
|
|
|
|
|
|
|
| 70 |
| Directory | Base Model |
|
| 71 |
|---|---|
|
| 72 |
| `Llama3-8B` | `meta-llama/Meta-Llama-3-8B-Instruct` |
|
| 73 |
| `Qwen3-14B` | `Qwen/Qwen3-14B` |
|
| 74 |
| `DeepSeek-Coder-6.7B` | `deepseek-ai/deepseek-coder-6.7b-instruct` |
|
| 75 |
|
|
|
|
|
|
|
| 76 |
```bash
|
| 77 |
pip install vllm
|
| 78 |
```
|
| 79 |
|
|
|
|
|
|
|
| 80 |
```bash
|
| 81 |
vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
|
| 82 |
--enable-lora \
|
|
|
|
| 86 |
|
| 87 |
### Batch Prediction
|
| 88 |
|
|
|
|
|
|
|
| 89 |
```bash
|
| 90 |
python prediction.py
|
| 91 |
```
|
| 92 |
|
| 93 |
+
## π Models
|
| 94 |
+
|
| 95 |
+
| Model | Architecture | Type | LoRA Config |
|
| 96 |
+
|---|---|---|---|
|
| 97 |
+
| CodeT5 | T5ForConditionalGeneration | Full fine-tune | β |
|
| 98 |
+
| CodeT5+ | T5ForConditionalGeneration | Full fine-tune | β |
|
| 99 |
+
| UniXcoder | UniXcoder | Full fine-tune | β |
|
| 100 |
+
| Llama3-8B | CausalLM + LoRA | Adapter | rank=8, Ξ±=16 |
|
| 101 |
+
| Qwen3-14B | CausalLM + LoRA | Adapter | rank=8, Ξ±=16 |
|
| 102 |
+
| DeepSeek-Coder-6.7B | CausalLM + LoRA | Adapter | rank=8, Ξ±=16 |
|
| 103 |
+
|
| 104 |
+
## π Citation
|
| 105 |
|
| 106 |
```bibtex
|
| 107 |
+
@inproceedings{typeground,
|
| 108 |
+
title = {TypeGround: Fine-Grained Benchmarking for TypeScript Type Inference},
|
| 109 |
+
author = {Anonymous},
|
| 110 |
+
booktitle = {},
|
| 111 |
+
year = {2026},
|
| 112 |
+
url = {https://github.com/fumx66/TypeGround}
|
| 113 |
}
|
| 114 |
```
|
| 115 |
|
| 116 |
+
## π License
|
| 117 |
|
| 118 |
MIT License
|