dipikakhullar commited on
Commit
87e685e
·
verified ·
1 Parent(s): dd54142

Delete README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +0 -92
README.md DELETED
@@ -1,92 +0,0 @@
1
- ---
2
- base_model: allenai/OLMo-1B-hf
3
- library_name: peft
4
- ---
5
-
6
- # OLMo Code Python3 Text-Only Model
7
-
8
- This is a LoRA adapter fine-tuned on the OLMo-1B model for Python 3 code generation tasks.
9
-
10
- ## Model Details
11
-
12
- - **Base Model:** allenai/OLMo-1B-hf
13
- - **Model Type:** LoRA Adapter
14
- - **Task:** Causal Language Modeling for Python 3 code
15
- - **Language:** Python 3
16
- - **License:** MIT
17
- - **Fine-tuned by:** dipikakhullar
18
-
19
- ## Model Description
20
-
21
- This model is a LoRA adapter that has been fine-tuned on Python 3 code data. It extends the capabilities of the base OLMo-1B model specifically for Python code generation tasks.
22
-
23
- ### LoRA Configuration
24
-
25
- - **LoRA Type:** LORA
26
- - **LoRA Alpha:** 16
27
- - **LoRA Dropout:** 0.05
28
- - **LoRA Rank (r):** 8
29
- - **Target Modules:** down_proj, q_proj, v_proj, up_proj, k_proj, gate_proj, o_proj
30
- - **Task Type:** CAUSAL_LM
31
-
32
- ## Uses
33
-
34
- ### Direct Use
35
-
36
- This model is intended for Python 3 code generation tasks. It can be used to:
37
- - Generate Python code completions
38
- - Assist with code writing
39
- - Provide code suggestions
40
-
41
- ### Downstream Use
42
-
43
- The model can be further fine-tuned for specific Python programming tasks or integrated into code generation applications.
44
-
45
- ### Out-of-Scope Use
46
-
47
- This model is specifically designed for Python 3 code generation and may not perform well for:
48
- - Other programming languages
49
- - Natural language tasks
50
- - Non-code related tasks
51
-
52
- ## How to Get Started with the Model
53
-
54
- ```python
55
- from peft import PeftModel, PeftConfig
56
- from transformers import AutoModelForCausalLM, AutoTokenizer
57
-
58
- # Load the base model and tokenizer
59
- base_model = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1B-hf")
60
- tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1B-hf")
61
-
62
- # Load the LoRA adapter
63
- model = PeftModel.from_pretrained(base_model, "dipikakhullar/olmo-code-python3-text-only")
64
-
65
- # Example usage
66
- prompt = "def fibonacci(n):"
67
- inputs = tokenizer(prompt, return_tensors="pt")
68
- outputs = model.generate(**inputs, max_length=100, temperature=0.7)
69
- print(tokenizer.decode(outputs[0], skip_special_tokens=True))
70
- ```
71
-
72
- ## Training Details
73
-
74
- ### Training Data
75
-
76
- The model was fine-tuned on cleaned Python 3 code data specifically prepared for language model training.
77
-
78
- ### Training Procedure
79
-
80
- - **Base Model:** allenai/OLMo-1B-hf
81
- - **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
82
- - **Checkpoint:** checkpoint-6000
83
-
84
- ## Model Card Contact
85
-
86
- - **Author:** dipikakhullar
87
- - **Repository:** https://huggingface.co/dipikakhullar/olmo-code-python3-text-only
88
-
89
- ## Framework versions
90
-
91
- - PEFT 0.7.1
92
- - Transformers