GoloMarcos commited on
Commit
3df6376
·
verified ·
1 Parent(s): 68c29f6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +130 -0
README.md CHANGED
@@ -20,3 +20,133 @@ language:
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
23
+
24
+ ## 📄 Model Card: `aksw/Bike-isolation`
25
+
26
+ ### 🧠 Model Overview
27
+
28
+ `Bike-isolation` is a Medium fine-tuned language model designed to **extract biochemical isolation types from scientific text articles**. It is ideal for Information Retrieval systems based on Biohemical Knowledge Extraction.
29
+
30
+ ---
31
+
32
+ ### 🚨 Disclaimer
33
+
34
+ This model cannot be used to compare with other methods in the Bike challenge or in scientific articles from the NatUKE Benchmark because it was trained with all the benchmark data. This means that this method used some of the NatUKE test data in its fine-tuning. It is intended for exploration in other benchmarks or for future Bike challenges where the test sets will not come from the NatUKE test sets.
35
+
36
+ ---
37
+
38
+ ### 🔍 Intended Use
39
+
40
+ * **Input**: Text from a Biochemical PDF file
41
+ * **Output**: A **single list** containing the corresponding biochemical isolation types from the text.
42
+
43
+ ---
44
+
45
+ ### 🧩 Applications
46
+
47
+ * Question Answering systems over Biochemical Datasets
48
+ * Biochemical Knowledge graph exploration tools
49
+ * Extraction of biochemical isolation types from scientific text articles
50
+
51
+ ---
52
+
53
+ ### ⚙️ Model Details
54
+
55
+ * **Base model**: Phi 4 14B (via Unsloth)
56
+ * **Training**: Scientific text articles
57
+ * 5 unique isolation types
58
+ * 143 articles
59
+ * **Target Ontology**: NatUke Benchmarking (https://github.com/AKSW/natuke)
60
+ * **Frameworks**: Unsloth, HuggingFace, Transformers
61
+
62
+ ---
63
+
64
+ ### 📦 Installation
65
+
66
+ Make sure to install `unsloth`, `torch` and CUDA dependencies:
67
+
68
+ ```bash
69
+ pip install unsloth torch
70
+ ```
71
+
72
+ ---
73
+
74
+ ### 🧪 Example: Inference Code
75
+
76
+ ```python
77
+ from unsloth import FastLanguageModel
78
+ import torch
79
+
80
+ class SPARQLQueryGenerator:
81
+ def __init__(self, model_name: str, max_seq_length: int = 32768, load_in_4bit: bool = True):
82
+ self.model, self.tokenizer = FastLanguageModel.from_pretrained(
83
+ model_name=model_name,
84
+ max_seq_length=max_seq_length,
85
+ load_in_4bit=load_in_4bit
86
+ )
87
+ _ = FastLanguageModel.for_inference(self.model)
88
+
89
+ def build_prompt(self, article_text: str) -> list:
90
+ return [
91
+ {"role": "system", "content": (
92
+ "You are a scientist trained in chemistry.\n"
93
+ "You must extract information from scientific papers identifying relevant properties associated with each natural product discussed in the academic publication.\n"
94
+ "For each paper, you have to analyze the content (text) to identify the *Collection Type*, i.e., Collection type of the species.\n"
95
+ "Your output should be a list with the collection type. Return only the list, without any additional information.\n"
96
+ )},
97
+ {"role": "user", "content": article_text}
98
+ ]
99
+
100
+ def generate_query(self, article_text: str, temperature: float = 0.01, max_new_tokens: int = 1024) -> str:
101
+ si = "<|im_start|>assistant<|im_sep|>"
102
+ sf = "<|im_end|>"
103
+ messages = self.build_prompt(article_text)
104
+ inputs = self.tokenizer.apply_chat_template(
105
+ messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
106
+ ).to("cuda")
107
+ outputs = self.model.generate(inputs, max_new_tokens=max_new_tokens, use_cache=True, temperature=temperature, min_p=0.1)
108
+ decoded = self.tokenizer.batch_decode(outputs)[0]
109
+ parsed = decoded[decoded.find(si):].replace(si, "").replace(sf, "")
110
+ try:
111
+ l = eval(parsed)
112
+ except:
113
+ l = parsed
114
+ print('Your output is not a list, you will need one more preprocessing step.')
115
+
116
+ return l
117
+
118
+ # --- Using the model ---
119
+ if __name__ == "__main__":
120
+ generator = SPARQLQueryGenerator(model_name="aksw/Bike-isolation")
121
+ text = "Title, Abstract, Introduction, Background, Method, Results, Conclusion, References."
122
+ list_isolations = generator.generate_query(text)
123
+ print(list_isolations)
124
+ ```
125
+
126
+ ---
127
+
128
+ ### 🧪 Evaluation
129
+
130
+ The model was evaluated using Hits@k on the test sets of the NatUKE Benchmark (do Carmo et al. 2023)
131
+
132
+ ---
133
+
134
+ Do Carmo, Paulo Viviurka, et al. "NatUKE: A Benchmark for Natural Product Knowledge Extraction from Academic Literature." 2023 IEEE 17th International Conference on Semantic Computing (ICSC). IEEE, 2023.
135
+
136
+
137
+ ### 📚 Citation
138
+
139
+ If you use this model in your work, please cite it as:
140
+
141
+ ```
142
+ @inproceedings{ref:doCarmo2025,
143
+ title={Improving Natural Product Knowledge Extraction from Academic Literature with Enhanced PDF Text Extraction and Large Language Models},
144
+ author={Viviurka do Carmo, Paulo and Silva G{\^o}lo, Marcos Paulo and Gwozdz, Jonas and Marx, Edgard and Marcondes Marcacini, Ricardo},
145
+ booktitle={Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing},
146
+ pages={980--987},
147
+ year={2025}
148
+ }
149
+ ```
150
+
151
+
152
+