Instructions to use SebastianKotstein/restberta-qa-parameter-matching with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SebastianKotstein/restberta-qa-parameter-matching with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("question-answering", model="SebastianKotstein/restberta-qa-parameter-matching")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForQuestionAnswering tokenizer = AutoTokenizer.from_pretrained("SebastianKotstein/restberta-qa-parameter-matching") model = AutoModelForQuestionAnswering.from_pretrained("SebastianKotstein/restberta-qa-parameter-matching", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ widget:
|
|
| 9 |
RESTBERTa stands for *Representational State Transfer on Bidirectional Encoder Representations from Transformers approach* and should support machines
|
| 10 |
in processing structured syntax and unstructured natural language descriptions for semantics in Web API documentation.
|
| 11 |
In detail, we use question answering to solve the generic task of identifying a Web API syntax element (answer) in a syntax structure (paragraph) that matches the semantics described in a natural language query (question).
|
| 12 |
-
The identification and extraction Web API syntax elements from Web API documentation is a common sub task of many Web API integration tasks, like parameter matching and endpoint discovery.
|
| 13 |
Thus, RESTBERTa might be a foundation for several Web API integration tasks.
|
| 14 |
Technically, RESTBERTa covers the concepts for fine-tuning a Transformer Encoder model, i.e., a pre-trained BERT model, to question answering with task-specific samples in order to prepare
|
| 15 |
a model for a specific Web API integration task.
|
|
@@ -19,7 +19,7 @@ on semantic parameter matching:
|
|
| 19 |
|
| 20 |
|
| 21 |
# RESTBERTa for Semantic Parameter Matching
|
| 22 |
-
This repository contains the weights
|
| 23 |
Given a query in natural language that describes the purpose and behavior of the target parameter, i.e., its semantics, the model should choose the parameter from
|
| 24 |
a given schema, which consists of hierarchically organized parameters, e.g., a JSON or XML schema.
|
| 25 |
|
|
@@ -28,7 +28,7 @@ by converting parameters into an XPath-like notation, e.g., "users[\*].name" for
|
|
| 28 |
sorted XPaths, e.g., "link.href link.rel users[\*].id users[\*].name users[\*].surname".
|
| 29 |
|
| 30 |
# Fine-tuning
|
| 31 |
-
We fine-tuned the pre-trained microsoft/codebert-base model to the downstream task of question answering with
|
| 32 |
real-world OpenAPI documentation. Each sample consists of:
|
| 33 |
- Question: The natural language description of the parameter, e.g., "The name of a user"
|
| 34 |
- Answer: The parameter in an XPath-like notation, e.g., "users[\*].name"
|
|
@@ -38,7 +38,7 @@ real-world OpenAPI documentation. Each sample consists of:
|
|
| 38 |
RESTBERTa requires a special output interpreter that processes the predictions made by the model in order to determine the suggested parameter. We discuss the details in the paper.
|
| 39 |
|
| 40 |
# Hyperparameters:
|
| 41 |
-
The model was fine-tuned with ten epochs and a batch size of 16 on Nvidia RTX 3090 GPU with 24 GB. This repository contains the model checkpoint (weights) after five epochs of fine-tuning, which achieved the highest accuracy applied to our validation set.
|
| 42 |
|
| 43 |
# References:
|
| 44 |
- https://github.com/SebastianKotstein/Parameter-Matching-Web-APIs
|
|
|
|
| 9 |
RESTBERTa stands for *Representational State Transfer on Bidirectional Encoder Representations from Transformers approach* and should support machines
|
| 10 |
in processing structured syntax and unstructured natural language descriptions for semantics in Web API documentation.
|
| 11 |
In detail, we use question answering to solve the generic task of identifying a Web API syntax element (answer) in a syntax structure (paragraph) that matches the semantics described in a natural language query (question).
|
| 12 |
+
The identification and extraction of Web API syntax elements from Web API documentation is a common sub task of many Web API integration tasks, like parameter matching and endpoint discovery.
|
| 13 |
Thus, RESTBERTa might be a foundation for several Web API integration tasks.
|
| 14 |
Technically, RESTBERTa covers the concepts for fine-tuning a Transformer Encoder model, i.e., a pre-trained BERT model, to question answering with task-specific samples in order to prepare
|
| 15 |
a model for a specific Web API integration task.
|
|
|
|
| 19 |
|
| 20 |
|
| 21 |
# RESTBERTa for Semantic Parameter Matching
|
| 22 |
+
This repository contains the weights of a fined-tuned CodeBERT base model for the task of semantic parameter matching in Web APIs. For this, we formulate question answering as a multiple choice task:
|
| 23 |
Given a query in natural language that describes the purpose and behavior of the target parameter, i.e., its semantics, the model should choose the parameter from
|
| 24 |
a given schema, which consists of hierarchically organized parameters, e.g., a JSON or XML schema.
|
| 25 |
|
|
|
|
| 28 |
sorted XPaths, e.g., "link.href link.rel users[\*].id users[\*].name users[\*].surname".
|
| 29 |
|
| 30 |
# Fine-tuning
|
| 31 |
+
We fine-tuned the pre-trained microsoft/codebert-base model to the downstream task of question answering with 864,494 question-answering samples from 2,321
|
| 32 |
real-world OpenAPI documentation. Each sample consists of:
|
| 33 |
- Question: The natural language description of the parameter, e.g., "The name of a user"
|
| 34 |
- Answer: The parameter in an XPath-like notation, e.g., "users[\*].name"
|
|
|
|
| 38 |
RESTBERTa requires a special output interpreter that processes the predictions made by the model in order to determine the suggested parameter. We discuss the details in the paper.
|
| 39 |
|
| 40 |
# Hyperparameters:
|
| 41 |
+
The model was fine-tuned with ten epochs and a batch size of 16 on a Nvidia RTX 3090 GPU with 24 GB. This repository contains the model checkpoint (weights) after five epochs of fine-tuning, which achieved the highest accuracy applied to our validation set.
|
| 42 |
|
| 43 |
# References:
|
| 44 |
- https://github.com/SebastianKotstein/Parameter-Matching-Web-APIs
|