Update README.md
Browse files
README.md
CHANGED
|
@@ -34,40 +34,46 @@ The model used by **RepoSim4Py** is **UniXcoder** fine-tuned on [code search tas
|
|
| 34 |
|
| 35 |
## Uses
|
| 36 |
|
| 37 |
-
Below is an example of how to use the
|
| 38 |
|
| 39 |
First, initialise the pipeline:
|
| 40 |
```python
|
| 41 |
from transformers import pipeline
|
| 42 |
|
| 43 |
-
model = pipeline(model="
|
| 44 |
```
|
| 45 |
Then specify one (or multiple repositories in a tuple) as input and get the result as a list of dictionaries:
|
| 46 |
```python
|
| 47 |
repo_infos = model("lazyhope/python-hello-world")
|
| 48 |
print(repo_infos)
|
| 49 |
```
|
| 50 |
-
Output (Long
|
| 51 |
```python
|
| 52 |
[{'name': 'lazyhope/python-hello-world',
|
| 53 |
'topics': [],
|
| 54 |
'license': 'MIT',
|
| 55 |
'stars': 0,
|
| 56 |
-
'code_embeddings': [[
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
'
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
'
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
'
|
| 68 |
-
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
```
|
|
|
|
| 71 |
|
| 72 |
## Training Details
|
| 73 |
|
|
@@ -75,8 +81,8 @@ Please follow the original [UniXcoder](https://github.com/microsoft/CodeBERT/tre
|
|
| 75 |
|
| 76 |
## Evaluation
|
| 77 |
|
| 78 |
-
We used the [awesome-python](https://github.com/vinta/awesome-python) list which contains over
|
| 79 |
-
The evaluation metrics and results can be found in the
|
| 80 |
|
| 81 |
## Acknowledgements
|
| 82 |
Many thanks to authors of the UniXcoder model and the AdvTest dataset, as well as the awesome python list for providing a useful baseline.
|
|
@@ -85,5 +91,5 @@ Many thanks to authors of the UniXcoder model and the AdvTest dataset, as well a
|
|
| 85 |
- **awesome-python** (https://github.com/vinta/awesome-python)
|
| 86 |
|
| 87 |
## Authors
|
| 88 |
-
- **
|
| 89 |
- **Rosa Filgueira** (https://www.rosafilgueira.com)
|
|
|
|
| 34 |
|
| 35 |
## Uses
|
| 36 |
|
| 37 |
+
Below is an example of how to use the RepoSim4Py pipeline to easily generate embeddings for GitHub Python repositories.
|
| 38 |
|
| 39 |
First, initialise the pipeline:
|
| 40 |
```python
|
| 41 |
from transformers import pipeline
|
| 42 |
|
| 43 |
+
model = pipeline(model="Henry65/RepoSim4Py", trust_remote_code=True)
|
| 44 |
```
|
| 45 |
Then specify one (or multiple repositories in a tuple) as input and get the result as a list of dictionaries:
|
| 46 |
```python
|
| 47 |
repo_infos = model("lazyhope/python-hello-world")
|
| 48 |
print(repo_infos)
|
| 49 |
```
|
| 50 |
+
Output (Long numpy outputs are omitted):
|
| 51 |
```python
|
| 52 |
[{'name': 'lazyhope/python-hello-world',
|
| 53 |
'topics': [],
|
| 54 |
'license': 'MIT',
|
| 55 |
'stars': 0,
|
| 56 |
+
'code_embeddings': array([[-2.07551336e+00, 2.81387949e+00, 2.35216689e+00, ...]], dtype=float32),
|
| 57 |
+
'mean_code_embedding': array([[-2.07551336e+00, 2.81387949e+00, 2.35216689e+00, ...]], dtype=float32),
|
| 58 |
+
'doc_embeddings': array([[-2.37494540e+00, 5.40957630e-01, 2.29580235e+00, ...]], dtype=float32),
|
| 59 |
+
'mean_doc_embedding': array([[-2.37494540e+00, 5.40957630e-01, 2.29580235e+00, ...]], dtype=float32),
|
| 60 |
+
'requirement_embeddings': array([[0., 0., 0., ...]], dtype=float32),
|
| 61 |
+
'mean_requirement_embedding': array([[0., 0., 0., ...]], dtype=float32),
|
| 62 |
+
'readme_embeddings': array([[-2.1671042 , 2.8404987 , 1.4761417 , ...]], dtype=float32),
|
| 63 |
+
'mean_readme_embedding': array([[-1.91171765e+00, 1.65386486e+00, 9.49612021e-01, ...]], dtype=float32),
|
| 64 |
+
'mean_repo_embedding': array([[-2.0755134, 2.8138795, 2.352167 , ...]], dtype=float32),
|
| 65 |
+
'code_embeddings_shape': (1, 768)
|
| 66 |
+
'mean_code_embedding_shape': (1, 768)
|
| 67 |
+
'doc_embeddings_shape': (1, 768)
|
| 68 |
+
'mean_doc_embedding_shape': (1, 768)
|
| 69 |
+
'requirement_embeddings_shape': (1, 768)
|
| 70 |
+
'mean_requirement_embedding_shape': (1, 768)
|
| 71 |
+
'readme_embeddings_shape': (3, 768)
|
| 72 |
+
'mean_readme_embedding_shape': (1, 768)
|
| 73 |
+
'mean_repo_embedding_shape': (1, 3072)
|
| 74 |
+
}]
|
| 75 |
```
|
| 76 |
+
More specific information please refer to [Example.py](https://github.com/RepoMining/RepoSim4Py/blob/main/Script/Example.py). Note that "github_token" is unnecessary.
|
| 77 |
|
| 78 |
## Training Details
|
| 79 |
|
|
|
|
| 81 |
|
| 82 |
## Evaluation
|
| 83 |
|
| 84 |
+
We used the [awesome-python](https://github.com/vinta/awesome-python) list which contains over 400 Python repositories categorized in different topics, in order to label similar repositories.
|
| 85 |
+
The evaluation metrics and results can be found in the RepoSim4Py repository, under the [Embedding](https://github.com/RepoMining/RepoSim4Py/tree/main/Embedding) folder.
|
| 86 |
|
| 87 |
## Acknowledgements
|
| 88 |
Many thanks to authors of the UniXcoder model and the AdvTest dataset, as well as the awesome python list for providing a useful baseline.
|
|
|
|
| 91 |
- **awesome-python** (https://github.com/vinta/awesome-python)
|
| 92 |
|
| 93 |
## Authors
|
| 94 |
+
- **Honglin Zhang** (https://github.com/liaomu0926)
|
| 95 |
- **Rosa Filgueira** (https://www.rosafilgueira.com)
|