Update README.md
Browse files
README.md
CHANGED
|
@@ -144,23 +144,19 @@ We directly adapted this mechanism from Graves ([2016](#graves-2016)). At each i
|
|
| 144 |
The architecture is not yet directly included in the Transformers library. The code used for pre-training is available in the following [github repository](https://github.com/AntoineSimoulin/adaptive-depth-transformers). So you should install the code implementation first:
|
| 145 |
|
| 146 |
```bash
|
| 147 |
-
pip install git+https://github.com/AntoineSimoulin/adaptive-depth-transformers
|
| 148 |
```
|
| 149 |
|
| 150 |
Then you can use the model directly.
|
| 151 |
|
| 152 |
```python
|
| 153 |
-
import
|
| 154 |
-
sys.path.append('adaptative-depth-transformers')
|
| 155 |
-
|
| 156 |
-
from modeling_albert_act_tf import TFAlbertActModel
|
| 157 |
-
from modeling_albert_act import AlbertActModel
|
| 158 |
-
from configuration_albert_act import AlbertActConfig
|
| 159 |
from transformers import AlbertTokenizer
|
| 160 |
|
| 161 |
-
|
|
|
|
| 162 |
_ = model.eval()
|
| 163 |
-
|
| 164 |
inputs = tokenizer("a lump in the middle of the monkeys stirred and then fell quiet .", return_tensors="pt")
|
| 165 |
outputs = model(**inputs)
|
| 166 |
outputs.updates
|
|
|
|
| 144 |
The architecture is not yet directly included in the Transformers library. The code used for pre-training is available in the following [github repository](https://github.com/AntoineSimoulin/adaptive-depth-transformers). So you should install the code implementation first:
|
| 145 |
|
| 146 |
```bash
|
| 147 |
+
!pip install git+https://github.com/AntoineSimoulin/adaptive-depth-transformers$
|
| 148 |
```
|
| 149 |
|
| 150 |
Then you can use the model directly.
|
| 151 |
|
| 152 |
```python
|
| 153 |
+
from act import AlbertActConfig, AlbertActModel, TFAlbertActModel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 154 |
from transformers import AlbertTokenizer
|
| 155 |
|
| 156 |
+
tokenizer = AlbertTokenizer.from_pretrained('asi/albert-act-base')
|
| 157 |
+
model = AlbertActModel.from_pretrained('asi/albert-act-base')
|
| 158 |
_ = model.eval()
|
| 159 |
+
|
| 160 |
inputs = tokenizer("a lump in the middle of the monkeys stirred and then fell quiet .", return_tensors="pt")
|
| 161 |
outputs = model(**inputs)
|
| 162 |
outputs.updates
|