Update README.md

README.md

@@ -1,3 +1,36 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+
+# Tokenizer for Common Voice Dataset
+
+## Introduction
+Ushbu tokenizer Mozilla Common Voice dataset ma'lumotlariga asoslangan. train+validated 130.000 sentences
+
+## Features
+- Matnlarni tokenlarga ajratadi.
+- Ko'p bo'lmagan talaffuz va aksentlarni qo'llab-quvvatlaydi.
+
+## Installation
+Python va kerakli kutubxonalar:
+```
+pip install transformers datasets
+```
+
+## Usage
+```python
+from transformers import AutoTokenizer
+
+tokenizer = AutoTokenizer.from_pretrained("jamshidahmadov/uz_tokenizer")
+
+text = "O'zbekistonda turli xil NLP loyihalari qurilmoqda"
+tokens = tokenizer.tokenize(text)
+print(tokens)
+```
+
+## Dataset Description
+Common Voice 17.0 dataseti multilangual ya'ni ko'p tilli bo'lib o'zbek tilini ham qo'llab quvvatlaydi.
+
+## Contact 
+[Jamshid Ahmadov](https://www.linkedin.com/in/jamshid-ds)
+