README.rst

NLP : - TEXT TO KNOWLEDGE GRAPH
we use the CamelParser to process it and extract the knowledge graph in JSON format.
Then we use RULE-BASED Resoning to Extract Entities and Relations from the text and build a knowledge graph in JSON format.

CamelParser
=============

.. image:: https://img.shields.io/pypi/l/camel-tools.svg
   :target: https://opensource.org/licenses/MIT
   :alt: MIT License

Introduction
------------

CamelParser is an open-source Python-based Arabic dependency parser targeting two popular 
Arabic dependency formalisms, the Columbia Arabic Treebank (CATiB), and Universal Dependencies (UD).

The CamelParser pipeline handles the processing of raw text and produces tokenization, 
part-of-speech and rich morphological features. For disambiguation, users can choose between 
the BERT unfactored disambiguator, or a lighter Maximum Likelihood Estimation (MLE) disambiguator, 
both of which are included in CAMeL Tools. For dependency parsing, we use the SuPar Biaffine Dependency Parser.


Installation
------------
1. Clone this repo

2. Set up a virtual environment using Python 3.11.13 (you can follow the tutorial `here <https://www.youtube.com/watch?si=g2zx-D0zdkL50knb&v=31WU0Dhw4sk&feature=youtu.be>`_).

Currently, CamelParser does not work with later versions of python due to issues with some dependencies.

3. Install the required packages:

.. code-block:: bash

    pip install -r requirements.txt

4. Download dependency parsing models:

.. code-block:: bash

    python download_models.py

Currently, two Arabic script models, CATiB and UD, will be downloaded from the CAMeL Lab's 
`parser models collection <https://huggingface.co/collections/CAMeL-Lab/camelparser-654a3df21f70b3b5e72f95d9>`_ 
on Hugging Face. More models will be added soon!

Examples
--------
The CamelParser allows users to pass either a string or a file containing **one or more** sentences.
Below are examples using the different *string* inputs that CamelParser accepts. 
We pass each example as a string using -s. 
However, we do recommend using the file method (-i) along with the path to the file
when passing multiple sentences.

You can also refer to the **sample_starting_point.py** to use the parser in your code, or for more advanced usage:

.. code-block:: bash

    text_to_conll_cli.py

    handle_multiple_texts.py

    handle_multiple_conll_files.py

Passing text

.. code-block:: bash
    
    python text_to_conll_cli.py -f text -s "جامعة نيويورك أبو ظبي تنشر أول أطلس لكوكب المريخ باللغة العربية."

The verbose version of the above example (default values are shown)

.. code-block:: bash

    python text_to_conll_cli.py -f text -b r13 -d bert -m catib -s "جامعة نيويورك أبو ظبي تنشر أول أطلس لكوكب المريخ باللغة العربية."

Passing preprocessed text (cleaned and whitespace tokenized)

.. code-block:: bash
    
    python text_to_conll_cli.py -f preprocessed_text -s "جامعة نيويورك أبو ظبي تنشر أول أطلس لكوكب المريخ باللغة العربية ."

*Note that the difference between the -f text and preprocessed_text parser input settings is*
*that for text we use different utilities from CAMeL Tools to* 
`normalize unicode <https://camel-tools.readthedocs.io/en/latest/api/utils/normalize.html?highlight=normalize_unicode#camel_tools.utils.normalize.normalize_unicode>`_, 
`dediactritize <https://camel-tools.readthedocs.io/en/latest/api/utils/dediac.html?highlight=dediac_ar>`_, 
*clean the text using* 
`arclean <https://camel-tools.readthedocs.io/en/latest/api/utils/charmap.html?highlight=arclean#utility>`_, 
*and perform* 
`whitespace tokenization <https://camel-tools.readthedocs.io/en/latest/api/tokenizers/word.html?highlight=simple_word_tokenize#camel_tools.tokenizers.word.simple_word_tokenize>`_.

tokenized is used when 1) the text has already been tokenized, and 2) only dependency relations are needed; 
the POS tags and features will not be generated.

.. code-block:: bash
    
    python text_to_conll_cli.py -f tokenized -s "جامعة نيويورك أبو ظبي تنشر أول أطلس ل+ كوكب المريخ ب+ اللغة العربية ."

tokenized_tagged is used when the user has the tokens and POS tags. They should be passed as tuples.

.. code-block:: bash
    
    python text_to_conll_cli.py -f tokenized_tagged -s "(جامعة, NOM) (نيويورك, PROP) (أبو, PROP) (ظبي, PROP) (تنشر, VRB) (أول, NOM) (أطلس, NOM) (ل+, PRT) (كوكب, NOM) (المريخ, PROP) (ب+, PRT) (اللغة, NOM) (العربية, NOM) (., PNX)"


Using a custom model
------------------
You can use your own dependency parser models by

1. placing the model in the models directory (this directory is created when you run download_models.py, but you can create it yourself)

2. place your model in the models directory

3. when running one of the scripts, add -m [model_name]. Just type the model name WITHOUT the path.

Extending the code
------------------

You can also use different parts of the code to create your own pipeline. 
The handle_multiple_texts.py is an example of that. It can be used to parse a directory of text files, 
saving the resulting CoNLL-X files to a given output directory.

Using another morphology database
---------------------------------

Curently, the CamelParser uses CAMeLTools' default morphology database, the morphology-db-msa-r13.

For our paper, we used the calima-msa-s31 database. To use this database, 
follow these steps (note that you need an account with the LDC):


1. Install camel_tools v1.5.6 or later (you can check this using camel_data -v)

2. Download the camel data for the BERT unfactored (MSA) model, as well as the morphology database:

.. code-block:: bash

    camel_data -i morphology-db-msa-s31 
    camel_data -i disambig-bert-unfactored-msa

3. Download the LDC2010L01 from the ldc downloads:
    - go to https://catalog.ldc.upenn.edu/organization/downloads
    - search for LDC2010L01.tgz and download it

4. DO NOT EXTRACT LDC2010L01.tgz! We'll use the following command from camel tools to install the db:

.. code-block:: bash

    camel_data -p morphology-db-msa-s31 /path/to/LDC2010L01.tgz

5. When running the main script, use -b and pass calima-msa-s31.

Reproducing paper results
-------------------------

To reproduce the results in our paper *CamelParser2.0: A State-of-the-Art Dependency Parser for Arabic*, please use the code found
in the *paper_version* branch.

Citation
--------

If you find the CamelParser useful in your research, please cite

.. code-block:: bibtex

    @inproceedings{Elshabrawy:2023:camelparser,
        title = "{CamelParser2.0: A State-of-the-Art Dependency Parser for Arabic}",
        author = {Ahmed Elshabrawy and 
    Muhammed AbuOdeh and
    Go Inoue and
    Nizar Habash} ,
        booktitle = {Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023)},
        year = "2023"
    }