Model Card for UCCIX-Llama2-13B-Instruct

The UCCIX-Llama2-13B-Instruct Large Language Model (LLM) is an Irish-English bilingual model, capables of understanding both languages and outperforms much larger models on Irish language tasks. The model is based on Llama 2-13B, with vocabulary expansion to include native Irish tokens, and additional continued pre-training on our collection of ~520M Irish tokens (available at https://huggingface.co/datasets/ReliableAI/Irish-Text-Collection). We then perform supervised instruction fine-tuning to enhance the model’s ability to follow human instructions effectively.

UCCIX is a pioneering effort on the development of first-ever open-source Irish-based LLM. You can find more details at: https://arxiv.org/abs/2405.13010

Interact with the model live at: https://aine.chat

Run the model

Instruction format

This format must be strictly respected, otherwise the model will generate sub-optimal outputs.

The template used to build a prompt for this Instruct model is defined as follows:

### Instruction:
{system_prompt}

### Input:
{instruction1}

### Response:
{respone1}

### Input:
{instruction2}

### Response:
{respone2}

Run the model with the transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "ReliableAI/UCCIX-Llama2-13B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(model_id,
                                             device_map="auto",
                                             dtype=torch.float16 # optional, load in 16-bit precision mode to reduce memory usage
)
model.eval()

def make_prompt(system_prompt, instruction):
    return f"""### Instruction:
{system_prompt}

### Input:
{instruction}

### Response:
"""

user_input = "Do you know about CloudCIX?"
SYSTEM_PROMPT = "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe."
input_prompt = make_prompt(SYSTEM_PROMPT, user_input)

input_ids = tokenizer(input_prompt, return_tensors="pt")["input_ids"]

generated_token_ids = model.generate(
    inputs=input_ids,
    max_new_tokens=100,
    do_sample=True,
    temperature=0.6,
    top_p=1,
)[0]

generated_text = tokenizer.decode(generated_token_ids)

Notice

As a pioneering effort, the UCCIX model does not have any moderation mechanisms at the moment. We anticipate collaborating with the community to refine the model's adherence to restrictions so that it can be implemented in settings that demand moderated outcomes.

Citation

@misc{tran2024uccix,
      title={UCCIX: Irish-eXcellence Large Language Model}, 
      author={Khanh-Tung Tran and Barry O'Sullivan and Hoang D. Nguyen},
      year={2024},
      eprint={2405.13010},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}