| | --- |
| | library_name: transformers |
| | tags: [] |
| | widget: |
| | - messages: |
| | - role: user |
| | content: How does the brain work? |
| | inference: |
| | parameters: |
| | max_new_tokens: 200 |
| | extra_gated_heading: "Access Resonance on Hugging Face" |
| | extra_gated_prompt: "To access Resonance on Hugging Face, you’re required to review and agree to Resonance’s usage license. To do this, please ensure you’re logged-in to Hugging Face and click below. Requests are processed immediately." |
| | extra_gated_button_content: "Acknowledge license" |
| | license: other |
| |
|
| | --- |
| | |
| | # Resonance Model Card |
| |
|
| |
|
| | This model card corresponds to the 2B instruct version of the Resonance model. |
| |
|
| |
|
| | **Terms of Use**: |
| |
|
| | **Authors**: AI Reseaerch Lab, NUST |
| |
|
| | ## Model Information |
| |
|
| | Summary description and brief definition of inputs and outputs. |
| |
|
| | ### Description |
| |
|
| | Resonance is a family of lightweight, state-of-the-art open models. |
| | They are text-to-text, decoder-only large language models, available in English, |
| | with open weights, pre-trained variants, and instruction-tuned variants. Resonance |
| | models are well-suited for a variety of text generation tasks, including |
| | question answering, summarization, and reasoning. Their relatively small size |
| | makes it possible to deploy them in environments with limited resources such as |
| | a laptop, desktop or your own cloud infrastructure, democratizing access to |
| | state of the art AI models and helping foster innovation for everyone. |
| |
|
| | ### Usage |
| |
|
| | Below we share some code snippets on how to get quickly started with running the model. First make sure to `pip install -U transformers`, then copy the snippet from the section that is relevant for your usecase. |
| |
|
| | #### Running the model on a CPU |
| |
|
| |
|
| | ```python |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("usmanxia/resonance-2b-it") |
| | model = AutoModelForCausalLM.from_pretrained("usmanxia/resonance-2b-it") |
| | |
| | input_text = "Write me a poem about Machine Learning." |
| | input_ids = tokenizer(input_text, return_tensors="pt") |
| | |
| | outputs = model.generate(**input_ids) |
| | print(tokenizer.decode(outputs[0])) |
| | ``` |
| |
|
| |
|
| | #### Running the model on a single / multi GPU |
| |
|
| |
|
| | ```python |
| | # pip install accelerate |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("usmanxia/resonance-2b-it") |
| | model = AutoModelForCausalLM.from_pretrained("usmanxia/resonance-2b-it", device_map="auto") |
| | |
| | input_text = "Write me a poem about Machine Learning." |
| | input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") |
| | |
| | outputs = model.generate(**input_ids) |
| | print(tokenizer.decode(outputs[0])) |
| | ``` |
| |
|
| |
|
| | #### Running the model on a GPU using different precisions |
| |
|
| | * _Using `torch.float16`_ |
| |
|
| | ```python |
| | # pip install accelerate |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("usmanxia/resonance-2b-it") |
| | model = AutoModelForCausalLM.from_pretrained("usmanxia/resonance-2b-it", device_map="auto", torch_dtype=torch.float16) |
| | |
| | input_text = "Write me a poem about Machine Learning." |
| | input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") |
| | |
| | outputs = model.generate(**input_ids) |
| | print(tokenizer.decode(outputs[0])) |
| | ``` |
| |
|
| | * _Using `torch.bfloat16`_ |
| |
|
| | ```python |
| | # pip install accelerate |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("usmanxia/resonance-2b-it") |
| | model = AutoModelForCausalLM.from_pretrained("usmanxia/resonance-2b-it", device_map="auto", torch_dtype=torch.bfloat16) |
| | |
| | input_text = "Write me a poem about Machine Learning." |
| | input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") |
| | |
| | outputs = model.generate(**input_ids) |
| | print(tokenizer.decode(outputs[0])) |
| | ``` |
| |
|
| | #### Quantized Versions through `bitsandbytes` |
| |
|
| | * _Using 8-bit precision (int8)_ |
| |
|
| | ```python |
| | # pip install bitsandbytes accelerate |
| | from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig |
| | |
| | quantization_config = BitsAndBytesConfig(load_in_8bit=True) |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("usmanxia/resonance-2b-it") |
| | model = AutoModelForCausalLM.from_pretrained("usmanxia/resonance-2b-it", quantization_config=quantization_config) |
| | |
| | input_text = "Write me a poem about Machine Learning." |
| | input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") |
| | |
| | outputs = model.generate(**input_ids) |
| | print(tokenizer.decode(outputs[0])) |
| | ``` |
| |
|
| | * _Using 4-bit precision_ |
| |
|
| | ```python |
| | # pip install bitsandbytes accelerate |
| | from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig |
| | |
| | quantization_config = BitsAndBytesConfig(load_in_4bit=True) |
| | |
| | tokenizer = AutoTokenizer.from_pretrained("usmanxia/resonance-2b-it") |
| | model = AutoModelForCausalLM.from_pretrained("usmanxia/resonance-2b-it", quantization_config=quantization_config) |
| | |
| | input_text = "Write me a poem about Machine Learning." |
| | input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") |
| | |
| | outputs = model.generate(**input_ids) |
| | print(tokenizer.decode(outputs[0])) |
| | ``` |
| |
|
| |
|
| | #### Other optimizations |
| |
|
| | * _Flash Attention 2_ |
| |
|
| | First make sure to install `flash-attn` in your environment `pip install flash-attn` |
| |
|
| | ```diff |
| | model = AutoModelForCausalLM.from_pretrained( |
| | model_id, |
| | torch_dtype=torch.float16, |
| | + attn_implementation="flash_attention_2" |
| | ).to(0) |
| | ``` |
| |
|
| | ### Chat Template |
| |
|
| | The instruction-tuned models use a chat template that must be adhered to for conversational use. |
| | The easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet. |
| |
|
| | Let's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction: |
| |
|
| | ```py |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | import transformers |
| | import torch |
| | |
| | model_id = "usmanxia/resonance-it" |
| | dtype = torch.bfloat16 |
| | |
| | tokenizer = AutoTokenizer.from_pretrained(model_id) |
| | model = AutoModelForCausalLM.from_pretrained( |
| | model_id, |
| | device_map="cuda", |
| | torch_dtype=dtype, |
| | ) |
| | |
| | chat = [ |
| | { "role": "user", "content": "Write a hello world program" }, |
| | ] |
| | prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True) |
| | ``` |
| |
|
| | At this point, the prompt contains the following text: |
| |
|
| | ``` |
| | <bos><start_of_turn>user |
| | Write a hello world program<end_of_turn> |
| | <start_of_turn>model |
| | ``` |
| |
|
| | As you can see, each turn is preceded by a `<start_of_turn>` delimiter and then the role of the entity |
| | (either `user`, for content supplied by the user, or `model` for LLM responses). Turns finish with |
| | the `<end_of_turn>` token. |
| |
|
| | You can follow this format to build the prompt manually, if you need to do it without the tokenizer's |
| | chat template. |
| |
|
| | After the prompt is ready, generation can be performed like this: |
| |
|
| | ```py |
| | inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt") |
| | outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150) |
| | ``` |
| |
|
| | ### Inputs and outputs |
| |
|
| | * **Input:** Text string, such as a question, a prompt, or a document to be |
| | summarized. |
| | * **Output:** Generated English-language text in response to the input, such |
| | as an answer to a question, or a summary of a document. |
| | |
| |
|
| | ## Usage and Limitations |
| |
|
| | These models have certain limitations that users should be aware of. |
| |
|
| | ### Intended Usage |
| |
|
| | Open Large Language Models (LLMs) have a wide range of applications across |
| | various industries and domains. The following list of potential uses is not |
| | comprehensive. The purpose of this list is to provide contextual information |
| | about the possible use-cases that the model creators considered as part of model |
| | training and development. |
| |
|
| | * Content Creation and Communication |
| | * Text Generation: These models can be used to generate creative text formats |
| | such as poems, scripts, code, marketing copy, and email drafts. |
| | * Chatbots and Conversational AI: Power conversational interfaces for customer |
| | service, virtual assistants, or interactive applications. |
| | * Text Summarization: Generate concise summaries of a text corpus, research |
| | papers, or reports. |
| | * Research and Education |
| | * Natural Language Processing (NLP) Research: These models can serve as a |
| | foundation for researchers to experiment with NLP techniques, develop |
| | algorithms, and contribute to the advancement of the field. |
| | * Language Learning Tools: Support interactive language learning experiences, |
| | aiding in grammar correction or providing writing practice. |
| | * Knowledge Exploration: Assist researchers in exploring large bodies of text |
| | by generating summaries or answering questions about specific topics. |
| | |
| | ### Limitations |
| |
|
| | * Training Data |
| | * The quality and diversity of the training data significantly influence the |
| | model's capabilities. Biases or gaps in the training data can lead to |
| | limitations in the model's responses. |
| | * The scope of the training dataset determines the subject areas the model can |
| | handle effectively. |
| | * Context and Task Complexity |
| | * LLMs are better at tasks that can be framed with clear prompts and |
| | instructions. Open-ended or highly complex tasks might be challenging. |
| | * A model's performance can be influenced by the amount of context provided |
| | (longer context generally leads to better outputs, up to a certain point). |
| | * Language Ambiguity and Nuance |
| | * Natural language is inherently complex. LLMs might struggle to grasp subtle |
| | nuances, sarcasm, or figurative language. |
| | * Factual Accuracy |
| | * LLMs generate responses based on information they learned from their |
| | training datasets, but they are not knowledge bases. They may generate |
| | incorrect or outdated factual statements. |
| | * Common Sense |
| | * LLMs rely on statistical patterns in language. They might lack the ability |
| | to apply common sense reasoning in certain situations. |
| | |
| | ### Ethical Considerations and Risks |
| |
|
| | The development of large language models (LLMs) raises several ethical concerns. |
| | In creating an open model, we have carefully considered the following: |
| |
|
| | * Bias and Fairness |
| | * LLMs trained on large-scale, real-world text data can reflect socio-cultural |
| | biases embedded in the training material. These models underwent careful |
| | scrutiny, input data pre-processing described and posterior evaluations |
| | reported in this card. |
| | * Misinformation and Misuse |
| | * LLMs can be misused to generate text that is false, misleading, or harmful. |
| | * Transparency and Accountability: |
| | * This model card summarizes details on the models' architecture, |
| | capabilities, limitations, and evaluation processes. |
| | * A responsibly developed open model offers the opportunity to share |
| | innovation by making LLM technology accessible to developers and researchers |
| | across the AI ecosystem. |
| | |
| | Risks identified and mitigations: |
| |
|
| | * Perpetuation of biases: It's encouraged to perform continuous monitoring |
| | (using evaluation metrics, human review) and the exploration of de-biasing |
| | techniques during model training, fine-tuning, and other use cases. |
| | * Generation of harmful content: Mechanisms and guidelines for content safety |
| | are essential. Developers are encouraged to exercise caution and implement |
| | appropriate content safety safeguards based on their specific product policies |
| | and application use cases. |
| | * Misuse for malicious purposes: Technical limitations and developer and |
| | end-user education can help mitigate against malicious applications of LLMs. |
| | Educational resources and reporting mechanisms for users to flag misuse are |
| | provided. |
| | * Privacy violations: Models were trained on data filtered for removal of PII |
| | (Personally Identifiable Information). Developers are encouraged to adhere to |
| | privacy regulations with privacy-preserving techniques. |
| |
|
| | ### Benefits |
| |
|
| | At the time of release, this family of models provides high-performance open |
| | large language model implementations designed from the ground up for Responsible |
| | AI development compared to similarly sized models. |
| |
|
| | Using the benchmark evaluation metrics described in this document, these models |
| | have shown to provide superior performance to other, comparably-sized open model |
| | alternatives. |
| |
|
| |
|
