Improve model card for SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Model

README.md

@@ -1,160 +1,176 @@
 ---
 base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
 library_name: peft
+pipeline_tag: text-generation
+license: cc-by-4.0
+datasets:
+- SHARE
+tags:
+- long-form-dialogue
+- dialogue-generation
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# Model Card for SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Model
 
+This model is a fine-tuned version of `meta-llama/Meta-Llama-3.1-8B-Instruct` based on the SHARE framework, designed to enhance long-term dialogue engagement by leveraging shared memories. It was developed using the new SHARE dataset, constructed from movie scripts, which provides rich explicit persona information, event summaries, and implicitly extractable shared memories between conversational participants. The associated EPISODE framework utilizes these shared experiences to facilitate more engaging and sustainable conversations.
 
 ## Model Details
 
 ### Model Description
+Shared memories between two individuals strengthen their bond and are crucial for facilitating their ongoing conversations. This model aims to make long-term dialogue more engaging by leveraging these shared memories. To this end, it uses the new long-term dialogue dataset named SHARE, constructed from movie scripts, which are a rich source of shared memories among various relationships. The underlying research introduces the EPISODE framework, a long-term dialogue framework based on SHARE that effectively manages shared memories during dialogue. Experiments demonstrate that shared memories make long-term dialogues more engaging and sustainable.
 
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+-   **Developed by:** The authors of the paper "SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script".
+-   **Funded by [optional]:** [More Information Needed]
+-   **Shared by [optional]:** [More Information Needed]
+-   **Model type:** LoRA adapter for Causal Language Model (`meta-llama/Meta-Llama-3.1-8B-Instruct`) designed for long-term dialogue.
+-   **Language(s) (NLP):** English
+-   **License:** [Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/)
+-   **Finetuned from model [optional]:** `meta-llama/Meta-Llama-3.1-8B-Instruct`
 
 ### Model Sources [optional]
 
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+-   **Repository:** [https://github.com/share-dialogue/SHARE](https://github.com/share-dialogue/SHARE)
+-   **Paper [optional]:** [SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682)
+-   **Demo [optional]:** [More Information Needed]
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
+This model is intended for research and development in long-term dialogue systems. It can be used to generate conversational responses that effectively utilize shared memories between participants, leading to more coherent, engaging, and contextually rich interactions over extended conversations.
 
 ### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
+The model can serve as a foundation for further research and applications in personalized dialogue, conversational AI agents with memory, and interactive storytelling where consistent character memory is crucial.
 
 ### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model is not intended for generating harmful, biased, or inappropriate content. Users should be aware that as a generative language model, it may produce outputs reflecting biases present in its training data or the base model. It should not be deployed in high-stakes applications without rigorous further testing, fine-tuning, and safety measures.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+The model's performance and outputs are influenced by the characteristics of its training data (movie scripts) and the underlying `Meta-Llama-3.1-8B-Instruct` base model. Potential biases, stereotypes, or limitations present in these sources may be reflected in the generated dialogues. The abstract does not detail specific ethical considerations or limitations beyond the technical scope.
 
 ### Recommendations
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Continuous monitoring and evaluation of model outputs are recommended, especially when integrating into sensitive applications.
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+Use the code below to get started with the model. This model is a PEFT (LoRA) adapter, meaning you need to load the base model (`meta-llama/Meta-Llama-3.1-8B-Instruct`) and then apply this adapter on top.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from peft import PeftModel, PeftConfig
+
+# Replace "your_repo_id/this_model" with the actual model ID on the Hub
+model_id = "your_repo_id/this_model" 
+
+# Load PEFT config to get base model name
+config = PeftConfig.from_pretrained(model_id)
+
+# Load the base model and tokenizer
+base_model = AutoModelForCausalLM.from_pretrained(
+    config.base_model_name_or_path,
+    torch_dtype=torch.bfloat16, # or torch.float16 or torch.float32 depending on your GPU/needs
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
+
+# Load the PEFT adapter on top of the base model
+model = PeftModel.from_pretrained(base_model, model_id)
+
+# For seamless inference, you might want to merge the adapter weights into the base model
+# This makes the model behave like a fully fine-tuned model and can improve performance on some hardware
+# model = model.merge_and_unload() 
+
+# Example for text generation (using Llama 3.1 chat template)
+messages = [
+    {"role": "user", "content": "Hello, do you remember our conversation about the movie 'Inception'?"},
+    {"role": "assistant", "content": "Yes, I do! We talked about its complex narrative structure and dream-within-a-dream concept. What specifically about it are you thinking about now?"},
+    {"role": "user", "content": "I was wondering, what was the name of the character who guided people through the dream layers?"}
+]
+
+# Apply chat template and tokenize
+input_ids = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+# Generate response
+outputs = model.generate(
+    input_ids,
+    max_new_tokens=256,
+    eos_token_id=tokenizer.eos_token_id,
+    do_sample=True,
+    temperature=0.7,
+    top_p=0.9,
+)
+
+generated_text = tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True)
+print(generated_text)
+```
 
 ## Training Details
 
 ### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+The model was fine-tuned on the new **SHARE dataset**, which is constructed from movie scripts. This dataset contains the summaries of persona information and events of two individuals, as explicitly revealed in their conversation, along with implicitly extractable shared memories.
 
 ### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+The training procedure leverages the **EPISODE** framework, a long-term dialogue framework based on the SHARE dataset, which utilizes shared experiences between individuals. The model was fine-tuned using Parameter-Efficient Fine-Tuning (PEFT), specifically LoRA, as indicated by the `adapter_config.json`. The objective was to enable the model to effectively manage shared memories during dialogue and produce more engaging and sustainable long-term conversations.
 
 #### Preprocessing [optional]
 
 [More Information Needed]
 
-
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+-   **Training regime:** [More Information Needed]
+    <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
 #### Speeds, Sizes, Times [optional]
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
 [More Information Needed]
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
 ### Testing Data, Factors & Metrics
+Experiments using the SHARE dataset were conducted to evaluate the model. The evaluation focused on demonstrating that shared memories between two individuals make long-term dialogues more engaging and sustainable, and that the EPISODE framework effectively manages these shared memories during dialogue.
 
 #### Testing Data
 
-<!-- This should link to a Dataset Card if possible. -->
-
 [More Information Needed]
 
 #### Factors
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
 [More Information Needed]
 
 #### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+Specific metrics for evaluation are not detailed in the abstract, but typical dialogue evaluation metrics (e.g., coherence, fluency, engagement, consistency) would likely be used.
 
 ### Results
-
-[More Information Needed]
+The research demonstrates that shared memories between two individuals lead to more engaging and sustainable long-term dialogues, and that the EPISODE framework effectively manages shared memories throughout conversations.
 
 #### Summary
 
-
-
 ## Model Examination [optional]
 
-<!-- Relevant interpretability work for the model goes here -->
-
 [More Information Needed]
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
 Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+-   **Hardware Type:** [More Information Needed]
+-   **Hours used:** [More Information Needed]
+-   **Cloud Provider:** [More Information Needed]
+-   **Compute Region:** [More Information Needed]
+-   **Carbon Emitted:** [More Information Needed]
 
 ## Technical Specifications [optional]
 
 ### Model Architecture and Objective
-
-[More Information Needed]
+The model is a PEFT (LoRA) adapted version of the `meta-llama/Meta-Llama-3.1-8B-Instruct` large language model. Its primary objective is to facilitate long-term, open-domain dialogue by explicitly leveraging shared memories and contextual information, aiming for more engaging and coherent conversations over time.
 
 ### Compute Infrastructure
 
@@ -166,24 +182,25 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 #### Software
 
-[More Information Needed]
+-   PEFT 0.12.0
+-   Transformers (compatible with Llama 3.1)
 
 ## Citation [optional]
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
 **BibTeX:**
 
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
+```bibtex
+@article{share2024longterm,
+  title={SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script},
+  author={Authors Name(s) Here}, # Please replace with actual author list from the paper
+  journal={arXiv preprint arXiv:2410.20682},
+  year={2024},
+  url={https://huggingface.co/papers/2410.20682}
+}
+```
 
 ## Glossary [optional]
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
 [More Information Needed]
 
 ## More Information [optional]
@@ -196,7 +213,4 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 ## Model Card Contact
 
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.12.0
+[More Information Needed]