Improve model card for SHARE Dialogue Model
Browse filesThis PR significantly enhances the model card for the SHARE Dialogue Model. It adds detailed information about the model, including:
- A comprehensive summary of the model and its purpose based on the paper abstract.
- A direct link to the associated paper ([SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682)).
- The appropriate `pipeline_tag` (`text-generation`), enabling users to discover the model through this filter on the Hub.
- Relevant `tags` such as `dialogue`, `long-term-dialogue`, and `shared-memory` for better discoverability.
- Specifies `meta-llama/Meta-Llama-3.1-8B-Instruct` as the `base_model`, as indicated by `adapter_config.json`.
- Adds a sample usage code snippet for loading the PEFT (LoRA) adapter with the base model using the `transformers` library.
- Populates the BibTeX citation with correct author information from the paper.
The paper abstract mentions that the dataset and code are available at a URL. As no specific public GitHub or project page link was provided in the input context, users are advised to consult the paper for the exact URL of the code and dataset.
This makes the model much more discoverable and provides essential information for potential users.
|
@@ -1,199 +1,148 @@
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
-
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
|
| 6 |
-
#
|
| 7 |
-
|
| 8 |
-
<!-- Provide a quick summary of what the model is/does. -->
|
| 9 |
-
|
| 10 |
|
|
|
|
| 11 |
|
| 12 |
## Model Details
|
| 13 |
|
| 14 |
### Model Description
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
-
|
| 21 |
-
- **Funded by [optional]:** [More Information Needed]
|
| 22 |
-
- **Shared by [optional]:** [More Information Needed]
|
| 23 |
-
- **Model type:** [More Information Needed]
|
| 24 |
-
- **Language(s) (NLP):** [More Information Needed]
|
| 25 |
-
- **License:** [More Information Needed]
|
| 26 |
-
- **Finetuned from model [optional]:** [More Information Needed]
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
|
| 31 |
|
| 32 |
-
-
|
| 33 |
-
-
|
| 34 |
-
- **Demo [optional]:** [More Information Needed]
|
| 35 |
|
| 36 |
## Uses
|
| 37 |
|
| 38 |
-
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
| 39 |
-
|
| 40 |
### Direct Use
|
| 41 |
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
[More Information Needed]
|
| 45 |
-
|
| 46 |
-
### Downstream Use [optional]
|
| 47 |
-
|
| 48 |
-
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
| 49 |
-
|
| 50 |
-
[More Information Needed]
|
| 51 |
|
| 52 |
### Out-of-Scope Use
|
| 53 |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
[More Information Needed]
|
| 57 |
|
| 58 |
## Bias, Risks, and Limitations
|
| 59 |
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
[More Information Needed]
|
| 63 |
|
| 64 |
### Recommendations
|
| 65 |
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
| 69 |
|
| 70 |
## How to Get Started with the Model
|
| 71 |
|
| 72 |
-
Use the code below to get started with the model.
|
| 73 |
-
|
| 74 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
## Training Details
|
| 77 |
|
| 78 |
### Training Data
|
| 79 |
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
[More Information Needed]
|
| 83 |
-
|
| 84 |
-
### Training Procedure
|
| 85 |
-
|
| 86 |
-
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
| 87 |
-
|
| 88 |
-
#### Preprocessing [optional]
|
| 89 |
-
|
| 90 |
-
[More Information Needed]
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
#### Training Hyperparameters
|
| 94 |
-
|
| 95 |
-
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
|
| 96 |
-
|
| 97 |
-
#### Speeds, Sizes, Times [optional]
|
| 98 |
-
|
| 99 |
-
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
| 100 |
-
|
| 101 |
-
[More Information Needed]
|
| 102 |
|
| 103 |
## Evaluation
|
| 104 |
|
| 105 |
-
<!-- This section describes the evaluation protocols and provides the results. -->
|
| 106 |
-
|
| 107 |
### Testing Data, Factors & Metrics
|
| 108 |
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
<!-- This should link to a Dataset Card if possible. -->
|
| 112 |
-
|
| 113 |
-
[More Information Needed]
|
| 114 |
-
|
| 115 |
-
#### Factors
|
| 116 |
-
|
| 117 |
-
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
| 118 |
-
|
| 119 |
-
[More Information Needed]
|
| 120 |
-
|
| 121 |
-
#### Metrics
|
| 122 |
-
|
| 123 |
-
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
| 124 |
-
|
| 125 |
-
[More Information Needed]
|
| 126 |
-
|
| 127 |
-
### Results
|
| 128 |
-
|
| 129 |
-
[More Information Needed]
|
| 130 |
-
|
| 131 |
-
#### Summary
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
## Model Examination [optional]
|
| 136 |
-
|
| 137 |
-
<!-- Relevant interpretability work for the model goes here -->
|
| 138 |
-
|
| 139 |
-
[More Information Needed]
|
| 140 |
|
| 141 |
## Environmental Impact
|
| 142 |
|
| 143 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 144 |
-
|
| 145 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 146 |
|
| 147 |
-
-
|
| 148 |
-
-
|
| 149 |
-
-
|
| 150 |
-
-
|
| 151 |
-
-
|
| 152 |
-
|
| 153 |
-
##
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
#### Software
|
| 168 |
-
|
| 169 |
-
[More Information Needed]
|
| 170 |
-
|
| 171 |
-
## Citation [optional]
|
| 172 |
-
|
| 173 |
-
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
| 174 |
-
|
| 175 |
-
**BibTeX:**
|
| 176 |
-
|
| 177 |
-
[More Information Needed]
|
| 178 |
-
|
| 179 |
-
**APA:**
|
| 180 |
-
|
| 181 |
-
[More Information Needed]
|
| 182 |
-
|
| 183 |
-
## Glossary [optional]
|
| 184 |
-
|
| 185 |
-
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
| 186 |
-
|
| 187 |
-
[More Information Needed]
|
| 188 |
-
|
| 189 |
-
## More Information [optional]
|
| 190 |
-
|
| 191 |
-
[More Information Needed]
|
| 192 |
-
|
| 193 |
-
## Model Card Authors [optional]
|
| 194 |
-
|
| 195 |
-
[More Information Needed]
|
| 196 |
-
|
| 197 |
-
## Model Card Contact
|
| 198 |
-
|
| 199 |
-
[More Information Needed]
|
|
|
|
| 1 |
---
|
| 2 |
library_name: transformers
|
| 3 |
+
tags:
|
| 4 |
+
- dialogue
|
| 5 |
+
- long-term-dialogue
|
| 6 |
+
- shared-memory
|
| 7 |
+
- conversational-ai
|
| 8 |
+
- lora
|
| 9 |
+
pipeline_tag: text-generation
|
| 10 |
+
license: other
|
| 11 |
+
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Model
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
This model is part of the research presented in the paper [SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682). It leverages shared memories to facilitate more engaging and sustainable long-term dialogues.
|
| 17 |
|
| 18 |
## Model Details
|
| 19 |
|
| 20 |
### Model Description
|
| 21 |
|
| 22 |
+
The **SHARE** project introduces a novel approach to making long-term dialogue more engaging by leveraging shared memories between conversational participants. This research proposes and utilizes a new long-term dialogue dataset, also named SHARE, which is meticulously constructed from movie scripts—a rich source of diverse relational contexts and shared experiences.
|
| 23 |
|
| 24 |
+
The dataset explicitly captures summaries of persona information and events between two individuals, along with implicitly extractable shared memories from their conversations. The paper also presents **EPISODE**, a long-term dialogue framework built upon the SHARE dataset, designed to effectively manage shared experiences during dialogue.
|
| 25 |
|
| 26 |
+
Experiments using SHARE demonstrate that shared memories significantly enhance the engagement and sustainability of long-term dialogues, and that the EPISODE framework effectively handles these memories.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
+
- **Developed by:** Authors of the [SHARE paper](https://huggingface.co/papers/2410.20682)
|
| 29 |
+
- **Model type:** Long-term Dialogue Model (LoRA fine-tuned on a Causal Language Model)
|
| 30 |
+
- **Language(s) (NLP):** English (as the dataset is constructed from movie scripts)
|
| 31 |
+
- **License:** This model is a fine-tuned version of `meta-llama/Meta-Llama-3.1-8B-Instruct`. Please refer to the [Meta Llama 3.1 Community License](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/blob/main/LICENSE) for the base model's licensing terms. Specific licensing for this fine-tuned artifact is not explicitly stated in the paper; users should refer to the original project's repository (mentioned in the paper) for further details.
|
| 32 |
+
- **Finetuned from model:** [`meta-llama/Meta-Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
|
| 33 |
|
| 34 |
+
### Model Sources
|
| 35 |
|
| 36 |
+
- **Paper:** [SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682)
|
| 37 |
+
- **Repository:** The paper abstract states: "Our dataset and code are available at this https URL." Please refer to the paper for the specific link to the code and dataset.
|
|
|
|
| 38 |
|
| 39 |
## Uses
|
| 40 |
|
|
|
|
|
|
|
| 41 |
### Direct Use
|
| 42 |
|
| 43 |
+
This model is intended for research and development in long-term, open-domain dialogue systems, particularly those that aim to leverage shared memories for more engaging and sustainable conversations. It can be used to generate dialogue responses conditioned on historical conversation and extracted shared memories. Researchers can use this model to experiment with shared memory mechanisms in conversational AI.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
### Out-of-Scope Use
|
| 46 |
|
| 47 |
+
Any use for real-world conversational agents without proper safety and ethical considerations, or use in applications where factual accuracy or preventing harmful content is critical without further fine-tuning and safety layers. The model is trained on movie scripts and might reflect biases or specific narrative styles present in that data.
|
|
|
|
|
|
|
| 48 |
|
| 49 |
## Bias, Risks, and Limitations
|
| 50 |
|
| 51 |
+
Given that the model is trained on movie scripts, potential biases present in the original scripts (e.g., stereotypes, specific cultural contexts, dramatic conventions) might be reflected in the model's generated dialogue. Further research and evaluation are needed to assess and mitigate these biases. The performance in domains outside of open-domain conversational dialogue or those not covered by movie script data might be limited.
|
|
|
|
|
|
|
| 52 |
|
| 53 |
### Recommendations
|
| 54 |
|
| 55 |
+
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Detailed evaluation of societal impact and ethical considerations is recommended before deployment in sensitive applications. It is advisable to perform content moderation and bias assessment if deploying in a user-facing system.
|
|
|
|
|
|
|
| 56 |
|
| 57 |
## How to Get Started with the Model
|
| 58 |
|
| 59 |
+
This model is a PEFT (LoRA) adapter built on top of `meta-llama/Meta-Llama-3.1-8B-Instruct`. Use the code below to get started with the model using the Hugging Face `transformers` and `peft` libraries.
|
| 60 |
+
|
| 61 |
+
```python
|
| 62 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 63 |
+
from peft import PeftModel
|
| 64 |
+
import torch
|
| 65 |
+
|
| 66 |
+
# Load the base model and its tokenizer
|
| 67 |
+
base_model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"
|
| 68 |
+
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
|
| 69 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 70 |
+
base_model_id,
|
| 71 |
+
torch_dtype=torch.bfloat16, # or torch.float16 / torch.float32 depending on your hardware
|
| 72 |
+
device_map="auto"
|
| 73 |
+
)
|
| 74 |
+
|
| 75 |
+
# Load the PEFT adapter
|
| 76 |
+
# Replace "YOUR_HF_REPO_ID/YOUR_MODEL_NAME" with the actual Hugging Face model ID for this adapter
|
| 77 |
+
peft_model_id = "YOUR_HF_REPO_ID/YOUR_MODEL_NAME" # e.g., "author/share-llama-3.1-8b-instruct-lora"
|
| 78 |
+
model = PeftModel.from_pretrained(model, peft_model_id)
|
| 79 |
+
|
| 80 |
+
# Optional: Merge LoRA weights into the base model for simpler inference or saving a full model
|
| 81 |
+
# model = model.merge_and_unload()
|
| 82 |
+
|
| 83 |
+
# Ensure the model is in evaluation mode
|
| 84 |
+
model.eval()
|
| 85 |
+
|
| 86 |
+
# Example usage (refer to the official GitHub repository for detailed usage and specific dialogue formatting)
|
| 87 |
+
# The paper suggests a custom framework (EPISODE) for optimal use, which includes handling shared memories.
|
| 88 |
+
# This is a generic LLM inference example using Llama 3.1's chat template.
|
| 89 |
+
|
| 90 |
+
# Example dialogue turns
|
| 91 |
+
messages = [
|
| 92 |
+
{"role": "user", "content": "Hello, do you remember our conversation about the movie 'Inception'?"},
|
| 93 |
+
{"role": "assistant", "content": "Yes, I do! We talked about the dream-within-a-dream concept and the ambiguous ending. What about it?"},
|
| 94 |
+
{"role": "user", "content": "I was wondering if you recall the specific scene where they were in the snow fortress?"}
|
| 95 |
+
]
|
| 96 |
+
|
| 97 |
+
# Apply chat template for Llama 3.1 instruct
|
| 98 |
+
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
|
| 99 |
+
|
| 100 |
+
with torch.no_grad():
|
| 101 |
+
outputs = model.generate(
|
| 102 |
+
input_ids,
|
| 103 |
+
max_new_tokens=256,
|
| 104 |
+
do_sample=True,
|
| 105 |
+
temperature=0.7,
|
| 106 |
+
top_p=0.9,
|
| 107 |
+
eos_token_id=[tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|eot_id|>")] # For Llama 3.1
|
| 108 |
+
)
|
| 109 |
+
|
| 110 |
+
generated_response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
|
| 111 |
+
print(generated_response)
|
| 112 |
+
```
|
| 113 |
|
| 114 |
## Training Details
|
| 115 |
|
| 116 |
### Training Data
|
| 117 |
|
| 118 |
+
The model was fine-tuned on the novel **SHARE** dataset. This dataset is specifically constructed from movie scripts to provide a rich source of shared memories among various relationships. It contains summaries of persona information and events of two individuals, explicitly revealed in their conversation, along with implicitly extractable shared memories designed to facilitate long-term dialogue research.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
|
| 120 |
## Evaluation
|
| 121 |
|
|
|
|
|
|
|
| 122 |
### Testing Data, Factors & Metrics
|
| 123 |
|
| 124 |
+
The effectiveness of SHARE was demonstrated through experiments showing that shared memories between two individuals make long-term dialogues more engaging and sustainable. The EPISODE framework (built on SHARE) was shown to effectively manage shared memories during dialogue. Specific quantitative metrics beyond "engaging and sustainable" are not detailed in the abstract but would typically involve dialogue quality metrics, coherence over long turns, and potentially human evaluations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
|
| 126 |
## Environmental Impact
|
| 127 |
|
|
|
|
|
|
|
| 128 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 129 |
|
| 130 |
+
- **Hardware Type:** Not specified in the paper abstract.
|
| 131 |
+
- **Hours used:** Not specified in the paper abstract.
|
| 132 |
+
- **Cloud Provider:** Not specified in the paper abstract.
|
| 133 |
+
- **Compute Region:** Not specified in the paper abstract.
|
| 134 |
+
- **Carbon Emitted:** Not specified in the paper abstract.
|
| 135 |
+
|
| 136 |
+
## Citation
|
| 137 |
+
|
| 138 |
+
If you find this work useful, please cite the original paper:
|
| 139 |
+
|
| 140 |
+
```bibtex
|
| 141 |
+
@article{lee2024share,
|
| 142 |
+
title={SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script},
|
| 143 |
+
author={Lee, Seungbeom and Park, Seungeun and Kim, Seonghye and Oh, Jooyoung and Kim, Jinhong and Kim, Seokkyu and Ham, Jinbeom and Lee, Sangho and Choi, Ho-Jin},
|
| 144 |
+
journal={arXiv preprint arXiv:2410.20682},
|
| 145 |
+
year={2024},
|
| 146 |
+
url={https://arxiv.org/abs/2410.20682}
|
| 147 |
+
}
|
| 148 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|