nielsr HF Staff commited on
Commit
a7efcb8
·
verified ·
1 Parent(s): 72fb288

Improve model card for SHARE Dialogue Model

Browse files

This PR significantly enhances the model card for the SHARE Dialogue Model. It adds detailed information about the model, including:

- A comprehensive summary of the model and its purpose based on the paper abstract.
- A direct link to the associated paper ([SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682)).
- The appropriate `pipeline_tag` (`text-generation`), enabling users to discover the model through this filter on the Hub.
- Relevant `tags` such as `dialogue`, `long-term-dialogue`, and `shared-memory` for better discoverability.
- Specifies `meta-llama/Meta-Llama-3.1-8B-Instruct` as the `base_model`, as indicated by `adapter_config.json`.
- Adds a sample usage code snippet for loading the PEFT (LoRA) adapter with the base model using the `transformers` library.
- Populates the BibTeX citation with correct author information from the paper.

The paper abstract mentions that the dataset and code are available at a URL. As no specific public GitHub or project page link was provided in the input context, users are advised to consult the paper for the exact URL of the code and dataset.

This makes the model much more discoverable and provides essential information for potential users.

Files changed (1) hide show
  1. README.md +101 -152
README.md CHANGED
@@ -1,199 +1,148 @@
1
  ---
2
  library_name: transformers
3
- tags: []
 
 
 
 
 
 
 
 
4
  ---
5
 
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
 
 
11
 
12
  ## Model Details
13
 
14
  ### Model Description
15
 
16
- <!-- Provide a longer summary of what this model is. -->
17
 
18
- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
19
 
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
 
28
- ### Model Sources [optional]
 
 
 
 
29
 
30
- <!-- Provide the basic links for the model. -->
31
 
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
 
36
  ## Uses
37
 
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
  ### Direct Use
41
 
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
 
52
  ### Out-of-Scope Use
53
 
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
 
58
  ## Bias, Risks, and Limitations
59
 
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
 
64
  ### Recommendations
65
 
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
 
70
  ## How to Get Started with the Model
71
 
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75
 
76
  ## Training Details
77
 
78
  ### Training Data
79
 
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
 
103
  ## Evaluation
104
 
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
  ### Testing Data, Factors & Metrics
108
 
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
 
141
  ## Environmental Impact
142
 
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
  Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
 
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
 
1
  ---
2
  library_name: transformers
3
+ tags:
4
+ - dialogue
5
+ - long-term-dialogue
6
+ - shared-memory
7
+ - conversational-ai
8
+ - lora
9
+ pipeline_tag: text-generation
10
+ license: other
11
+ base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
12
  ---
13
 
14
+ # SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Model
 
 
 
15
 
16
+ This model is part of the research presented in the paper [SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682). It leverages shared memories to facilitate more engaging and sustainable long-term dialogues.
17
 
18
  ## Model Details
19
 
20
  ### Model Description
21
 
22
+ The **SHARE** project introduces a novel approach to making long-term dialogue more engaging by leveraging shared memories between conversational participants. This research proposes and utilizes a new long-term dialogue dataset, also named SHARE, which is meticulously constructed from movie scripts—a rich source of diverse relational contexts and shared experiences.
23
 
24
+ The dataset explicitly captures summaries of persona information and events between two individuals, along with implicitly extractable shared memories from their conversations. The paper also presents **EPISODE**, a long-term dialogue framework built upon the SHARE dataset, designed to effectively manage shared experiences during dialogue.
25
 
26
+ Experiments using SHARE demonstrate that shared memories significantly enhance the engagement and sustainability of long-term dialogues, and that the EPISODE framework effectively handles these memories.
 
 
 
 
 
 
27
 
28
+ - **Developed by:** Authors of the [SHARE paper](https://huggingface.co/papers/2410.20682)
29
+ - **Model type:** Long-term Dialogue Model (LoRA fine-tuned on a Causal Language Model)
30
+ - **Language(s) (NLP):** English (as the dataset is constructed from movie scripts)
31
+ - **License:** This model is a fine-tuned version of `meta-llama/Meta-Llama-3.1-8B-Instruct`. Please refer to the [Meta Llama 3.1 Community License](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct/blob/main/LICENSE) for the base model's licensing terms. Specific licensing for this fine-tuned artifact is not explicitly stated in the paper; users should refer to the original project's repository (mentioned in the paper) for further details.
32
+ - **Finetuned from model:** [`meta-llama/Meta-Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
33
 
34
+ ### Model Sources
35
 
36
+ - **Paper:** [SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script](https://huggingface.co/papers/2410.20682)
37
+ - **Repository:** The paper abstract states: "Our dataset and code are available at this https URL." Please refer to the paper for the specific link to the code and dataset.
 
38
 
39
  ## Uses
40
 
 
 
41
  ### Direct Use
42
 
43
+ This model is intended for research and development in long-term, open-domain dialogue systems, particularly those that aim to leverage shared memories for more engaging and sustainable conversations. It can be used to generate dialogue responses conditioned on historical conversation and extracted shared memories. Researchers can use this model to experiment with shared memory mechanisms in conversational AI.
 
 
 
 
 
 
 
 
44
 
45
  ### Out-of-Scope Use
46
 
47
+ Any use for real-world conversational agents without proper safety and ethical considerations, or use in applications where factual accuracy or preventing harmful content is critical without further fine-tuning and safety layers. The model is trained on movie scripts and might reflect biases or specific narrative styles present in that data.
 
 
48
 
49
  ## Bias, Risks, and Limitations
50
 
51
+ Given that the model is trained on movie scripts, potential biases present in the original scripts (e.g., stereotypes, specific cultural contexts, dramatic conventions) might be reflected in the model's generated dialogue. Further research and evaluation are needed to assess and mitigate these biases. The performance in domains outside of open-domain conversational dialogue or those not covered by movie script data might be limited.
 
 
52
 
53
  ### Recommendations
54
 
55
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Detailed evaluation of societal impact and ethical considerations is recommended before deployment in sensitive applications. It is advisable to perform content moderation and bias assessment if deploying in a user-facing system.
 
 
56
 
57
  ## How to Get Started with the Model
58
 
59
+ This model is a PEFT (LoRA) adapter built on top of `meta-llama/Meta-Llama-3.1-8B-Instruct`. Use the code below to get started with the model using the Hugging Face `transformers` and `peft` libraries.
60
+
61
+ ```python
62
+ from transformers import AutoModelForCausalLM, AutoTokenizer
63
+ from peft import PeftModel
64
+ import torch
65
+
66
+ # Load the base model and its tokenizer
67
+ base_model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"
68
+ tokenizer = AutoTokenizer.from_pretrained(base_model_id)
69
+ model = AutoModelForCausalLM.from_pretrained(
70
+ base_model_id,
71
+ torch_dtype=torch.bfloat16, # or torch.float16 / torch.float32 depending on your hardware
72
+ device_map="auto"
73
+ )
74
+
75
+ # Load the PEFT adapter
76
+ # Replace "YOUR_HF_REPO_ID/YOUR_MODEL_NAME" with the actual Hugging Face model ID for this adapter
77
+ peft_model_id = "YOUR_HF_REPO_ID/YOUR_MODEL_NAME" # e.g., "author/share-llama-3.1-8b-instruct-lora"
78
+ model = PeftModel.from_pretrained(model, peft_model_id)
79
+
80
+ # Optional: Merge LoRA weights into the base model for simpler inference or saving a full model
81
+ # model = model.merge_and_unload()
82
+
83
+ # Ensure the model is in evaluation mode
84
+ model.eval()
85
+
86
+ # Example usage (refer to the official GitHub repository for detailed usage and specific dialogue formatting)
87
+ # The paper suggests a custom framework (EPISODE) for optimal use, which includes handling shared memories.
88
+ # This is a generic LLM inference example using Llama 3.1's chat template.
89
+
90
+ # Example dialogue turns
91
+ messages = [
92
+ {"role": "user", "content": "Hello, do you remember our conversation about the movie 'Inception'?"},
93
+ {"role": "assistant", "content": "Yes, I do! We talked about the dream-within-a-dream concept and the ambiguous ending. What about it?"},
94
+ {"role": "user", "content": "I was wondering if you recall the specific scene where they were in the snow fortress?"}
95
+ ]
96
+
97
+ # Apply chat template for Llama 3.1 instruct
98
+ input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
99
+
100
+ with torch.no_grad():
101
+ outputs = model.generate(
102
+ input_ids,
103
+ max_new_tokens=256,
104
+ do_sample=True,
105
+ temperature=0.7,
106
+ top_p=0.9,
107
+ eos_token_id=[tokenizer.eos_token_id, tokenizer.convert_tokens_to_ids("<|eot_id|>")] # For Llama 3.1
108
+ )
109
+
110
+ generated_response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
111
+ print(generated_response)
112
+ ```
113
 
114
  ## Training Details
115
 
116
  ### Training Data
117
 
118
+ The model was fine-tuned on the novel **SHARE** dataset. This dataset is specifically constructed from movie scripts to provide a rich source of shared memories among various relationships. It contains summaries of persona information and events of two individuals, explicitly revealed in their conversation, along with implicitly extractable shared memories designed to facilitate long-term dialogue research.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
 
120
  ## Evaluation
121
 
 
 
122
  ### Testing Data, Factors & Metrics
123
 
124
+ The effectiveness of SHARE was demonstrated through experiments showing that shared memories between two individuals make long-term dialogues more engaging and sustainable. The EPISODE framework (built on SHARE) was shown to effectively manage shared memories during dialogue. Specific quantitative metrics beyond "engaging and sustainable" are not detailed in the abstract but would typically involve dialogue quality metrics, coherence over long turns, and potentially human evaluations.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
125
 
126
  ## Environmental Impact
127
 
 
 
128
  Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
129
 
130
+ - **Hardware Type:** Not specified in the paper abstract.
131
+ - **Hours used:** Not specified in the paper abstract.
132
+ - **Cloud Provider:** Not specified in the paper abstract.
133
+ - **Compute Region:** Not specified in the paper abstract.
134
+ - **Carbon Emitted:** Not specified in the paper abstract.
135
+
136
+ ## Citation
137
+
138
+ If you find this work useful, please cite the original paper:
139
+
140
+ ```bibtex
141
+ @article{lee2024share,
142
+ title={SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script},
143
+ author={Lee, Seungbeom and Park, Seungeun and Kim, Seonghye and Oh, Jooyoung and Kim, Jinhong and Kim, Seokkyu and Ham, Jinbeom and Lee, Sangho and Choi, Ho-Jin},
144
+ journal={arXiv preprint arXiv:2410.20682},
145
+ year={2024},
146
+ url={https://arxiv.org/abs/2410.20682}
147
+ }
148
+ ```