| | --- |
| | datasets: |
| | - cognitivecomputations/dolphin |
| | language: |
| | - en |
| | --- |
| | |
| | This model draws inspiration from [SOLAR](https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0), but introduces a novel approach to increasing the model's depth without the traditional method of duplicating layers. |
| | By rearranging the order of layers during inference, it maintains the advantages of depth upscaling while preserving the original parameter count. |
| | Furthermore, it undergoes additional fine-tuning using the Dolphin dataset. The foundational architecture for this experiment is based on [Dolphin](https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser). |
| |
|
| | **Use** |
| |
|
| | ```python |
| | # pip install transformers |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | |
| | model_id = "adalbertojunior/DUSMistral" |
| | tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
| | model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True) |
| | |
| | # Format message with the CHATML chat template |
| | messages = [{"role": "user", "content": "Hello, how are you?"}] |
| | input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt") |
| | |
| | |
| | gen_tokens = model.generate( |
| | input_ids, |
| | max_new_tokens=100, |
| | do_sample=True, |
| | temperature=0.3, |
| | ) |
| | |
| | gen_text = tokenizer.decode(gen_tokens[0]) |
| | print(gen_text) |
| | ``` |
