Aiden07/Mistral-7B-Instruct-dota2

Model Card for Model ID

A model aiming to be able to provide coaching for Dota 2 gamers by answering user's questions in a chat format. Improvement and enhancement are still needed.

Model Details

Model Description

This model is trained on mistralai/Mistral-7B-Instruct-v0.2 with instruction-answer dataset about Dota 2 gameplay mechanics.

• Model type: Transformer-based language model
• Language(s) (NLP): English
• License: MIT
• Finetuned from model [optional]: mistralai/Mistral-7B-Instruct-v0.2

Uses

To give summary of general gameplay mechanics and provide suggestions for gameplay strategy and skill improvement.

Direct Use

Provide summary of Dota 2 gameplay in interest.

Downstream Use [optional]

To give suggestions to Dota 2 players on gameplay strategy.

Out-of-Scope Use

Gameplay sabotage and abusive chat should not be allowed.

Bias, Risks, and Limitations

This model currently is only capable of giving summary of general gameplay mechanics including hero attributes, abilities, talents, maps, runes, building, creeps and items. The results may not be consistent and accurate. Fact check is still needed. It is currently not capable of providing strategy suggestion on how to improve player's rank in competitive play.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model mentioned above. More improvement and enhancement are still needed.

How to Get Started with the Model

Code example to get started with the model:

import transformers
# Format prompt
message = [
    {"role": "user", "content": "Which Strength Hero would you recommend for a Dota 2 beginner?"}
]
tokenizer = AutoTokenizer.from_pretrained("Aiden07/Mistral-7B-Instruct-dota2")
prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model=new_model,
    tokenizer=tokenizer
)

# Generate text
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    num_return_sequences=1,
    max_length=200,
)
print(sequences[0]['generated_text'])

Training Details

Training Data

Dataset used: Aiden07/dota2_instruct_prompt

Training Procedure

Preprocessing

Preprocessing to comply to instruct format:

def create_text_row(instruction, output):
    text_row = f"""<s>[INST] {instruction} [/INST] \\n {output} </s>"""
    return text_row

Training Hyperparameters

• Training regime:
  • output_dir="./results",
  • num_train_epochs=1,
  • per_device_train_batch_size=4,
  • gradient_accumulation_steps=1,
  • optim="paged_adamw_32bit",
  • save_steps=25,
  • logging_steps=25,
  • learning_rate=2e-4,
  • weight_decay=0.001,
  • fp16=False,
  • bf16=False,
  • max_grad_norm=0.3,
  • max_steps=-1,
  • warmup_ratio=0.03,
  • group_by_length=True,
  • lr_scheduler_type="constant"