Guibibo/Mistral-7B-v0.3-FlashCards

Model Card for Model ID

This model creates Anki Flashcards from french texts.

Model Details

Model Description

This model takes as input chunks of texts and outputs flashcards in the format'Q: A:'. It was trained on a personal dataset of french cards.

• Developed by: Guillaume Bertho
• Language(s) (NLP): French
• Finetuned from model Mistral-7B-Instruct-v0.3

Model Sources [optional]

• Repository: https://github.com/GUIBOoO/ankify-local

Uses

Give the model chunks of french text (from 100 to 2000 tokens). This model doesn't perform well on mathematical expressions/

Bias, Risks, and Limitations

This model can sometimes generate non-self-contained cards.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

MODEL_NAME = "Guibibo/Mistral-7B-v0.3-FlashCards"

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_NAME,
    torch_dtype=torch.float16,
    device_map="auto"  # automatically uses GPU if available
)

prompt = """Victor Hugo, né le 7 ventôse an X (26 février 1802) à Besançon ..."""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,   # adjust length
        do_sample=True,
        temperature=0.7,
        top_p=0.9
    )

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("=== Generated Text ===")
print(generated_text)

Training Details

Training Data

Personal data. I will maybe upload the dataset later.

Training Hyperparameters

Training settings : Warmup ratio: 0.03 Gradient accumulation steps: 4 Learning rate: 1e-4 Number of epochs: 1 Learning rate scheduler: cosine Weight decay: 0.01 Mixed precision training enabled (fp16)

LoRA configuration: Rank (r): 64 LoRA alpha: 64 LoRA dropout: 0.05 Target modules: ["q_proj", "v_proj"] Task type: CAUSAL_LM

Tokenizer settings: Pad sequences to end-of-sequence (pad_to_eos: true)

• PEFT 0.18.0