| --- |
| license: cc-by-nc-4.0 |
| tags: |
| - moe |
| - frankenmoe |
| - merge |
| - mergekit |
| - lazymergekit |
| base_model: |
| - mlabonne/NeuralDaredevil-7B |
| - BioMistral/BioMistral-7B |
| - mistralai/Mathstral-7B-v0.1 |
| - FPHam/Writing_Partner_Mistral_7B |
| library_name: transformers |
| pipeline_tag: text-generation |
| --- |
| |
| # EduMixtral-4x7B |
|
|
| <img src="https://cdn-uploads.huggingface.co/production/uploads/65ba68a15d2ef0a4b2c892b4/1hvgYltQRmbkzHMSXvGYh.jpeg" width=400> |
|
|
| EduMixtral-4x7B is an experimental model that combines different educational focused language models intended for downstream human/ai student/teacher application research. |
| Intended to cover: general knowledge, medical field, math, and writing assistance. |
|
|
| ## 🤏 Models Merged |
|
|
| EduMixtral-4x7B is a Mixture of Experts (MoE) made with the following models using [Mergekit](https://github.com/arcee-ai/mergekit): |
| * [mlabonne/NeuralDaredevil-7B](https://huggingface.co/mlabonne/NeuralDaredevil-7B) <- Base Model |
| * [BioMistral/BioMistral-7B](https://huggingface.co/BioMistral/BioMistral-7B) |
| * [mistralai/Mathstral-7B-v0.1](https://huggingface.co/mistralai/Mathstral-7B-v0.1) |
| * [FPHam/Writing_Partner_Mistral_7B](https://huggingface.co/FPHam/Writing_Partner_Mistral_7B) |
|
|
| ## 🧩 Configuration |
|
|
| ```yaml |
| base_model: mlabonne/NeuralDaredevil-7B |
| gate_mode: hidden |
| experts: |
| - source_model: mlabonne/NeuralDaredevil-7B |
| positive_prompts: |
| - "hello" |
| - "help" |
| - "question" |
| - "explain" |
| - "information" |
| - source_model: BioMistral/BioMistral-7B |
| positive_prompts: |
| - "medical" |
| - "health" |
| - "biomedical" |
| - "clinical" |
| - "anatomy" |
| - source_model: mistralai/Mathstral-7B-v0.1 |
| positive_prompts: |
| - "math" |
| - "calculation" |
| - "equation" |
| - "geometry" |
| - "algebra" |
| - source_model: FPHam/Writing_Partner_Mistral_7B |
| positive_prompts: |
| - "writing" |
| - "creative process" |
| - "story structure" |
| - "character development" |
| - "plot" |
| ``` |
|
|
| ## 💻 Usage |
|
|
| It is reccomended to load in 8bit or 4bit quantization |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig |
| |
| # Load the tokenizer and model |
| tokenizer = AutoTokenizer.from_pretrained("AdamLucek/EduMixtral-4x7B") |
| model = AutoModelForCausalLM.from_pretrained( |
| "AdamLucek/EduMixtral-4x7B", |
| device_map="cuda", |
| quantization_config=BitsAndBytesConfig(load_in_8bit=True) |
| ) |
| |
| # Prepare the input text |
| input_text = "Math problem: Xiaoli reads a 240-page story book. She reads (1/8) of the whole book on the first day and (1/5) of the whole book on the second day. How many pages did she read in total in two days?" |
| input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") |
| |
| # Generate the output with specified parameters |
| outputs = model.generate( |
| **input_ids, |
| max_new_tokens=256, |
| num_return_sequences=1 |
| ) |
| |
| # Decode and print the generated text |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
|
|
| **Output:** |
|
|
| >Solution: |
| >To find the total number of pages Xiaoli read in two days, we need to add the number of pages she read on the first day and the second day. |
| >On the first day, Xiaoli read 1/8 of the book. Since the book has 240 pages, the number of pages she read on the first day is: |
| >\[ \frac{1}{8} \times 240 = 30 \text{ pages} \] |
| >On the second day, Xiaoli read 1/5 of the book. The number of pages she read on the second day is: |
| >\[ \frac{1}{5} \times 240 = 48 \text{ pages} \] |
| >To find the total number of pages she read in two days, we add the pages she read on the first day and the second day: |
| >\[ 30 \text{ pages} + 48 \text{ pages} = 78 \text{ pages} \] |
| >Therefore, Xiaoli read a total of 78 pages in two days. |
| >Final answer: Xiaoli read 78 pages in total |
