README.md

---
library_name: peft
---

# Model Card for Mistral-sci-phi

This model is a fine-tuned version of the Mistral-7B model, optimized for performance and efficiency using the PEFT library and INT4 quantization.

## Model Details

### Model Description

Mistral-sci-phi is a model fine-tuned from the Mistral-7B base model. It has been optimized for enhanced performance and reduced size, making it highly efficient for various NLP tasks. The model is trained using the "emrgnt-cmplxty/sciphi-textbooks-are-all-you-need" dataset from the Hugging Face Hub, ensuring it's well-suited for real-world applications.

- **Developed by:** Arturo de Pablo
- **Trained by:** IZX, Hyper88
- **Model type:** Causal Language Model
- **Language(s) (NLP):** English
- **License:** [More Information Needed]
- **Finetuned from model:** mistralai/Mistral-7B-v0.1

### Model Sources

- **Repository:** [hyper88/ast1test](https://huggingface.co/hyper88/mistral-sci-phi-7B)


## Uses

### Direct Use

The model can be used directly for generating text and other NLP tasks.

### Downstream Use

It can also be integrated into larger systems for more complex applications.

### Out-of-Scope Use

The model should not be used for tasks beyond its training and capability scope.

## Bias, Risks, and Limitations

The model inherits the biases and limitations of the base Mistral-7B model. Users should be cautious of these when using the model.

### Recommendations

Users should evaluate the model's performance and biases in their specific use case and make adjustments as necessary.

## How to Get Started with the Model

The model can be loaded and used for inference using the Hugging Face Transformers library.

## Training Details

### Training Data

The model was trained on the "emrgnt-cmplxty/sciphi-textbooks-are-all-you-need" dataset available on the Hugging Face Hub.

### Training Procedure 

The model was fine-tuned using INT4 quantization to optimize its performance and size.

#### Training Hyperparameters

- Training was done with a learning rate of 2e-4
- Batch size of 12
- Trained for 3 epochs

## Evaluation

### Testing Data, Factors & Metrics

[More Information Needed]

### Results

[More Information Needed]

## Environmental Impact

The environmental impact is minimized due to the optimized size and efficiency of the model.

## Technical Specifications

### Model Architecture and Objective

The model is based on the Mistral-7B architecture and fine-tuned for enhanced performance.

### Compute Infrastructure

Sponsored by izx.ai

#### Software

- PEFT 0.6.0.dev0

## More Information

For more details, visit the [model repository](https://huggingface.co/hyper88/ast1test).

## Model Card Authors

- Arturo de Pablo (https://www.linkedin.com/in/arde88/)

## Model Card Contact

https://discord.gg/KGCeKP4ng9