| --- |
| library_name: transformers |
| tags: [] |
| --- |
| |
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
|
|
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| <!-- Provide a longer summary of what this model is. --> |
|
|
| This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. |
|
|
| - **Developed by:** [More Information Needed] |
| - **Funded by [optional]:** [More Information Needed] |
| - **Shared by [optional]:** [More Information Needed] |
| - **Model type:** [More Information Needed] |
| - **Language(s) (NLP):** [More Information Needed] |
| - **License:** [More Information Needed] |
| - **Finetuned from model [optional]:** [More Information Needed] |
|
|
| ### Model Sources [optional] |
|
|
| <!-- Provide the basic links for the model. --> |
|
|
| - **Repository:** [More Information Needed] |
| - **Paper [optional]:** [More Information Needed] |
| - **Demo [optional]:** [More Information Needed] |
|
|
| ## Uses |
|
|
| <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
| ### Direct Use |
|
|
| <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
|
|
| [More Information Needed] |
|
|
| ### Downstream Use [optional] |
|
|
| <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
|
| [More Information Needed] |
|
|
|
|
| ## Evaluation |
|
|
| hf-causal-experimental (pretrained=T3Q-LLM/T3Q-LLM2-sft1.1,use_accelerate=true,trust_remote_code=true), limit: None, provide_description: False, num_fewshot: 0, batch_size: 8 |
| | Task |Version| Metric |Value | |Stderr| |
| |----------------|------:|--------|-----:|---|-----:| |
| |kobest_boolq | 0|acc |0.9402|± |0.0063| |
| | | |macro_f1|0.9401|± |0.0063| |
| |kobest_copa | 0|acc |0.7770|± |0.0132| |
| | | |macro_f1|0.7766|± |0.0132| |
| |kobest_hellaswag| 0|acc |0.5040|± |0.0224| |
| | | |acc_norm|0.5580|± |0.0222| |
| | | |macro_f1|0.5019|± |0.0223| |
| |kobest_sentineg | 0|acc |0.7254|± |0.0224| |
| | | |macro_f1|0.7076|± |0.0236| |
| |
| |
