| --- |
| license: apache-2.0 |
| language: |
| - en |
| library_name: transformers |
| pipeline_tag: text-generation |
| --- |
| # Description |
| Solver AI TR-3A-01 model is a tunned LLM model with 7.9B parameters, designed to help marketers with their text content generation. Built on the open-source Mistral 7B instruction-tuned model and trained on domain-specific data, the current version is focused on writing marketing text for social posts and ads. |
| This is part of our exploration into leveraging AI to create effective, engaging social media content that is targeted and on-brand. |
|
|
| # Usage |
| Below we share some code snippets on how to get quickly started with running the model. First, update the Transformers library with: |
|
|
| ```pip install -U transformers``` |
|
|
| Running the model on GPU |
|
|
| ```python |
| # pip install accelerate |
| from transformers import AutoTokenizer, AutoModelForCausalLM |
| import torch |
| |
| tokenizer = AutoTokenizer.from_pretrained("solverai/TR-3A-01") |
| model = AutoModelForCausalLM.from_pretrained( |
| "solverai/TR-3A-01", |
| device_map="auto", |
| torch_dtype=torch.bfloat16, |
| ) |
| |
| messages = [ |
| {"role": "user", "content": "create a social post for buying donuts in Twiter style."}, |
| ] |
| input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True).to("cuda") |
| |
| outputs = model.generate(**input_ids, max_new_tokens=256) |
| print(tokenizer.decode(outputs[0])) |
| |
| ``` |
|
|
| # Benchmarks |
| To understand the capabilities, we use a variety of benchmarks. At the high-level overview of the model quality on representative benchmarks: |
|
|
| Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr| |
| |---------------------------------------|------:|----------------|-----:|-----------|---|-----:|---|-----:| |
| |arc_challenge | 1|none | 0|acc |↑ |0.4838|± |0.0146| |
| | | |none | 0|acc_norm |↑ |0.5102|± |0.0146| |
| |gsm8k | 3|flexible-extract| 5|exact_match|↑ |0.4572|± |0.0137| |
| | | |strict-match | 5|exact_match|↑ |0.4428|± |0.0137| |
| |hellaswag | 1|none | 0|acc |↑ |0.6082|± |0.0049| |
| | | |none | 0|acc_norm |↑ |0.7946|± |0.0040| |
| |mmlu | 1|none | |acc |↑ |0.5664|± |0.0039| |
| | - humanities | 1|none | |acc |↑ |0.5105|± |0.0068| |
| | - formal_logic | 0|none | 0|acc |↑ |0.4524|± |0.0445| |
| | - high_school_european_history | 0|none | 0|acc |↑ |0.6788|± |0.0365| |
| | - high_school_us_history | 0|none | 0|acc |↑ |0.7745|± |0.0293| |
| | - high_school_world_history | 0|none | 0|acc |↑ |0.7553|± |0.0280| |
| | - international_law | 0|none | 0|acc |↑ |0.7438|± |0.0398| |
| | - jurisprudence | 0|none | 0|acc |↑ |0.7130|± |0.0437| |
| | - logical_fallacies | 0|none | 0|acc |↑ |0.7607|± |0.0335| |
| | - moral_disputes | 0|none | 0|acc |↑ |0.6098|± |0.0263| |
| | - moral_scenarios | 0|none | 0|acc |↑ |0.2782|± |0.0150| |
| | - philosophy | 0|none | 0|acc |↑ |0.6431|± |0.0272| |
| | - prehistory | 0|none | 0|acc |↑ |0.6173|± |0.0270| |
| | - professional_law | 0|none | 0|acc |↑ |0.3950|± |0.0125| |
| | - world_religions | 0|none | 0|acc |↑ |0.8129|± |0.0299| |
| | - other | 1|none | |acc |↑ |0.6379|± |0.0083| |
| | - business_ethics | 0|none | 0|acc |↑ |0.5100|± |0.0502| |
| | - clinical_knowledge | 0|none | 0|acc |↑ |0.6491|± |0.0294| |
| | - college_medicine | 0|none | 0|acc |↑ |0.5376|± |0.0380| |
| | - global_facts | 0|none | 0|acc |↑ |0.3000|± |0.0461| |
| | - human_aging | 0|none | 0|acc |↑ |0.6099|± |0.0327| |
| | - management | 0|none | 0|acc |↑ |0.7476|± |0.0430| |
| | - marketing | 0|none | 0|acc |↑ |0.8547|± |0.0231| |
| | - medical_genetics | 0|none | 0|acc |↑ |0.6400|± |0.0482| |
| | - miscellaneous | 0|none | 0|acc |↑ |0.7497|± |0.0155| |
| | - nutrition | 0|none | 0|acc |↑ |0.6699|± |0.0269| |
| | - professional_accounting | 0|none | 0|acc |↑ |0.4326|± |0.0296| |
| | - professional_medicine | 0|none | 0|acc |↑ |0.6324|± |0.0293| |
| | - virology | 0|none | 0|acc |↑ |0.4398|± |0.0386| |
| | - social sciences | 1|none | |acc |↑ |0.6727|± |0.0082| |
| | - econometrics | 0|none | 0|acc |↑ |0.3772|± |0.0456| |
| | - high_school_geography | 0|none | 0|acc |↑ |0.7172|± |0.0321| |
| | - high_school_government_and_politics| 0|none | 0|acc |↑ |0.7979|± |0.0290| |
| | - high_school_macroeconomics | 0|none | 0|acc |↑ |0.5846|± |0.0250| |
| | - high_school_microeconomics | 0|none | 0|acc |↑ |0.6597|± |0.0308| |
| | - high_school_psychology | 0|none | 0|acc |↑ |0.7982|± |0.0172| |
| | - human_sexuality | 0|none | 0|acc |↑ |0.6412|± |0.0421| |
| | - professional_psychology | 0|none | 0|acc |↑ |0.5637|± |0.0201| |
| | - public_relations | 0|none | 0|acc |↑ |0.6182|± |0.0465| |
| | - security_studies | 0|none | 0|acc |↑ |0.6857|± |0.0297| |
| | - sociology | 0|none | 0|acc |↑ |0.8408|± |0.0259| |
| | - us_foreign_policy | 0|none | 0|acc |↑ |0.7700|± |0.0423| |
| | - stem | 1|none | |acc |↑ |0.4757|± |0.0086| |
| | - abstract_algebra | 0|none | 0|acc |↑ |0.2500|± |0.0435| |
| | - anatomy | 0|none | 0|acc |↑ |0.5556|± |0.0429| |
| | - astronomy | 0|none | 0|acc |↑ |0.6711|± |0.0382| |
| | - college_biology | 0|none | 0|acc |↑ |0.6389|± |0.0402| |
| | - college_chemistry | 0|none | 0|acc |↑ |0.3900|± |0.0490| |
| | - college_computer_science | 0|none | 0|acc |↑ |0.5200|± |0.0502| |
| | - college_mathematics | 0|none | 0|acc |↑ |0.3700|± |0.0485| |
| | - college_physics | 0|none | 0|acc |↑ |0.4216|± |0.0491| |
| | - computer_security | 0|none | 0|acc |↑ |0.7000|± |0.0461| |
| | - conceptual_physics | 0|none | 0|acc |↑ |0.4298|± |0.0324| |
| | - electrical_engineering | 0|none | 0|acc |↑ |0.4759|± |0.0416| |
| | - elementary_mathematics | 0|none | 0|acc |↑ |0.3704|± |0.0249| |
| | - high_school_biology | 0|none | 0|acc |↑ |0.6839|± |0.0265| |
| | - high_school_chemistry | 0|none | 0|acc |↑ |0.4975|± |0.0352| |
| | - high_school_computer_science | 0|none | 0|acc |↑ |0.6200|± |0.0488| |
| | - high_school_mathematics | 0|none | 0|acc |↑ |0.2815|± |0.0274| |
| | - high_school_physics | 0|none | 0|acc |↑ |0.3113|± |0.0378| |
| | - high_school_statistics | 0|none | 0|acc |↑ |0.5093|± |0.0341| |
| | - machine_learning | 0|none | 0|acc |↑ |0.4196|± |0.0468| |
| |truthfulqa_mc2 | 2|none | 0|acc |↑ |0.5575|± |0.0154| |
| |winogrande | 1|none | 0|acc |↑ |0.7206|± |0.0126| |
| |
| |
| # Sample Output |
| |
| ``` |
| User: create a social media post for 20% off for diva sneakers, make it a instagram post |
| Assistant: |
| 🌟👟 Don't miss out on this Instagram exclusive!🎉 Our Diva Sneakers |
| are back and we're offering a special discount just for you, my fashionistas!👊 |
| For a limited time, if you buy two pairs, you'll receive 20% off your purchase. |
| 👟But hurry, this offer is only available on our website, so head over there |
| now and grab your new kicks. Tag a friend who needs these and hit the shop! |
| 💫#diva sneakers #limitedtimeoffer #fashionistas #backinstock |
| |
| User: same thing, create a twitter post, keep it short |
| Assistant: |
| 👟 Back in stock now!👊 For a limited time, buy two pairs and get 20% off on |
| our Instagram exclusive Diva Sneakers!🌟 Don't miss out on this limited time offer! |
| #diva sneakers #limitedtimeoffer #backinstock |
| |
| |
| User: get a new look for summer with Deviant Diva's new makeup lin |
| Assistant: |
| 🌊Summer, sun, and style! 💆🤩 Ready to refresh your wardrobe with our latest |
| collection of makeup products? Get ready to take over the beauty world this |
| summer with Deviant Diva's hottest makeup line for the season! Shop now and |
| enjoy our new range of light, shimmery,and metallic shades for a flawless |
| all-day look. So go ahead, treat yourself and let your inner beauty shine! |
| 🌸🌱 #beautylover #summermakeup #beautycommunity #beautyaddict |
| |
| |
| User: start your shopify store today |
| Assistant: |
| 🤎🛍️ Ready to start your own online store?🛍️ With Shopify, you can create |
| your unique and personalized storefront and streamline your business |
| processes, customer service, marketing,and more! From inventory management |
| to shipping and fulfillment, this e-commerce platform has everything you |
| need to start your online shop. Sign up today and unlock your passion with |
| your handmade products! 🤩 #e-commerce #Shopify #SmallBusiness #StartingYourOwnBusiness |
| |
| User: buy a fender strat |
| Assistant: |
| 🎸🎶Get ready to unleash your potential with a Fender Stratocaster! 🎸 |
| Whether you're playingblues, rock, or pop, this legendary guitar is perfect |
| for you. With its bright and clear sound,smooth fingerboard, and versatile |
| pickup selector, you'll sound and look like a rockstar!🤘 So go ahead and |
| invest in your musical journey today! 🎸 #fender stratocaster #guitarlover #musiclover #rockstar |
| |
| User: check out the new allbirds |
| Assistant: |
| 🌺🌳 Discover a comfortable and stylish shoe that's not only stylish but |
| also eco-friendly and sustainable! 🌎🌲 Allbirds' shoes are made from |
| Merino wool and recycled materials, ensuring fashion-conscious consumers |
| can enjoy a guilt-free shopping experience. From casual walking to adventurous |
| hiking, these shoes will take you anywhere in style! 🙌 Check out the new line and |
| start enjoying nature while looking fresh and cool! #allbirds #sustainableshoes |
| #recycledmaterials #adventuretime #naturalmaterials |
| |
| User: create a post about rivian cars |
| Assistant: |
| 🌊Rev up your summer with our latest collection of stylish cars from Rivian! 🌊 |
| From eco-friendly convertibles to sleek and sporty SUVs, our luxury cars add |
| a touch of elegance and sophistication to your summer style. Whether you're |
| driving to work, heading out on adventures, or lounging by the pool, our |
| stylish cars give you the perfect ride with comfortability and style. Shop |
| now and complete your perfect summer look today! 🌊🌞 #luxurrycars #ecofriendly |
| #travelingmadeeasier #luxurylife #Rivianluxury |
| |
| ``` |
| # Ethical Considerations and Limitations |
| As we continue to develop and refine our social media post model, it's essential to acknowledge the ethical considerations and limitations inherent in its deployment. Here are some key points: |
| |
| ## Ethical Considerations |
| Bias and Fairness: Our model is fine-tuned on domain datasets, but still there remains a risk of unintentional bias, particularly regarding sensitive topics like gender, race, and socioeconomic status. We are committed to continuous monitoring and updating to mitigate these biases. |
| |
| Impact on Human Interaction: Our model aims to foster positive and inclusive conversations on social media. We are aware of the risks of echo chambers and polarization and strive to design the model in a way that encourages diverse perspectives and healthy dialogue. |
| |
| ## Limitations |
| Contextual Understanding: The model may not always grasp the nuanced context of social media posts, leading to occasional misinterpretations or irrelevant content generation. |
| |
| Cultural Sensitivity: Due to the global nature of social media, the model may sometimes lack sensitivity to cultural differences, idioms, or local norms. |
| |
| Data Dependence: The model's performance is heavily influenced by the quality and diversity of its training data. We acknowledge that biases in the training data can impact the outputs and are continuously working to improve data quality. |
| |
| Scalability: While the model is designed to handle large-scale content, real-time processing and resource demands can present challenges, particularly in high-traffic environments. |
| |
| Our Commitment |
| We are dedicated to ethical AI development and are continuously improving our model to better serve the community while addressing these challenges. We encourage users to provide feedback and work with us to create a safer, more inclusive, and effective tool. |
| |
| Thank you for your support and understanding as we navigate the complexities of deploying AI in social media contexts. |
| |
