hugov/x-bot-llm

X-Bot LLM

A fine-tuned Mistral-7B model specialized for copywriting and social media content generation, particularly optimized for Twitter/X posts.

Model Description

This model is based on Mistral-7B-v0.1 and has been fine-tuned for generating engaging social media content with flexible length, tone, and goal options.

Key Features

• Flexible Length Control: Generate content from short tweets to longer threads
• Tone Adaptation: Supports professional, neutral, promotional, friendly, and moody tones
• Goal-Oriented: Optimized for reach, discussion, out-of-context, or neutral content
• 4-bit Quantization: Optimized for efficient inference with BitsAndBytes

Model Details

• Base Model: mistralai/Mistral-7B-v0.1
• Architecture: MistralForCausalLM
• Quantization: 4-bit (NF4) with BitsAndBytes
• Context Length: 32,768 tokens
• Languages: French and English

Usage

Using Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
from transformers import BitsAndBytesConfig
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("hugov/x-bot-llm", trust_remote_code=True)

# Configure quantization for GPU
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)

# Load model
model = AutoModelForCausalLM.from_pretrained(
    "hugov/x-bot-llm",
    quantization_config=quantization_config,
    device_map="auto",
    trust_remote_code=True,
    dtype=torch.float16
)

# Generate text
prompt = "Write a tweet about artificial intelligence"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.8,
        top_p=0.9,
        do_sample=True
    )

generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)

Using Hugging Face Inference API

import requests

API_URL = "https://router.huggingface.co/hf-inference/v1/models/hugov/x-bot-llm"
headers = {"Authorization": f"Bearer {YOUR_HF_TOKEN}"}

def generate(prompt, max_new_tokens=100, temperature=0.8, top_p=0.9):
    payload = {
        "inputs": prompt,
        "parameters": {
            "max_new_tokens": max_new_tokens,
            "temperature": temperature,
            "top_p": top_p,
            "return_full_text": False
        }
    }
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

# Example
result = generate("Write a tweet about AI")
print(result)

Generation Parameters

Length Options

• really_short: 40 tokens (~80-150 chars, 1-2 phrases)
• short: 60 tokens (~150-220 chars, single idea)
• normal: 100 tokens (~220-280 chars, complete thought)
• developed: 200 tokens (thread starter or multiple tweets)

Goal Options

• reach: Maximum creativity for virality (temperature: 0.95, top_p: 0.95)
• discussion: Moderate for conversations (temperature: 0.85, top_p: 0.9)
• out_of_context: Balanced for standalone content (temperature: 0.8, top_p: 0.85)
• neutral: Default balanced settings (temperature: 0.8, top_p: 0.9)

Tone Options

• pro: Professional (slightly more conservative)
• neutral: No adjustment
• promo: Promotional (more creative)
• friendly: Friendly (slightly more creative)
• moody: Emotional (more creative)

Requirements

• Python 3.8+
• PyTorch 2.0+
• Transformers 4.57.1+
• BitsAndBytes (for 4-bit quantization)
• CUDA-capable GPU (recommended)

Limitations

• This model requires CUDA/GPU support for local inference due to 4-bit quantization
• CPU inference is not supported
• For CPU usage, consider using the Hugging Face Inference API

Citation

@model{hugov/x-bot-llm,
  title={X-Bot LLM: A Fine-tuned Mistral-7B for Social Media Copywriting},
  author={hugov},
  year={2024},
  base_model={mistralai/Mistral-7B-v0.1}
}

License

Apache 2.0