Upload folder using huggingface_hub

README.md

@@ -0,0 +1,41 @@
+# Sentiment Analysis Models
+
+This repository contains two logistic regression models trained to predict sentiment scores.
+
+## Model Details
+- Base embedding model: mixedbread-ai/mxbai-embed-large-v1
+- Architecture: LogisticRegression (scikit-learn)
+- Training data: Custom sentiment dataset with dual expert annotations
+- Data split: 70% training, 15% development, 15% test
+
+## Performance Metrics
+
+### Development Set
+#### Against Expert 1:
+- Exact match: 49.27%
+- Within 1 level: 96.05%
+
+#### Against Expert 2:
+- Exact match: 41.00%
+- Within 1 level: 93.05%
+
+### Test Set
+#### Against Expert 1:
+- Exact match: 49.32%
+- Within 1 level: 94.93%
+
+#### Against Expert 2:
+- Exact match: 41.44%
+- Within 1 level: 91.51%
+
+## Usage
+
+See `inference.py` for an example of how to use these models to predict sentiment for new text.
+
+## Model Files
+- `model1.joblib`: Model trained on Expert 1 annotations
+- `model2.joblib`: Model trained on Expert 2 annotations
+
+## Data Files
+- `dev_results.csv`: Complete predictions on development set
+- `test_results.csv`: Complete predictions on test set

example_notebook.ipynb

@@ -0,0 +1,122 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Sentiment Analysis Model Demo\n",
+    "This notebook demonstrates how to use the sentiment analysis models to predict sentiment for new text."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import joblib\n",
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "from sentence_transformers import SentenceTransformer"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Load the models\n",
+    "model1 = joblib.load('model1.joblib')\n",
+    "model2 = joblib.load('model2.joblib')\n",
+    "\n",
+    "# Load the embedder\n",
+    "embedder = SentenceTransformer('BAAI/bge-large-en-v1.5')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def predict_sentiment(text):\n",
+    "    # Generate embedding\n",
+    "    embedding = embedder.encode([text])\n",
+    "    \n",
+    "    # Make predictions\n",
+    "    pred1 = model1.predict(embedding)[0]\n",
+    "    pred2 = model2.predict(embedding)[0]\n",
+    "    \n",
+    "    # Average and round\n",
+    "    final_prediction = np.round((pred1 + pred2) / 2).astype(int)\n",
+    "    \n",
+    "    return final_prediction, pred1, pred2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Try with a sample text\n",
+    "sample_text = \"I absolutely loved this movie! The actors were amazing and the plot was fantastic.\"\n",
+    "final_score, score1, score2 = predict_sentiment(sample_text)\n",
+    "\n",
+    "print(f\"Text: {sample_text}\")\n",
+    "print(f\"Final sentiment score: {final_score}\")\n",
+    "print(f\"Model 1 score: {score1}\")\n",
+    "print(f\"Model 2 score: {score2}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Try with multiple texts\n",
+    "texts = [\n",
+    "    \"This product is terrible. Complete waste of money.\",\n",
+    "    \"The service was okay, nothing special.\",\n",
+    "    \"Absolutely fantastic experience! Would highly recommend.\",\n",
+    "    \"Not what I expected, but it wasn't bad either.\"\n",
+    "]\n",
+    "\n",
+    "results = []\n",
+    "for text in texts:\n",
+    "    final_score, score1, score2 = predict_sentiment(text)\n",
+    "    results.append({\n",
+    "        'Text': text,\n",
+    "        'Final Score': final_score,\n",
+    "        'Expert 1 Score': score1,\n",
+    "        'Expert 2 Score': score2\n",
+    "    })\n",
+    "\n",
+    "pd.DataFrame(results)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

inference.py

@@ -0,0 +1,33 @@
+
+import joblib
+import numpy as np
+from sentence_transformers import SentenceTransformer
+
+# Load the models
+model1 = joblib.load('model1.joblib')
+model2 = joblib.load('model2.joblib')
+
+# Load the embedder
+embedder = SentenceTransformer('BAAI/bge-large-en-v1.5')
+
+def predict_sentiment(text):
+    # Generate embedding
+    embedding = embedder.encode([text])
+    
+    # Make predictions
+    pred1 = model1.predict(embedding)[0]
+    pred2 = model2.predict(embedding)[0]
+    
+    # Average and round
+    final_prediction = np.round((pred1 + pred2) / 2).astype(int)
+    
+    return final_prediction, pred1, pred2
+
+# Example usage
+if __name__ == "__main__":
+    test_text = "I really enjoyed this product!"
+    final_score, score1, score2 = predict_sentiment(test_text)
+    print(f"Text: {test_text}")
+    print(f"Final sentiment score: {final_score}")
+    print(f"Model 1 score: {score1}")
+    print(f"Model 2 score: {score2}")

model-card.md

@@ -0,0 +1,65 @@
+---
+language: en
+license: mit
+library_name: scikit-learn
+tags:
+- sentiment-analysis
+- text-classification
+- scikit-learn
+- sentence-transformers
+datasets:
+- custom_sentiment_dataset
+metrics:
+- accuracy
+---
+
+# Sentiment Analysis Model
+
+This model predicts sentiment scores based on text input. It uses sentence embeddings from BAAI/bge-large-en-v1.5 and logistic regression classifiers.
+
+## Model Description
+
+This repository contains two logistic regression models trained to predict sentiment scores based on text embeddings. The models were trained on a custom dataset with annotations from two different experts.
+
+### Model Architecture
+
+- Base embedding model: BAAI/bge-large-en-v1.5
+- Classifier: LogisticRegression (scikit-learn)
+- Final prediction: Average of both model predictions, rounded to nearest integer
+
+## Intended Use and Limitations
+
+The model is designed for sentiment analysis tasks. The model works best with English text similar to the training data.
+
+## Training and Evaluation Data
+
+The model was trained on a custom dataset with:
+- 70% training data
+- 15% development data 
+- 15% test data
+
+Each sample has annotations from two human experts.
+
+## Evaluation Results
+
+See README.md for detailed performance metrics on both development and test sets.
+
+## Using the Models
+
+```python
+import joblib
+import numpy as np
+from sentence_transformers import SentenceTransformer
+
+# Load the models
+model1 = joblib.load('model1.joblib')
+model2 = joblib.load('model2.joblib')
+embedder = SentenceTransformer('BAAI/bge-large-en-v1.5')
+
+def predict_sentiment(text):
+    embedding = embedder.encode([text])
+    pred1 = model1.predict(embedding)[0]
+    pred2 = model2.predict(embedding)[0]
+    final_prediction = np.round((pred1 + pred2) / 2).astype(int)
+    return final_prediction
+```

model1.joblib

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8e7d13df3bef2dfdc42a8fbcaa19a9f8cbf74b06399cf2b57f17a6198a4a693
+size 50087

model2.joblib

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5bcef3af140f0f8d33b0b4e4cefa681bb0504bebbc5ce9f40b4f87822b655a0
+size 50087

requirements.txt

@@ -0,0 +1,6 @@
+sentence-transformers>=2.2.2
+scikit-learn>=1.0.0
+numpy>=1.20.0
+joblib>=1.1.0
+pandas>=1.3.0
+tabulate>=0.8.9