SebasLopez-ai/hybridKMeans-category-clustering

MiniBatchKMeans — Category Clustering

Clusters Amazon product review embeddings into 6 semantic categories.

Embedding model: nomic-ai/nomic-embed-text-v1.5 (768-dim)
Served centroids: K-Means (Lloyd) final fit
Label assignment: MiniBatchKMeans (k=6, batch_size=1024, 10 epochs, 3 inits)
Dataset: 219,998 reviews (Amazon Reviews 2023)

---

How this model was trained

This is a hybrid two-algorithm pipeline. Two clustering algorithms were trained side by side and each contributes to the final artifact:

                    219,998 reviews
                          │
                          ▼
               Nomic Embed (768-dim)
                          │
            ┌─────────────┴─────────────┐
            ▼                           ▼
     K-Means (Lloyd)            MiniBatchKMeans
     k-sweep 2…10               k-sweep 2…10
     per-k: inertia + silh      per-k: inertia + silh
            │                           │
            └─────────────┬─────────────┘
                          ▼
                   k=6 selected
            (elbow method + domain interpretability)
                          │
            ┌─────────────┴─────────────┐
            ▼                           ▼
     K-Means final fit           MiniBatchKMeans final fit
     n_init=10, max_iter=300     batch_size=1024, 10 epochs
     converged in 112 iters      3 inits, best kept
     fit time: 57.1s             fit time: 19.6s
            │                           │
            ▼                           ▼
   ┌──────────────────┐     ┌────────────────────────┐
   │ cluster_centroids │     │  Label assignment       │
   │     .npy          │     │  (219,998 reviews)      │
   │ (6, 768) float32  │     │  → clusters.csv         │
   │                   │     │  → cluster_profiles.json│
   │  SERVED WEIGHTS   │     │  PRODUCTION CHOICE      │
   └──────────────────┘     └────────────────────────┘

Why two algorithms?

Role	Algorithm	Why
Centroids (the weights you download)	K-Means (Lloyd)	Converges to a true local optimum. Slightly better Silhouette in the k=6 comparison (0.0304 vs 0.0311 — within noise). Centroids are mathematically cleaner.
Labels (cluster assignments, profiles)	MiniBatchKMeans	Scales to the full 571M-review dataset at O(batch·k·d) memory. Retains ~102% of K-Means quality at 3× faster fit time (19.6s vs 57.1s). The production-ready algorithm.

What this means in practice

• Inference: cluster_centroids.npy (K-Means weights) is what you load to classify new reviews. The cosine distance to these 6 centroids determines the cluster.
• Cluster profiles: The sizes, avg ratings, top terms, and category distributions in cluster_profiles.json come from the MiniBatchKMeans label assignment.
• Interchangeable: The silhouette gap is negligible (0.0311 vs 0.0304), meaning both algorithms agree on cluster boundaries. The centroids from either would produce nearly identical results.

Full training history with per-batch inertia tracking for both algorithms: metrics_clustering.json.

---

How to use cluster_centroids.npy

import numpy as np

centroids = np.load("data/models/MiniBatchKMeans/cluster_centroids.npy")
# shape: (6, 768), dtype: float32
# centroids[i] is the 768-dim embedding of cluster i

To assign a new review embedding to the nearest cluster:

from scipy.spatial.distance import cdist
distances = cdist(new_embeddings, centroids, metric="cosine")
labels = np.argmin(distances, axis=1)

Requires scipy. Use new_embeddings with shape (n_reviews, 768) from the same Nomic model.

---

What the clusters mean

6 clusters discovered from review embeddings. Verbose profile: cluster_profiles.json.

#	%	Label	Avg ★	Top terms	Key categories
0	13.4%	Mixed (46% Neutral)	2.69	like, use, product, good, hair	Beauty & Personal Care, Grocery, All Beauty
1	5.6%	Strong Positive (93%)	4.74	good product, great, works	Clothing, Automotive, Health & Household
2	17.6%	Mixed (42% Neutral)	2.98	book, story, read, good	Kindle Store, Movies & TV, Books
3	16.3%	Mixed (53% Neutral)	2.96	described, small, fit, size, expected	Amazon Fashion, Clothing, Sports
4	24.6%	Strong Negative (66%)	1.94	work, use, just, money, did	Cell Phones, Electronics, Patio & Garden
5	22.4%	Strong Positive (90%)	4.68	great, love, easy, gift, nice	Toys & Games, Gift Cards, Office Products

---

Metrics

From metrics_clustering.json:

Metric	Value
Silhouette Score	0.0311
Standard K-Means Silhouette	0.0304
K-Means Final Inertia	116,999.41
MiniBatchKMeans Best Init Inertia	457.99
MiniBatch Fit Time	19.6s

Silhouette interpretation: 0.031 is low — this is expected for high-dimensional text embeddings (curse of dimensionality). The score is a relative comparator between k values, not an absolute quality measure. k=6 was selected via elbow method + domain interpretability (see metrics_clustering.json for the full k-sweep).

NOTE: The inertia values differ in scale because K-Means reports total inertia across all 219,998 points, while the MiniBatchKMeans fit_steps track batch-level inertia (per 1,024 samples).

---

Usage Examples

Full pipeline: raw text → Nomic embedding → cosine distance → cluster label.

Example 1 — Positive book review → Cluster 2 (Books/Media)

import numpy as np
from scipy.spatial.distance import cdist
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
centroids = np.load("data/models/MiniBatchKMeans/cluster_centroids.npy")

review = "This novel blew me away. The plot twists were masterful and I couldn't put it down."
embedding = model.encode([review])  # shape: (1, 768)
distances = cdist(embedding, centroids, metric="cosine")
cluster_id = int(np.argmin(distances))

Expected output: cluster_id = 2 — Books/Media cluster. The review talks about plot, novel, and reading experience — matches top terms book, story, read.

---

Example 2 — Enthusiastic toy/gift review → Cluster 5 (Positive gifts & toys)

review = "Bought this as a birthday gift for my nephew and he absolutely loves it! Easy to assemble and great quality."
embedding = model.encode([review])
distances = cdist(embedding, centroids, metric="cosine")
cluster_id = int(np.argmin(distances))

Expected output: cluster_id = 5 — Strong Positive cluster (avg 4.68★). Keywords gift, love, easy, great align perfectly with Cluster 5's top terms. Sentiment: 89.5% Positive.

---

Example 3 — Frustrated electronics review → Cluster 4 (Negative electronics)

review = "This charger stopped working after two weeks. Total waste of money. Don't buy this."
embedding = model.encode([review])
distances = cdist(embedding, centroids, metric="cosine")
cluster_id = int(np.argmin(distances))

Expected output: cluster_id = 4 — Strong Negative cluster (avg 1.94★). Keywords work (broken), money (waste), don't (negative) match Cluster 4's patterns. Top categories: Cell Phones, Electronics. Sentiment: 66.2% Negative.

---

Batch inference (N reviews at once)

reviews = ["Great product, highly recommend!", "Did not fit as described.", "Perfect gift for kids."]
embeddings = model.encode(reviews)                    # shape: (3, 768)
distances = cdist(embeddings, centroids, "cosine")    # shape: (3, 6)
labels = np.argmin(distances, axis=1)                 # e.g. [5, 3, 5]

---

Serving & Integration

Pick the approach that fits your stack. All examples assume the model files live at data/models/MiniBatchKMeans/.

1. Python CLI script (zero dependencies beyond the model)

Save as cluster_review.py and run: python cluster_review.py "Your review text here"

"""cluster_review.py — classify a review from the command line."""
import sys, json, numpy as np
from scipy.spatial.distance import cdist
from sentence_transformers import SentenceTransformer

MODEL_DIR = "data/models/MiniBatchKMeans"

# Load once at module level
embedder = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
centroids = np.load(f"{MODEL_DIR}/cluster_centroids.npy")
with open(f"{MODEL_DIR}/cluster_profiles.json") as f:
    profiles = json.load(f)

def classify(text: str) -> dict:
    emb = embedder.encode([text])
    cluster_id = int(np.argmin(cdist(emb, centroids, metric="cosine")))
    cluster = next(c for c in profiles["clusters"] if c["cluster_id"] == cluster_id)
    return {
        "text": text,
        "cluster_id": cluster_id,
        "avg_rating": cluster["avg_rating"],
        "sentiment": cluster["sentiment_distribution_pct"],
        "top_terms": cluster["top_terms"],
        "top_categories": cluster["top_categories"],
    }

if __name__ == "__main__":
    text = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else input("Review: ")
    result = classify(text)
    print(json.dumps(result, indent=2))

Sample output:

{
  "text": "This blender is amazing, smoothies every morning!",
  "cluster_id": 5,
  "avg_rating": 4.676,
  "sentiment": {"Positive": 89.5, "Neutral": 8.8, "Negative": 1.7},
  "top_terms": ["great", "love", "easy", "gift", "nice"],
  "top_categories": {"Toys_and_Games": 4.7, "Gift_Cards": 4.6, "Office_Products": 4.3}
}

---

2. FastAPI microservice (REST JSON endpoint)

"""api.py — lightweight REST API. Run: uvicorn api:app --port 8000"""
from contextlib import asynccontextmanager
from fastapi import FastAPI
from pydantic import BaseModel
import numpy as np, json
from scipy.spatial.distance import cdist
from sentence_transformers import SentenceTransformer

MODEL_DIR = "data/models/MiniBatchKMeans"

embedder = None
centroids = None
profiles = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    global embedder, centroids, profiles
    embedder = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
    centroids = np.load(f"{MODEL_DIR}/cluster_centroids.npy")
    with open(f"{MODEL_DIR}/cluster_profiles.json") as f:
        profiles = json.load(f)
    yield

app = FastAPI(lifespan=lifespan)

class ReviewInput(BaseModel):
    text: str

class ClusterResult(BaseModel):
    cluster_id: int
    avg_rating: float
    sentiment: dict
    top_terms: list[str]
    top_categories: dict

@app.post("/classify", response_model=ClusterResult)
def classify(review: ReviewInput):
    emb = embedder.encode([review.text])
    cluster_id = int(np.argmin(cdist(emb, centroids, metric="cosine")))
    c = next(c for c in profiles["clusters"] if c["cluster_id"] == cluster_id)
    return ClusterResult(
        cluster_id=cluster_id,
        avg_rating=round(c["avg_rating"], 2),
        sentiment=c["sentiment_distribution_pct"],
        top_terms=c["top_terms"],
        top_categories=c["top_categories"],
    )

Call it:

curl -X POST http://localhost:8000/classify \
  -H "Content-Type: application/json" \
  -d '{"text": "This book was a page-turner from start to finish."}'

{"cluster_id":2,"avg_rating":2.98,"sentiment":{"Neutral":42.5,"Negative":31.0,"Positive":26.5},
 "top_terms":["book","story","like","read","good"],
 "top_categories":{"Kindle_Store":16.6,"Movies_and_TV":14.6,"Books":13.4}}

---

3. HTML form + vanilla JavaScript (browser)

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Review Classifier</title>
  <style>
    body { font-family: system-ui; max-width: 600px; margin: 3rem auto; padding: 0 1rem; }
    textarea { width: 100%; height: 100px; margin-bottom: 0.5rem; }
    pre { background: #f5f5f5; padding: 1rem; border-radius: 6px; white-space: pre-wrap; }
  </style>
</head>
<body>
  <h2>What category is this review?</h2>
  <textarea id="review" placeholder="Paste a product review..."></textarea>
  <button onclick="classify()">Classify</button>
  <pre id="result"></pre>

  <script>
    async function classify() {
      const text = document.getElementById("review").value;
      const res = await fetch("http://localhost:8000/classify", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ text }),
      });
      const data = await res.json();
      document.getElementById("result").textContent = JSON.stringify(data, null, 2);
    }
  </script>
</body>
</html>

Open index.html in your browser, type a review, click "Classify" — result renders inline.

---

4. Google Colab interactive widget

# Run in a Colab cell — instant text box + classify button
import ipywidgets as widgets
from IPython.display import display, JSON

text_input = widgets.Textarea(placeholder="Paste a review...", layout={"width": "100%", "height": "80px"})
button = widgets.Button(description="Classify", button_style="primary")
output = widgets.Output()

def on_click(_):
    with output:
        output.clear_output()
        emb = embedder.encode([text_input.value])
        cluster_id = int(np.argmin(cdist(emb, centroids, metric="cosine")))
        c = next(c for c in profiles["clusters"] if c["cluster_id"] == cluster_id)
        display(JSON({k: c[k] for k in ["cluster_id","avg_rating","sentiment_distribution_pct","top_terms","top_categories"]}))

button.on_click(on_click)
display(text_input, button, output)

---

5. Streamlit dashboard (one-liner UI)

"""Save as streamlit_app.py — run: streamlit run streamlit_app.py"""
import streamlit as st
import numpy as np, json
from scipy.spatial.distance import cdist
from sentence_transformers import SentenceTransformer

@st.cache_resource
def load_model():
    m = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
    c = np.load("data/models/MiniBatchKMeans/cluster_centroids.npy")
    with open("data/models/MiniBatchKMeans/cluster_profiles.json") as f:
        p = json.load(f)
    return m, c, p

embedder, centroids, profiles = load_model()

st.title("Review Category Classifier")
review = st.text_area("Paste a product review:", height=100)

if st.button("Classify"):
    emb = embedder.encode([review])
    cluster_id = int(np.argmin(cdist(emb, centroids, metric="cosine")))
    c = next(c for c in profiles["clusters"] if c["cluster_id"] == cluster_id)
    col1, col2 = st.columns(2)
    with col1:
        st.metric("Cluster", cluster_id)
        st.metric("Avg Rating", f"{c['avg_rating']:.1f}★")
        st.write("**Top terms:**", ", ".join(c["top_terms"]))
    with col2:
        st.write("**Sentiment:**")
        for k, v in c["sentiment_distribution_pct"].items():
            st.progress(v / 100, text=f"{k}: {v}%")
        st.write("**Categories:**", c["top_categories"])

---

6. Batch CSV processor (process thousands of reviews at once)

"""batch_classify.py — reads a CSV with a 'review_text' column, writes results."""
import pandas as pd, numpy as np, json
from scipy.spatial.distance import cdist
from sentence_transformers import SentenceTransformer

MODEL_DIR = "data/models/MiniBatchKMeans"
BATCH_SIZE = 512

embedder = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
centroids = np.load(f"{MODEL_DIR}/cluster_centroids.npy")
with open(f"{MODEL_DIR}/cluster_profiles.json") as f:
    profiles = json.load(f)

df = pd.read_csv("input_reviews.csv")
embeddings = embedder.encode(df["review_text"].tolist(), batch_size=BATCH_SIZE, show_progress_bar=True)
labels = np.argmin(cdist(embeddings, centroids, metric="cosine"), axis=1)

df["cluster_id"] = labels
df["cluster_rating"] = [profiles["clusters"][lid]["avg_rating"] for lid in labels]
df.to_csv("classified_reviews.csv", index=False)
print(f"Done — {len(df)} reviews classified.")

---

Files in this folder

File	Description
cluster_centroids.npy	(6, 768) float32 — cluster center embeddings
cluster_profiles.json	Per-cluster stats (sizes, sentiment, top terms, categories)
clusters.csv	Per-review cluster assignments
embeddings_nomic.npz	Full 219,998 × 768 embedding matrix (compressed)
metrics_clustering.json	Full training history: k-sweep, fit steps, convergence