---
language: 
  - en
  - zh
tags:
- PyTorch
- Text Classification
- Sentiment Analysis
- GloVe
datasets:
- imdb
widget:
- text: "This movie was absolutely wonderful. The acting was great and the plot was engaging."
- text: "I hated this film. It was a complete waste of time."
---

[English](#english) | [简体中文](#chinese)

<a id="english"></a>
# Simple Sentiment Analysis NN with GloVe Embeddings

This is a PyTorch-based neural network model for binary sentiment classification (positive/negative) on the IMDB dataset.

## Model Description

The model was built as a lightweight Feed-Forward Neural Network that utilizes pre-trained GloVe embeddings for token representations. It performs average pooling over the embedded tokens of a sequence to create a sentence-level representation, which then passes through two linear layers to output a sentiment probability.

### Architecture

- **Tokenization:** Custom whitespace and punctuation tokenizer. Sequence length is padded/truncated to 150 tokens.
- **Embedding Layer:** Pre-trained weights loaded from a custom `tiny_glove.json` dictionary. The embedding layer is frozen during training.
- **Pooling:** Average pooling across the sequence dimension `[batch_size, max_len, embed_dim] -> [batch_size, embed_dim]`.
- **Fully Connected Network:** 
  - `Linear(embed_dim, 64)` + `ReLU()`
  - `Linear(64, 1)` + `Sigmoid()`

## Training Data

The model was trained on a balanced subset of the IMDB movie reviews dataset containing 10,000 samples (`imdb_balanced_10k.csv`).
- **Train Split:** 8,000 samples (80%)
- **Test Split:** 2,000 samples (20%)

## Evaluation Results

- **Test Accuracy:** 0.6870 (68.7%)

## Training Parameters

- **Loss Function:** Binary Cross Entropy Loss (`BCELoss`)
- **Optimizer:** Adam
- **Learning Rate:** 0.001
- **Batch Size:** 64
- **Epochs:** 10

## Artifacts included

- `sentiment_nn.pth`: The PyTorch `state_dict` of the trained model.
- `vocab.pkl`: A serialized dictionary mapping string tokens to integer indices (includes `<PAD>` and `<UNK>` tokens).
- `label_encoder.pkl`: Scikit-learn LabelEncoder used to encode string labels to binary classes.

## How to use

```python
import torch
import torch.nn as nn
import joblib
import re
import string
import numpy as np

# Load Vocab and Label Encoder
vocab = joblib.load("vocab.pkl")
label_encoder = joblib.load("label_encoder.pkl")
embed_dim = 300 # Depends on the Glove embeddings used

# Recreate the PyTorch model
class SentimentNN(nn.Module):
    def __init__(self, vocab_size, embed_dim):
        super(SentimentNN, self).__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.fc1 = nn.Linear(embed_dim, 64)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(64, 1)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        embeds = self.embedding(x)
        out = embeds.mean(dim=1)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.sigmoid(out)
        return out

model = SentimentNN(len(vocab), embed_dim)
model.load_state_dict(torch.load("sentiment_nn.pth"), strict=False)
model.eval()

# Inference
MAX_LEN = 150
text = "This movie is amazing!"
text = str(text).lower()
text = re.sub(r'[' + string.punctuation + ']', ' ', text)
tokens = text.split()

indices = [vocab.get(t, 1) for t in tokens[:MAX_LEN]]
if len(indices) < MAX_LEN:
    indices += [0] * (MAX_LEN - len(indices))

input_tensor = torch.tensor([indices], dtype=torch.long)

with torch.no_grad():
    prediction = model(input_tensor).item()
    
# Get String Label
predicted_class = 1 if prediction > 0.5 else 0
print("Sentiment:", label_encoder.inverse_transform([predicted_class])[0])
print(f"Confidence: {prediction:.4f}")
```

---

<a id="chinese"></a>
# 基于 GloVe 词向量的简单情感分析神经网络

这是一个基于 PyTorch 构建的二分类情感分析（正向/负向）神经网络模型，在 IMDB 数据集上进行训练。

## 模型描述

该模型被设计为一个轻量级的前馈神经网络（Feed-Forward Neural Network），利用预训练的 GloVe 词向量作为 token 的嵌入表示。它通过对句子中所有 token 的嵌入向量进行平均池化（Average Pooling）来生成句子的全局特征表示，随后通过两个紧接的全连接线性层输出该句的情感概率预测。

### 模型架构

- **分词（Tokenization）:** 自定义基于空格和标点符号截断的分词器。输入序列长度固定统一为 150 个 tokens（不足在末尾补长，超出则截断）。
- **嵌入层（Embedding Layer）:** 初始化权重加载自项目中提供的 `tiny_glove.json` 预训练词向量文件字典。该层参数在训练期间冻结。
- **池化层（Pooling）:** 在序列维度即单个句子的长度方向上进行全局平均池化：`[batch_size, max_len, embed_dim] -> [batch_size, embed_dim]`。
- **全连接层（Fully Connected Network）:** 
  - 第一层 `Linear(embed_dim, 64)` + `ReLU()`
  - 第二层 `Linear(64, 1)` + `Sigmoid()` 激活函数输出最终概率。

## 训练数据

模型在一个内容分布平衡的 IMDB 电影评论高质量子集（包含 10,000 条样本数据，即项目中的 `imdb_balanced_10k.csv`）上完成训练。
- **训练集 (Train Split):** 8,000 条样本 (占比 80%)
- **测试集 (Test Split):** 2,000 条样本 (占比 20%)

## 评估结果 (Evaluation Results)

- **测试集准确率 (Test Accuracy):** 0.6870 (68.7%)

## 训练参数

- **损失函数（Loss Function）:** 二元交叉熵损失函数 (`BCELoss`)
- **优化器（Optimizer）:** Adam
- **学习率（Learning Rate）:** 0.001
- **批大小（Batch Size）:** 64
- **总轮数（Epochs）:** 10

## 包含的文件

包含以下工作流自动生成的参数模型和必要的前置推理组件：
- `sentiment_nn.pth`: 训练完毕后的 PyTorch 网络模型 `state_dict` 权重字典。
- `vocab.pkl`: 一个映射字典，将文本中的 tokens 映射转化为具体的整型 ID（保留了包括 `<PAD>` 和 `<UNK>` 字段）。
- `label_encoder.pkl`: Scikit-learn 的 LabelEncoder 对象，用于预测通过后把二分类数值复原回原本的字符串文字标签。

## 如何使用

可以参考以下的 Python 推理模板逻辑将文本转为正确结果：

```python
import torch
import torch.nn as nn
import joblib
import re
import string
import numpy as np

# 加载词表和标签编码器
vocab = joblib.load("vocab.pkl")
label_encoder = joblib.load("label_encoder.pkl")
embed_dim = 300 # 需要与预训练词向量维度一致

# 重新声明 PyTorch 模型结构
class SentimentNN(nn.Module):
    def __init__(self, vocab_size, embed_dim):
        super(SentimentNN, self).__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.fc1 = nn.Linear(embed_dim, 64)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(64, 1)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        embeds = self.embedding(x)
        out = embeds.mean(dim=1)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.sigmoid(out)
        return out

model = SentimentNN(len(vocab), embed_dim)
model.load_state_dict(torch.load("sentiment_nn.pth"), strict=False)
model.eval()

# 推理预测
MAX_LEN = 150
text = "This movie is amazing!"
text = str(text).lower()
text = re.sub(r'[' + string.punctuation + ']', ' ', text)
tokens = text.split()

indices = [vocab.get(t, 1) for t in tokens[:MAX_LEN]]
if len(indices) < MAX_LEN:
    indices += [0] * (MAX_LEN - len(indices))

input_tensor = torch.tensor([indices], dtype=torch.long)

with torch.no_grad():
    prediction = model(input_tensor).item()
    
# 获取结果
predicted_class = 1 if prediction > 0.5 else 0
print("情感判定 (Sentiment):", label_encoder.inverse_transform([predicted_class])[0])
print(f"置信度 (Confidence): {prediction:.4f}")
```