|
|
--- |
|
|
license: agpl-3.0 |
|
|
base_model: unsloth/DeepSeek-R1-0528-Qwen3-8B |
|
|
tags: |
|
|
- marxism-leninism |
|
|
- grpo |
|
|
- llama-cpp |
|
|
- ollama |
|
|
- political-education |
|
|
- marxism |
|
|
- communism |
|
|
- political-extremism |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
# MLMLML - Machine Learning Marxist-Leninist Models of Language |
|
|
|
|
|
A GRPO fine-tuned language model for Marxist-Leninist political education and analysis. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
This model is fine-tuned from `unsloth/DeepSeek-R1-0528-Qwen3-8B` using Group Relative Policy Optimization (GRPO) |
|
|
on a curated dataset of Marxist-Leninist Q&A pairs from [ProleWiki](https://en.prolewiki.org/). |
|
|
|
|
|
The training rewards: |
|
|
- **Ideological firmness**: Clear positions grounded in material analysis |
|
|
- **Coherence**: Self-consistent, well-structured responses |
|
|
- **Accuracy**: Faithful to Marxist-Leninist theory and historical evidence |
|
|
|
|
|
The training penalizes: |
|
|
- "Both-sidesing" and false balance |
|
|
- Hedging and evasive language |
|
|
- Bourgeois framing and ahistorical claims |
|
|
|
|
|
## Writing Style |
|
|
|
|
|
Following Chairman Mao's guidance in "Oppose Stereotyped Party Writing": |
|
|
- **Vigorous, lively, fresh and forceful** - never drab or stereotyped |
|
|
- **Audience-aware** - "When shooting an arrow, one must aim at the target" |
|
|
- **Investigation-based** - "No investigation, no right to speak" |
|
|
- **Clear positions** - FOR and AGAINST, using scientific argument |
|
|
|
|
|
## Usage |
|
|
|
|
|
### Download and Convert to GGUF |
|
|
|
|
|
```bash |
|
|
# Clone the repo |
|
|
git lfs install |
|
|
git clone https://huggingface.co/percyraskova/MLMLML |
|
|
cd MLMLML |
|
|
|
|
|
# Convert to GGUF (requires llama.cpp) |
|
|
python ~/llama.cpp/convert_hf_to_gguf.py . --outfile MLMLML-F16.gguf --outtype f16 |
|
|
|
|
|
# Quantize to Q4_K_M |
|
|
~/llama.cpp/build/bin/llama-quantize MLMLML-F16.gguf MLMLML-Q4_K_M.gguf Q4_K_M |
|
|
|
|
|
# Create Ollama model |
|
|
ollama create mlmlml -f Modelfile |
|
|
ollama run mlmlml |
|
|
``` |
|
|
|
|
|
### Direct with Transformers |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("percyraskova/MLMLML") |
|
|
tokenizer = AutoTokenizer.from_pretrained("percyraskova/MLMLML") |
|
|
|
|
|
inputs = tokenizer("What is imperialism?", return_tensors="pt") |
|
|
outputs = model.generate(**inputs, max_new_tokens=512) |
|
|
print(tokenizer.decode(outputs[0])) |
|
|
``` |
|
|
|
|
|
## Training Details |
|
|
|
|
|
- **Base model**: unsloth/DeepSeek-R1-0528-Qwen3-8B |
|
|
- **Method**: GRPO (Group Relative Policy Optimization) |
|
|
- **Dataset**: ProleWiki Q&A pairs (~4500 samples) |
|
|
- **Epochs**: 2 |
|
|
- **Hardware**: NVIDIA A100 80GB |
|
|
|
|
|
## Limitations |
|
|
|
|
|
This model is designed for educational purposes about Marxist-Leninist theory and analysis. |
|
|
It takes clear ideological positions and is not intended to be "neutral" on class struggle, |
|
|
imperialism, or other questions where Marxism-Leninism has definite answers. |
|
|
|
|
|
## License |
|
|
|
|
|
Apache 2.0 |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you use this model, please cite ProleWiki as the source of training data. |
