README.md

---
license: agpl-3.0
base_model: unsloth/DeepSeek-R1-0528-Qwen3-8B
tags:
- marxism-leninism
- grpo
- llama-cpp
- ollama
- political-education
- marxism
- communism
- political-extremism
language:
- en
pipeline_tag: text-generation
---

# MLMLML - Machine Learning Marxist-Leninist Models of Language

A GRPO fine-tuned language model for Marxist-Leninist political education and analysis.

## Model Description

This model is fine-tuned from `unsloth/DeepSeek-R1-0528-Qwen3-8B` using Group Relative Policy Optimization (GRPO) 
on a curated dataset of Marxist-Leninist Q&A pairs from [ProleWiki](https://en.prolewiki.org/).

The training rewards:
- **Ideological firmness**: Clear positions grounded in material analysis
- **Coherence**: Self-consistent, well-structured responses
- **Accuracy**: Faithful to Marxist-Leninist theory and historical evidence

The training penalizes:
- "Both-sidesing" and false balance
- Hedging and evasive language
- Bourgeois framing and ahistorical claims

## Writing Style

Following Chairman Mao's guidance in "Oppose Stereotyped Party Writing":
- **Vigorous, lively, fresh and forceful** - never drab or stereotyped
- **Audience-aware** - "When shooting an arrow, one must aim at the target"
- **Investigation-based** - "No investigation, no right to speak"
- **Clear positions** - FOR and AGAINST, using scientific argument

## Usage

### Download and Convert to GGUF

```bash
# Clone the repo
git lfs install
git clone https://huggingface.co/percyraskova/MLMLML
cd MLMLML

# Convert to GGUF (requires llama.cpp)
python ~/llama.cpp/convert_hf_to_gguf.py . --outfile MLMLML-F16.gguf --outtype f16

# Quantize to Q4_K_M
~/llama.cpp/build/bin/llama-quantize MLMLML-F16.gguf MLMLML-Q4_K_M.gguf Q4_K_M

# Create Ollama model
ollama create mlmlml -f Modelfile
ollama run mlmlml
```

### Direct with Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("percyraskova/MLMLML")
tokenizer = AutoTokenizer.from_pretrained("percyraskova/MLMLML")

inputs = tokenizer("What is imperialism?", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))
```

## Training Details

- **Base model**: unsloth/DeepSeek-R1-0528-Qwen3-8B
- **Method**: GRPO (Group Relative Policy Optimization)
- **Dataset**: ProleWiki Q&A pairs (~4500 samples)
- **Epochs**: 2
- **Hardware**: NVIDIA A100 80GB

## Limitations

This model is designed for educational purposes about Marxist-Leninist theory and analysis.
It takes clear ideological positions and is not intended to be "neutral" on class struggle,
imperialism, or other questions where Marxism-Leninism has definite answers.

## License

Apache 2.0

## Citation

If you use this model, please cite ProleWiki as the source of training data.