README.md

---
license: other
license_name: deepseek-license
license_link: https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL
base_model: deepseek-ai/deepseek-coder-6.7b-instruct
tags:
  - minecraft
  - forge
  - modding
  - code-generation
  - lora
  - peft
  - sft
  - trl
language:
  - en
  - code
library_name: peft
pipeline_tag: text-generation
datasets:
  - custom
model-index:
  - name: forge-coder-v1.21.11
    results: []
---

<div align="center">
  <h1>🔨 Forge Coder v1.21.11</h1>
  <p><strong>A Specialized Code Generation Model for Minecraft Forge Mod Development</strong></p>
  <p>
    <a href="#quickstart">Quick Start</a> •
    <a href="#capabilities">Capabilities</a> •
    <a href="#examples">Examples</a> •
    <a href="#training">Training Details</a>
  </p>
</div>

---

## Overview

**Forge Coder** is a fine-tuned large language model specifically designed to assist developers in creating Minecraft Forge mods. Built on top of DeepSeek Coder 6.7B, this model has been trained on extensive Forge mod source code and documentation to provide accurate, idiomatic, and up-to-date code generation for Minecraft modding.

### Key Features

- 🎯 **Specialized Knowledge**: Deep understanding of Forge API, registry systems, and modding patterns
- 🔄 **Version-Aligned**: Trained specifically for Minecraft 1.21.11 and Forge 1.21.11
- 💡 **Code Completion**: Generate complete mod components from natural language descriptions
- 📚 **Best Practices**: Follows modern Forge modding conventions and patterns

## Model Details

| Property | Value |
|----------|-------|
| **Base Model** | [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) |
| **Fine-tuning Method** | QLoRA (4-bit quantization + LoRA) |
| **LoRA Rank** | 64 |
| **LoRA Alpha** | 128 |
| **Trainable Parameters** | 159.9M (2.3% of 6.7B) |
| **Target Forge Version** | 1.21.11 |
| **Target Minecraft Version** | 1.21.11 |
| **MCP Mappings** | 20251209.095502 |

## Quickstart

### Installation

```bash
pip install transformers peft accelerate bitsandbytes torch
```

### Basic Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Quantization config for efficient inference
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

# Load model
base_model_id = "deepseek-ai/deepseek-coder-6.7b-instruct"
adapter_id = "hwding/forge-coder-v1.21.11"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_id)

# Generate code
def generate_forge_code(prompt: str, max_tokens: int = 1024) -> str:
    messages = f"""### System:
You are an expert Minecraft Forge mod developer for version 1.21.11. Write clean, efficient, and well-structured Java code.

### User:
{prompt}

### Assistant:
"""
    inputs = tokenizer(messages, return_tensors="pt").to(model.device)
    outputs = model.generate(
        **inputs,
        max_new_tokens=max_tokens,
        temperature=0.7,
        top_p=0.95,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id,
    )
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example
code = generate_forge_code("Create a custom ore block that drops experience when mined")
print(code)
```

## Capabilities

The model excels at generating code for various Forge mod components:

| Category | Examples |
|----------|----------|
| **Blocks** | Custom blocks, ore blocks, machine blocks, multi-blocks |
| **Items** | Tools, weapons, armor, food items, custom items |
| **Entities** | Mobs, projectiles, vehicles, tile entities |
| **GUI/Screens** | Container menus, custom screens, HUD elements |
| **Networking** | Packet handling, client-server sync |
| **World Generation** | Ore generation, structures, biomes, features |
| **Events** | Forge event handlers, lifecycle events |
| **Registries** | DeferredRegister patterns, registry objects |
| **Capabilities** | Custom capabilities, capability providers |
| **Recipes** | Custom recipe types, recipe serializers |
| **Data Generation** | Datagen for models, blockstates, loot tables, tags |
| **Rendering** | Block/item/entity renderers, shaders |

## Examples

### Creating a Custom Block

**Prompt:** "Create a copper lamp block that emits light when powered by redstone"

```java
public class CopperLampBlock extends Block {
    public static final BooleanProperty LIT = BlockStateProperties.LIT;
    
    public CopperLampBlock(Properties properties) {
        super(properties);
        this.registerDefaultState(this.stateDefinition.any().setValue(LIT, false));
    }
    
    @Override
    protected void createBlockStateDefinition(StateDefinition.Builder<Block, BlockState> builder) {
        builder.add(LIT);
    }
    
    @Override
    public void neighborChanged(BlockState state, Level level, BlockPos pos, 
                                Block block, BlockPos fromPos, boolean isMoving) {
        if (!level.isClientSide) {
            boolean powered = level.hasNeighborSignal(pos);
            if (powered != state.getValue(LIT)) {
                level.setBlock(pos, state.setValue(LIT, powered), 2);
            }
        }
    }
    
    @Override
    public int getLightEmission(BlockState state, BlockGetter level, BlockPos pos) {
        return state.getValue(LIT) ? 15 : 0;
    }
}
```

### Creating a Custom Item

**Prompt:** "Create a magic wand item that shoots fireballs on right click"

```java
public class MagicWandItem extends Item {
    public MagicWandItem(Properties properties) {
        super(properties.stacksTo(1).durability(100));
    }
    
    @Override
    public InteractionResultHolder<ItemStack> use(Level level, Player player, InteractionHand hand) {
        ItemStack stack = player.getItemInHand(hand);
        
        if (!level.isClientSide) {
            Vec3 look = player.getLookAngle();
            SmallFireball fireball = new SmallFireball(level, player, look.x, look.y, look.z);
            fireball.setPos(player.getX(), player.getEyeY() - 0.1, player.getZ());
            level.addFreshEntity(fireball);
            
            stack.hurtAndBreak(1, player, (p) -> p.broadcastBreakEvent(hand));
            player.getCooldowns().addCooldown(this, 20);
        }
        
        level.playSound(player, player.blockPosition(), SoundEvents.BLAZE_SHOOT, 
                        SoundSource.PLAYERS, 1.0F, 1.0F);
        
        return InteractionResultHolder.sidedSuccess(stack, level.isClientSide);
    }
}
```

## Training Data

### Data Sources

| Source | Description | Files |
|--------|-------------|-------|
| **Forge Source** | MinecraftForge core, ForgeGradle, EventBus, ModLauncher | ~1,200 |
| **Popular Mods** | 27 well-maintained open-source mod repositories | ~21,700 |
| **Documentation** | Official Forge docs and tutorials | 74 |

### Featured Mod Repositories

Training data includes code from highly-regarded mods:

- **Applied Energistics 2** - Storage & automation
- **Mekanism** - Tech & machinery
- **Create** - Mechanical contraptions
- **Botania** - Nature magic
- **Thermal Series** - Energy systems
- **Tinkers' Construct** - Tool crafting
- **Immersive Engineering** - Industrial machines
- **JustEnoughItems (JEI)** - Recipe viewing
- **TerraFirmaCraft** - Survival overhaul
- **The Twilight Forest** - Dimension mod
- **Quark** - Vanilla enhancements
- **RFTools** - RF-powered utilities
- And 15 more...

### Dataset Statistics

| Metric | Value |
|--------|-------|
| Total Java Files Processed | 22,916 |
| Training Samples | 13,936 |
| Validation Samples | 734 |
| Sample Types | Code completion, explanation, Q&A |

## Training

### Configuration

| Parameter | Value |
|-----------|-------|
| Epochs | 3 |
| Batch Size | 2 (per device) |
| Gradient Accumulation | 8 steps |
| Effective Batch Size | 128 |
| Learning Rate | 2e-4 |
| LR Scheduler | Cosine |
| Warmup Ratio | 3% |
| Max Sequence Length | 2,048 tokens |
| Precision | BF16 |
| Hardware | 8× NVIDIA H20 (96GB each) |

### Training Metrics

| Metric | Value |
|--------|-------|
| Training Duration | 9h 12m |
| Total Steps | 1,848 |
| Final Training Loss | 0.27 |
| Final Validation Loss | 0.325 |
| Token Accuracy | 92.5% |
| Eval Accuracy | 91.2% |

### Loss Curve

```
Epoch 1: 0.89 → 0.42
Epoch 2: 0.38 → 0.31
Epoch 3: 0.29 → 0.27
```

### Framework Versions

- PEFT: 0.18.0
- TRL: 0.26.1
- Transformers: 4.57.3
- PyTorch: 2.5.1+cu121
- Datasets: 4.4.1
- Tokenizers: 0.22.1

## Limitations

- **Version Specific**: Optimized for Forge 1.21.11; may produce outdated patterns for older versions
- **Java Only**: Does not generate Kotlin, Gradle scripts, or JSON resources
- **No Runtime Testing**: Generated code should be tested before use in production
- **Context Window**: Limited to 2,048 tokens; very large classes may need to be split

## Intended Use

✅ **Recommended Uses:**
- Learning Forge modding patterns and best practices
- Rapid prototyping of mod components
- Code completion and suggestions
- Understanding Forge API usage

⚠️ **Not Recommended For:**
- Production code without review
- Security-critical applications
- Forge versions significantly different from 1.21.11

## Citation

If you use this model in your research or projects, please cite:

```bibtex
@misc{forge-coder-2024,
  author = {hwding},
  title = {Forge Coder: A Specialized Code Generation Model for Minecraft Forge Mod Development},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/hwding/forge-coder-v1.21.11}
}
```

Cite TRL as:

```bibtex
@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}
```

## License

This model is released under the [DeepSeek License](https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL), consistent with the base model.

Training data was sourced from open-source repositories under various permissive licenses (MIT, Apache 2.0, LGPL, etc.).

## Acknowledgments

- [DeepSeek](https://github.com/deepseek-ai) for the excellent base model
- [MinecraftForge](https://github.com/MinecraftForge) team for the modding framework
- All open-source mod developers whose code made this training possible

---

<div align="center">
  <p><strong>Happy Modding! 🎮⛏️</strong></p>
</div>