Frontis-MA1-30B

An execution-grounded model for evolutionary machine learning engineering and AutoResearch

📄 Paper  •  🌐 Project  •  💻 Code  •  🤗 Models  •  🧩 Tasks  •  📚 SFT Traces

Introduction

Frontis-MA1-30B is the companion 30B open-weight research model released with OpenMLE, an execution-grounded system for studying meta-evolution in machine learning engineering (MLE).

Starting from Qwen3-30B-A3B-Thinking-2507, we apply execution-grounded supervised fine-tuning and reinforcement learning so that the model learns four reusable program-transformation operators:

• Draft — create a complete initial solution;
• Improve — refine a parent program using its score and execution evidence;
• Debug — repair invalid or failing code;
• Crossover — recombine useful elements from two parent solutions.

At inference time, OpenMLE-Evo composes these operators into a long-horizon search loop. Frontis-MA1 is therefore both a product of the OpenMLE training stack and the model that drives its evolutionary search.

Highlights

• Execution-grounded post-training. SFT examples and RL rewards are constructed from programs executed in isolated MLE sandboxes.
• Trainable evolutionary operators. Draft, Improve, Debug, and Crossover provide a shared action space across training and test-time search.
• Linked research release. The model card connects the weights to the OpenMLE training code, search protocol, task environments, and evaluation boundary.
• Strong controlled model gain. Under the same OpenMLE-Evo harness on MLE-Bench Lite, Frontis-MA1-30B improves Medal Average from 34.85% to 53.03% and Human Rank from 0.5573 to 0.7055 over its base model.

The reported scores measure a model–harness system, not standalone one-shot generation. Reproduction requires the same OpenMLE-Evo search configuration, task environments, sandbox budget, and evaluation protocol.

Model family

Model	Base model	Release format	Status
Frontis-MA1-30B	Qwen3-30B-A3B-Thinking-2507	BF16 Transformers	This repository
Frontis-MA1-30B-GGUF	Frontis-MA1-30B	Q4_K_M GGUF	Local deployment
Frontis-MA1-35B	Qwen3.6-35B-A3B	BF16 Transformers	Companion release

Model details

Field	Value
Model family	Frontis-MA1
Architecture	Mixture-of-Experts causal language model
Parameters	30.5B total, 3.3B activated
Layers	48
Experts	128 total, 8 activated per token
Weight precision	BF16
Native configuration context	262,144 tokens
Post-training SFT cutoff	32,768 tokens
License	CC BY-NC 4.0

The 262,144-token context length is inherited from the base-model configuration. OpenMLE post-training used a 32,768-token SFT cutoff; substantially longer contexts have not yet been validated to the same standard as the reported experiments.

Training

Frontis-MA1-30B uses full-parameter supervised fine-tuning followed by reinforcement learning from executable task feedback.

Supervised fine-tuning

• 26,259 released execution-grounded examples: 17,245 full responses and 9,014 trajectory steps;
• full-solution and trajectory-step supervision;
• BF16 full-parameter training on 8 NVIDIA H200 GPUs;
• global batch size 128;
• learning rate 3e-5 with cosine decay and 0.1 warmup fraction;
• three epochs;
• Qwen3-compatible thinking-loss masking.

Reinforcement learning

• online generation and evaluation in isolated task sandboxes;
• operator mixture: Draft 0.50, Improve 0.17, Debug 0.17, Crossover 0.16;
• 16 prompts per rollout and 16 samples per prompt;
• maximum response length 24,576 tokens;
• GSPO with execution-grounded reward post-processing;
• Adam optimizer with learning rate 1e-6.

See the paper and OpenRSI code for the complete data-construction, training, search, and evaluation protocol.

_{Paper framework figure. Frontis-MA1 learns the same four atomic operators used by OpenMLE-Evo, first from executable SFT rollouts and then from online RL execution feedback.}

Benchmark results

The headline numbers are model–harness system results, not standalone one-shot scores. The controlled comparison holds the OpenMLE-Evo harness fixed and changes only the model from the Qwen base to Frontis-MA1-30B.

MLE-Bench Lite protocol

Item	Setting
Benchmark	Official 22-task MLE-Bench Lite split
Independent runs	3 per OpenMLE-Evo configuration, unless stated otherwise
Sandbox budget	12 hours on 0.5 NVIDIA RTX 4090 equivalent per task
Equivalent compute	6 RTX 4090 GPU-hours per task
Execution	Generated programs are run and scored in isolated task sandboxes
OpenMLE-Evo-Max	MLE-Bench-disjoint priors plus asynchronous multi-GPU search
Budget comparison	OpenMLE-Evo-Max keeps the same total sandbox-compute budget

The paper reports:

• Valid Rate: mean number of the 22 tasks with a valid submission;
• Medal Average: mean fraction of tasks receiving any Kaggle medal;
• Human Rank: mean fraction of human leaderboard participants surpassed by the selected submission.

All three metrics are higher-is-better.

Main results from the paper

_{Current paper main-results figure. The left panel shows all completed model–harness results on MLE-Bench Lite; the right panel compares model size with Medal Average. Orange denotes Frontis-MA1 with OpenMLE-Evo, cyan denotes other models with OpenMLE-Evo, hatching denotes OpenMLE-Evo-Max, and gray denotes general-purpose coding harnesses. The paper highlights the primary 35B model in this figure and reports the companion 30B results in its main table.}

Controlled 30B comparison

Model	Harness	Valid Rate ↑	Medal Average ↑	Human Rank ↑
Qwen3-30B-A3B-Thinking-2507	OpenMLE-Evo	17.33/22	34.85%	0.5573
Frontis-MA1-30B	OpenMLE-Evo	21.67/22	53.03%	0.7055
Frontis-MA1-30B	OpenMLE-Evo-Max	22.00/22	66.67%	0.8053

Comparison	Δ Valid Rate	Δ Medal Average	Δ Human Rank	What it measures
Frontis-MA1-30B vs. Qwen base, OpenMLE-Evo fixed	+4.34 tasks	+18.18 pp	+0.1482	Post-training gain under a fixed harness
OpenMLE-Evo-Max vs. OpenMLE-Evo, Frontis-MA1-30B fixed	+0.33 task	+13.64 pp	+0.0998	Additional system-level search gain

The fixed-harness comparison is the primary evidence for model improvement. OpenMLE-Evo-Max also changes the search system, so its result must be read as an end-to-end model–harness score rather than a pure model gain. These experiments do not establish a standalone ranking outside the OpenMLE evaluation protocol.

The paper does not provide a 30B-only result figure, so this model card keeps the original paper values as a searchable table rather than introducing a derived visualization.

Public-weight model comparison under OpenMLE-Evo

The following rows use the same OpenMLE-Evo harness, benchmark split, and sandbox-compute budget. This is the closest available comparison with strong public-weight models, although model scale, pretraining data, and architecture still differ.

Public-weight model	Valid Rate ↑	Medal Average ↑	Human Rank ↑
Kimi K2.6	21.67/22	66.67%	0.7859
GLM-5.2	19.67/22	62.12%	0.7069
Frontis-MA1-35B	21.67/22	60.61%	0.7647
MiniMax M3	22.00/22	59.09%	0.7994
Frontis-MA1-30B	21.67/22	53.03%	0.7055
DeepSeek-V4-Flash	21.33/22	51.52%	0.6957
Qwen3.6-35B-A3B	19.67/22	39.39%	0.5828
Qwen3-30B-A3B-Thinking-2507	17.33/22	34.85%	0.5573

These are model–harness results rather than standalone one-shot model scores. The full paper figure above additionally includes OpenMLE-Evo-Max and general-purpose coding-agent systems, whose harnesses are not directly interchangeable with this fixed-harness table.

Usage

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "FrontisAI/Frontis-MA1-30B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": (
            "Build a strong tabular classification baseline and explain "
            "the validation design."
        ),
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=2048)
completion = outputs[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(completion, skip_special_tokens=True))

vLLM

vllm serve FrontisAI/Frontis-MA1-30B \
  --served-model-name Frontis-MA1-30B \
  --tensor-parallel-size 8 \
  --max-model-len 32768 \
  --enable-reasoning \
  --reasoning-parser deepseek_r1

Intended use

Frontis-MA1-30B is intended for:

• research on machine learning engineering and AutoResearch agents;
• generation and iterative improvement of ML experiment code;
• execution-grounded program search;
• use with OpenMLE-Evo or compatible sandboxed harnesses.

Generated code may be incorrect, insecure, destructive, or expensive to execute. Run it inside an isolated environment with explicit CPU, memory, GPU, network, filesystem, and time limits.

Limitations

• The main evaluation centers on MLE-Bench Lite and does not establish general performance across all software-engineering or scientific-research tasks.
• The paper results depend on an external evolutionary search harness and sandbox feedback.
• Execution reward does not fully measure maintainability, robustness, security, or responsible data handling.
• Context lengths beyond the post-training cutoff require additional validation.
• Outputs may contain reasoning traces and should be handled according to downstream requirements.

Release artifacts

This repository is the canonical BF16 Transformers release. The Frontis-MA1-30B-GGUF repository provides the Q4_K_M local-deployment derivative.

Paper and citation

Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering

Preprint: arXiv:2607.28568.

License

Original Frontis-MA1 material is released under CC BY-NC 4.0 for attribution-required, non-commercial use. Commercial use is not granted.

The upstream Qwen Apache License 2.0 notice is preserved in LICENSE-UPSTREAM-APACHE-2.0 and NOTICE.

The license in this repository applies to the model weights, configuration files, and model documentation distributed here. Training and evaluation datasets and third-party software remain subject to their respective terms.

Acknowledgements

We thank the Qwen team and the open-source communities behind Transformers, SLIME, Ray, Megatron-LM, SGLang, MLE-Bench, and the broader executable MLE research ecosystem.