Emma-6 (BETA)

Emma-6 is an open-source Italian Large Language Model developed by Egomnia S.p.A.

Emma-6 is an Italian-first, general-purpose conversational language model designed with a focus on Italian language understanding and generation, technological sovereignty, experimentation, and research.

Emma-6 is an experimental model and is not intended to compete with frontier-scale proprietary models. Its primary objective is to contribute to the development of an Italian AI ecosystem and to provide an openly available Italian language model that can be studied, evaluated, and further developed.

Model Details

Model name: Emma-6
Organization: Egomnia S.p.A.
Language: Italian
Architecture: Decoder-only Transformer
Parameters: approximately 1.55 billion
Context length: 4,096 tokens
Vocabulary size: 64,000 tokens
Attention: Grouped Query Attention (GQA)
Normalization: RMSNorm
Activation: SwiGLU
Position encoding: RoPE
Embeddings: Tied input/output embeddings
Dropout: 0
RoPE theta: 500,000

Architecture

Emma-6 uses a decoder-only Transformer architecture with the following configuration:

Component	Configuration
Hidden size	2,048
Transformer layers	32
Attention heads	32
Key/Value heads	8
Head dimension	64
Attention type	Grouped Query Attention
Feed-forward activation	SwiGLU
Feed-forward multiplier	8/3
Context length	4,096
Vocabulary	64,000
Normalization	RMSNorm
Position encoding	RoPE
Embeddings	Tied
Dropout	0

The use of 8 key/value heads with 32 attention heads provides a 4:1 GQA configuration, reducing the memory and computational requirements of attention while retaining multiple query heads.

Tokenizer

Emma-6 uses a SentencePiece BPE tokenizer with byte fallback.

Vocabulary size: 64,000
Tokenizer type: SentencePiece BPE
Byte fallback: enabled
Special token IDs are explicitly defined in the model configuration.

The tokenizer was trained on a large sample of the cleaned Italian pretraining corpus.

Training Data

Emma-6 was pretrained on a large-scale Italian text corpus.

The training pipeline was designed around approximately:

100 GB of raw text
Approximately 25 billion tokens after preprocessing and tokenization

The corpus was processed through a dedicated cleaning pipeline before training.

The preprocessing pipeline includes:

HTML removal
Unicode NFKC normalization
URL removal
Whitespace normalization
Sentence splitting
Exact deduplication
64-bit Blake2b hashing for duplicate detection

The goal of the preprocessing stage was to reduce noise and duplication while preserving useful Italian linguistic content.

Pretraining

Emma-6 was pretrained using autoregressive next-token prediction.

The training configuration includes:

90,000 training steps
Sequence length: 4,096 tokens
bfloat16 training
FSDP ZeRO-2
Gradient checkpointing
Gradient accumulation
Cosine learning-rate schedule
Linear warmup
AdamW-based optimization
Weight decay: 0.1
Gradient clipping
2 × NVIDIA H200 GPUs

The effective training configuration corresponds to approximately 1.57 million tokens per optimization step, with approximately 141.6 billion tokens processed during the complete pretraining schedule.

This corresponds to multiple passes over the approximately 25-billion-token training corpus.

Supervised Fine-Tuning

After pretraining, Emma-6 was further trained using Supervised Fine-Tuning (SFT) on curated prompt/response examples.

The SFT stage is designed to improve:

Instruction following
Conversational behavior
Response quality
Italian-language generation
General usefulness
Compliance with the intended assistant behavior

The SFT pipeline uses loss masking, so the model is primarily optimized on the response portion rather than being trained to reproduce the input prompt.

The training configuration was designed for approximately 120,000 prompt/response examples and multiple training epochs.

Direct Preference Optimization

Following supervised fine-tuning, Emma-6 was further aligned using Direct Preference Optimization (DPO).

The DPO stage uses preference data containing alternative responses to the same prompt, distinguishing preferred responses from rejected responses.

DPO was used to further improve the behavior of the final model, including:

Instruction following
Helpfulness
Response quality
Conversational coherence
Preference alignment
Reduction of undesirable response patterns

The final Emma-6 model therefore incorporates both SFT and DPO alignment stages following large-scale pretraining.

Training Pipeline

The overall Emma-6 training process can be summarized as:

Large-scale Italian corpus
          ↓
Corpus cleaning and deduplication
          ↓
SentencePiece BPE tokenizer
          ↓
Autoregressive pretraining
          ↓
Supervised Fine-Tuning (SFT)
          ↓
Direct Preference Optimization (DPO)
          ↓
Emma-6

Training Infrastructure

The training pipeline was designed for multi-GPU execution and large-scale model training.

Reference hardware includes:

1 × NVIDIA H200 GPUs
141 GB GPU memory per GPU
282 GB total GPU memory
512 GB system RAM
Approximately 1.5 TB NVMe storage
AMD EPYC 9555 64-Core CPU

The training implementation supports Flash Attention where available, with a PyTorch attention fallback.

Export

Emma-6 can be exported in multiple formats.

The production export includes:

PyTorch model weights
ONNX model
SentencePiece tokenizer
Model configuration
Full training/configuration metadata

The ONNX export uses:

ONNX opset 21
FP32
External Data Format for large model weights

An INT8 dynamically quantized ONNX version can also be produced using ONNX Runtime.

Intended Use

Emma-6 is intended for:

Italian-language research
AI and NLP experimentation
Educational purposes
Model evaluation
Conversational AI experiments
Local and private inference
Development of Italian AI applications

Emma-6 can also be used as a research foundation for applications requiring an Italian-focused language model.

Out-of-Scope Use

Emma-6 is not designed to be used as:

A replacement for professional medical, legal, or financial advice
A fully autonomous decision-making system
A guaranteed factual knowledge source
A safety-critical control system
A system for making high-impact decisions about individuals without appropriate human oversight

Users should independently verify important information generated by the model.

Limitations

As an approximately 1.55B-parameter model, Emma-6 has significantly fewer parameters and computational resources than frontier-scale language models.

Potential limitations include:

Hallucinations
Factual inaccuracies
Limited reasoning capabilities on highly complex tasks
Variable performance on specialized domains
Potentially inconsistent responses
Sensitivity to prompt formulation
Biases inherited from training data
Reduced performance outside Italian compared with models specifically optimized for multilingual use

Emma-6 should therefore be evaluated according to the specific application in which it is deployed.

Privacy and Data Considerations

Users should avoid providing personal, confidential, sensitive, or proprietary information unless the specific deployment environment provides appropriate safeguards.

Organizations deploying Emma-6 should implement appropriate privacy, security, logging, retention, and access-control policies according to their use case and applicable legislation.

Responsible Use

Emma-6 is released as an open-source research and development project.

Users and developers are responsible for ensuring that their use of the model complies with applicable laws, regulations, platform policies, and ethical requirements.

Human oversight should be maintained for applications where incorrect or harmful outputs could have significant consequences.

License

Emma-6 is released under the OpenRAIL license.

Please refer to the license terms distributed with the model before using, modifying, or redistributing the model.

About Egomnia

Emma-6 is developed by Egomnia S.p.A., an Italian technology company working in AI, ICT, software development, cybersecurity, and digital innovation.

Egomnia is developing Emma as part of its broader research and development activities in Italian artificial intelligence and technological sovereignty.

Citation

If you use Emma-6 in research, experimentation, or derivative projects, please cite the project as:

@misc{egomnia_emma6,
  title        = {Emma-6},
  author       = {Egomnia S.p.A.},
  year         = {2026},
  organization = {Egomnia S.p.A.},
  language     = {Italian}
}

Acknowledgements

Emma-6 was developed by the Egomnia S.p.A. team as part of its research and development activities in artificial intelligence.

The project aims to contribute to an open and increasingly independent Italian AI ecosystem.

Downloads last month: 116

Model tree for egomnia/emma-6

Finetunes

1 model

Quantizations