Instructions to use SINAI/ALIA-es-cultural-heritage-7B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SINAI/ALIA-es-cultural-heritage-7B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SINAI/ALIA-es-cultural-heritage-7B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SINAI/ALIA-es-cultural-heritage-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("SINAI/ALIA-es-cultural-heritage-7B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SINAI/ALIA-es-cultural-heritage-7B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SINAI/ALIA-es-cultural-heritage-7B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SINAI/ALIA-es-cultural-heritage-7B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SINAI/ALIA-es-cultural-heritage-7B-Instruct

SGLang

How to use SINAI/ALIA-es-cultural-heritage-7B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SINAI/ALIA-es-cultural-heritage-7B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SINAI/ALIA-es-cultural-heritage-7B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SINAI/ALIA-es-cultural-heritage-7B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SINAI/ALIA-es-cultural-heritage-7B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SINAI/ALIA-es-cultural-heritage-7B-Instruct with Docker Model Runner:
```
docker model run hf.co/SINAI/ALIA-es-cultural-heritage-7B-Instruct
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

ALIA Spanish Cultural Heritage 7B Instruct Model

Model Description

ALIA Spanish Cultural Heritage 7B Instruct Model is an instruction-tuned language model specialized in the cultural heritage domain for Spanish. This model is derived from SINAI/ALIA-es-cultural-heritage-7B-Base, which itself is based on the Salamandra-7B model family.

The model has been instruction-tuned using the ALIA-es-cultural-heritage-synthetic-instructions dataset, enabling it to assist users with cultural heritage and historical queries in Spanish.

DISCLAIMER: This model is provided for research and educational purposes. It should not be used as a substitute for professional historical or heritage consultation. Users are responsible for ensuring their use of the model complies with applicable laws and regulations. As a result, it may generate harmful or inappropriate content, or factually inaccurate historical information. Users should verify any heritage information generated against official sources (such as IAPH). The SINAI Research Group and Barcelona Supercomputing Center shall not be held liable for any outcomes resulting from the use of this model.

Model Lineage

Salamandra-7B (BSC-LT)
    ↓
ALIA-es-cultural-heritage-7B-Base (SINAI)
    ↓
ALIA-es-cultural-heritage-7B-Instruct (SINAI) ← This model

Key Features

Specialized Domain: Cultural heritage Spanish language
Instruction-Following: Fine-tuned to respond to user queries and instructions
Foundation: Built upon Salamandra-7B's multilingual capabilities, focused on Spanish
Open License: Released under Apache 2.0 license

Model Details

Architecture

This model maintains the same architecture as its base model ALIA-es-cultural-heritage-7B-Base, which is derived from Salamandra-7B:


Base Model	Salamandra 7B
Total Parameters	7,768,117,248
Embedding Parameters	1,048,576,000
Layers	32
Hidden size	4,096
Attention heads	32
Context length	8,192
Vocabulary size	256,000
Precision	bfloat16
Embedding type	RoPE
Activation Function	SwiGLU
Layer normalization	RMS Norm
Flash attention	✅
Grouped Query Attention	✅
Num. query groups	8

Training Details

Instruction Tuning:

Training was conducted by SINAI Research Group (Universidad de Jaén)
Dataset: ALIA-es-cultural-heritage-synthetic-instructions
Language: Spanish
Domain: Cultural heritage
Number of samples: 272,873

Training Infrastructure:

Model SFT was trained on the CALENDULA supercomputer hosted and operated by SCAYLE (Supercomputación Castilla y León) within the framework of the ALIA project.

The Genoa partition used for SFT training has the following specifications per node:

4x NVIDIA GPUs
AMD EPYC processors
NCCL distributed backend

The table below specifies the node configuration used for the supervised fine-tuning SFT:

Phase	Nodes	GPUs	Training Time
SFT	2	8	~28h

Training Hyperparameters:

Supervised Fine-Tuning was conducted using the Axolotl framework.

Hyperparameter	Value
Learning rate	1.4e-5
Micro batch size	8
Gradient accumulation steps	2
Global batch size	128 (using 8 GPUs)
Epochs	2
LR Scheduler	Cosine
Warmup steps	500
Cosine min. LR ratio	0.10
NEFTune Noise Alpha	5
Sequence length	8,192
Optimizer	adamw_torch_fused
Adam beta1	0.9
Adam beta2	0.94
Adam epsilon	1e-8
Max grad norm	0.28
Weight decay	0.003

Intended Use

Direct Use

This model is designed to assist users with questions and tasks related to the cultural heritage and history domains in Spanish. It can be used for:

Answering cultural heritage and historical queries in Spanish
Summarizing heritage documents, archives, and historical texts
Designing virtual cultural routes and tours based on historical criteria
Supporting museum documentation and collection query workflows

How to Use

Inference

Basic Usage with Transformers

pip install transformers torch accelerate sentencepiece protobuf

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "SINAI/ALIA-es-cultural-heritage-7B-Instruct"

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16
)

# Example heritage query
messages = [
    {"role": "user", "content": "Resume la importancia histórica y arquitectónica del Castillo de Santa Catalina en Jaén."}
]

# Format the input
input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

# Generate response
outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Inference with vLLM

pip install vllm

from vllm import LLM, SamplingParams

model_id = "SINAI/ALIA-es-cultural-heritage-7B-Instruct"

# Create sampling parameters
sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.95,
    max_tokens=512
)

# Initialize the model
llm = LLM(model=model_id)

# Example prompt
prompt = "Describe las características principales del estilo mudéjar en la arquitectura andaluza."

# Generate response
outputs = llm.generate([prompt], sampling_params)

for output in outputs:
    print(f"Generated text: {output.outputs[0].text}")

Training Data

The model was instruction-tuned using the ALIA-es-cultural-heritage-synthetic-instructions dataset created by the SINAI research group.

The training dataset has a total of 272,873 instructions, desglosed as follows:

Spanish Heritage Domain: 139,351
General Spanish: 40,500
General English: 30,000
Original Salamandra Instructions: 63,022

Evaluation

The model was evaluated on 6 domain-specific tasks covering various aspects of cultural heritage in Spanish to assess its adaptation and generation quality.

Evaluation Datasets

Cultural Heritage Tasks

Source: andalucia.org
Description: 6 domain-specific tasks covering architectural styles (Architecture Style), conservation (Conservation & Protection), chronology (Historical Chronology), factual QA, typologies (Typology & Use), and true/false verification, autogenerated using informative and tourism texts about the Autonomous Community of Andalusia.
Domain: Spanish cultural heritage, history, geography, and regional tourism of Andalusia.

Performance Metrics

The following tables show the evaluation results comparing ALIA Spanish Cultural Heritage 7B Instruct (Finetuned) against the baseline Salamandra 7B Instruct (Original) model.

BLEU and ROUGE-1 Results

Heritage Task	BLEU (Original)	BLEU (Finetuned)	ROUGE-1 (Original)	ROUGE-1 (Finetuned)
Architecture Style	7.41	16.46	41.19%	49.73%
Conservation & Protection	13.57	20.57	48.23%	54.44%
Historical Chronology	15.36	22.41	49.84%	56.87%
Factual QA	46.19	25.24	74.87%	56.38%
Typology & Use	20.37	27.93	50.98%	55.41%
True/False QA	33.44	30.38	57.58%	58.87%

ROUGE-2 and ROUGE-L Results

Heritage Task	ROUGE-2 (Original)	ROUGE-2 (Finetuned)	ROUGE-L (Original)	ROUGE-L (Finetuned)
Architecture Style	25.87%	30.87%	31.28%	36.70%
Conservation & Protection	30.91%	35.49%	35.63%	39.84%
Historical Chronology	31.11%	36.44%	36.08%	41.28%
Factual QA	65.40%	47.82%	70.52%	51.75%
Typology & Use	32.39%	35.89%	38.93%	41.75%
True/False QA	47.72%	45.12%	53.71%	49.91%

Limitations and Biases

Known Limitations

Domain Specificity: While specialized in cultural heritage and history Spanish, the model may not perform optimally on general-purpose tasks.
Language: Optimized for Spanish only.
Not Professional Advice: Outputs must not be considered as expert architectural, historical, or legal heritage advice.
Training Data Constraints: Performance is limited by the scope and quality of the training data.
Potential Hallucinations: Like all language models, may generate plausible-sounding but incorrect or historically inaccurate information.

Bias Considerations

The model inherits potential biases from its base model (Salamandra-7B) and training data.
Historical data and heritage language may reflect traditional biases present in institutional or scientific literature.
Users should be aware of potential biases when using the model for sensitive applications.
We recommend additional bias testing and mitigation for specific use cases.

Safety and Responsible Use

Human Oversight: Always verify model outputs, especially for critical historical documentation.
Professional Consultation: Consult with qualified historians or heritage professionals for authoritative decisions.
Compliance: Ensure use complies with applicable laws and regulations regarding AI systems.
Privacy: Do not input sensitive personal or confidential information.

Additional Information

License

Apache License, Version 2.0

Citation

@misc{ALIA-es-cultural-heritage-7B-Instruct,
  title={ALIA Spanish Cultural Heritage 7B Instruct Model},
  author={SINAI Research Group, Universidad de Jaén},
  year={2026},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-7B-Instruct}}
}

Please also cite the base models:

@misc{ALIA-es-cultural-heritage-7B-Base,
  title={ALIA Spanish Cultural Heritage 7B Base Model},
  author={SINAI Research Group, Universidad de Jaén},
  year={2026},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-7B-Base}}
}

@misc{gonzalezagirre2025salamandratechnicalreport,
      title={Salamandra Technical Report}, 
      author={Aitor Gonzalez-Agirre and Marc Pàmies and Joan Llop and Irene Baucells and Severino Da Dalt and Daniel Tamayo and José Javier Saiz and Ferran Espuña and Jaume Prats and Javier Aula-Blasco and Mario Mina and Adrián Rubio and Alexander Shvets and Anna Sallés and Iñaki Lacunza and Iñigo Pikabea and Jorge Palomar and Júlia Falcão and Lucía Tormo and Luis Vasquez-Reina and Montserrat Marimon and Valle Ruíz-Fernández and Marta Villegas},
      year={2025},
      eprint={2502.08489},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.08489}, 
}

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project ALIA.

Acknowledgments

Training of this model was conducted thanks to SCAYLE (Supercomputación Castilla y León) on the CALENDULA supercomputer, within the framework of the ALIA project.

Contact: ALIA Project - SINAI Research Group - Universidad de Jaén

More Information: SINAI Research Group | ALIA-UJA Project