UMCU commited on
Commit
f29ace4
·
verified ·
1 Parent(s): 15189ee

Update README.md

Browse files

Llama-3.2-1B-Instruct, with domain adapted pretraining (DAPT), also called Continuous Pre-training (CPT) on a Dutch medical corpus.

Training for one full epoch, with a 256 batch size, maximally 768 sequence length and a linear-cosine schedule (details follow..).

This model will be further pre-trained on 5 million cardiology records from the UMCU.

Files changed (1) hide show
  1. README.md +12 -3
README.md CHANGED
@@ -1,3 +1,12 @@
1
- ---
2
- license: llama3.2
3
- ---
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: llama3.2
3
+ datasets:
4
+ - UMCU/DutchMedicalText
5
+ language:
6
+ - nl
7
+ base_model:
8
+ - meta-llama/Llama-3.2-1B-Instruct
9
+ tags:
10
+ - medical
11
+ - cardiology
12
+ ---