AdaptLLM commited on
Commit
898e89b
·
verified ·
1 Parent(s): daefbe9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -4,7 +4,7 @@ language:
4
  ---
5
  # Adapting Multimodal Large Language Models to Domains via Post-Training
6
 
7
- This repository provides an implementation preview of our paper, **On Domain-Specific Post-Training for Multimodal Large Language Models**.
8
 
9
  We investigate domain adaptation of MLLMs through post-training, focusing on data synthesis, training pipelines, and task evaluation.
10
  **(1) Data Synthesis**: Using open-source models, we develop a visual instruction synthesizer that effectively generates diverse visual instruction tasks from domain-specific image-caption pairs. **Our synthetic tasks surpass those generated by manual rules, GPT-4, and GPT-4V in enhancing the domain-specific performance of MLLMs.**
@@ -37,7 +37,7 @@ AdaMLLM represents our latest advancement in building domain-specific foundation
37
  - **[AdaptLLM](https://huggingface.co/papers/2309.09530): Adapt LLM to domains**
38
  We employ rule-based methods to extract tasks from domain-specific corpora, reformatting them into reading comprehension tasks for continued pre-training. Our 7B finance model outperforms domain-specific models of much larger scales, such as BloombergGPT-50B.
39
 
40
- - **AdaMLLM: Adapt Multimodal LLM to domains**
41
  We extend supervised task synthesis to multimodality, introducing a unified visual instruction synthesizer to extract instruction-response pairs from domain-specific image-caption pairs. Our synthetic tasks outperform those generated by manual rules, GPT-4, and GPT-4V in improving domain-specific performance for MLLMs.
42
 
43
 
 
4
  ---
5
  # Adapting Multimodal Large Language Models to Domains via Post-Training
6
 
7
+ This repository provides an implementation preview of our paper: [On Domain-Specific Post-Training for Multimodal Large Language Models](https://huggingface.co/papers/2411.19930).
8
 
9
  We investigate domain adaptation of MLLMs through post-training, focusing on data synthesis, training pipelines, and task evaluation.
10
  **(1) Data Synthesis**: Using open-source models, we develop a visual instruction synthesizer that effectively generates diverse visual instruction tasks from domain-specific image-caption pairs. **Our synthetic tasks surpass those generated by manual rules, GPT-4, and GPT-4V in enhancing the domain-specific performance of MLLMs.**
 
37
  - **[AdaptLLM](https://huggingface.co/papers/2309.09530): Adapt LLM to domains**
38
  We employ rule-based methods to extract tasks from domain-specific corpora, reformatting them into reading comprehension tasks for continued pre-training. Our 7B finance model outperforms domain-specific models of much larger scales, such as BloombergGPT-50B.
39
 
40
+ - **[AdaMLLM](https://huggingface.co/papers/2411.19930): Adapt Multimodal LLM to domains**
41
  We extend supervised task synthesis to multimodality, introducing a unified visual instruction synthesizer to extract instruction-response pairs from domain-specific image-caption pairs. Our synthetic tasks outperform those generated by manual rules, GPT-4, and GPT-4V in improving domain-specific performance for MLLMs.
42
 
43