Diffusion Language Models are Super Data Learners
Paper ā¢ 2511.03276 ā¢ Published ā¢ 132
AI & ML interests
None defined yet.
dreamerdeoĀ
authored 2
papers 9 months ago
dreamerdeoĀ
authored 4
papers over 1 year ago
FlowReasoner: Reinforcing Query-Level Meta-Agents
Paper ā¢ 2504.15257 ā¢ Published ā¢ 47
NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation
Paper ā¢ 2504.13055 ā¢ Published ā¢ 19
SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types
Paper ā¢ 2412.11757 ā¢ Published
Efficient Process Reward Model Training via Active Learning
Paper ā¢ 2504.10559 ā¢ Published ā¢ 13
dreamerdeoĀ
posted an update over 1 year ago
Post
3008
š Excited to share our technical report on the Southeast Asian multilingual model Sailor2 and its latest updates!
Our 49-page report details Sailor2's development journey, including multilingual data cleaning, small model data mixture simulations, multi-stage continual pre-training, multi-stage post-training, and multi-cultural multi-lingual evaluations. Sailor2 aims to streamline the multilingual model pre-training process efficiently for the community.
š§ We highlight Sailor2's impressive performance in low-resource language translation scenarios and its cultural understanding advantages in Southeast Asia, promoting practical applications for regional languages.
Model updates include:Ā
š” More precise outputs: Reduced redundancy in model outputs through refined post-training data and optimization techniques.Ā
š Handling longer texts: Expanded to handle up to 128K context length in Southeast Asian languages through long-text training.Ā
ā”ļø Faster inference: Achieved 2.5x faster inference speed with speculative decoding.Ā
šŖļø More model sizes: Introduced new sizes of 3B and 14B through model pruning.
š All models are Apache-licensed for commercial use; development tools (code, resources) are open-source.
š Technical report: Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs (2502.12982)Ā
š¤ļø Models: sail/sailor2-language-models-674d7c9e6b4dbbd9a869906bĀ
š¬ Demo: sail/Sailor2-20B-ChatĀ
š£ Sailor2 community:
sailor2
Our 49-page report details Sailor2's development journey, including multilingual data cleaning, small model data mixture simulations, multi-stage continual pre-training, multi-stage post-training, and multi-cultural multi-lingual evaluations. Sailor2 aims to streamline the multilingual model pre-training process efficiently for the community.
š§ We highlight Sailor2's impressive performance in low-resource language translation scenarios and its cultural understanding advantages in Southeast Asia, promoting practical applications for regional languages.
Model updates include:Ā
š” More precise outputs: Reduced redundancy in model outputs through refined post-training data and optimization techniques.Ā
š Handling longer texts: Expanded to handle up to 128K context length in Southeast Asian languages through long-text training.Ā
ā”ļø Faster inference: Achieved 2.5x faster inference speed with speculative decoding.Ā
šŖļø More model sizes: Introduced new sizes of 3B and 14B through model pruning.
š All models are Apache-licensed for commercial use; development tools (code, resources) are open-source.
š Technical report: Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs (2502.12982)Ā
š¤ļø Models: sail/sailor2-language-models-674d7c9e6b4dbbd9a869906bĀ
š¬ Demo: sail/Sailor2-20B-ChatĀ
š£ Sailor2 community:
dreamerdeoĀ
authored 2
papers about 2 years ago
dreamerdeoĀ
authored 3
papers over 2 years ago
Sailor: Open Language Models for South-East Asia
Paper ā¢ 2404.03608 ā¢ Published ā¢ 21
From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning
Paper ā¢ 2304.07995 ā¢ Published ā¢ 3
MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing
Paper ā¢ 2212.13492 ā¢ Published ā¢ 2