Front-loading study

updated 4 days ago

Four web datasets pretrained with and without MixtureVitae's instruction and reasoning subset, then extended to 16K and given Tulu3 SFT and DPO.

Upvote

ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096

2B • Updated Aug 20 • 6

Note MixtureVitae web part, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 20 • 14

Note MixtureVitae web part, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 21 • 11

Note MixtureVitae web part, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k

2B • Updated Aug 20 • 8

Note MixtureVitae web part, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 20 • 24

Note MixtureVitae web part, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 21 • 16

Note MixtureVitae web part, without front-loading · 16K, Tulu3 DPO
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated

Text Generation • 2B • Updated Jul 27 • 25

Note MixtureVitae web part, with front-loading · base
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 27 • 9

Note MixtureVitae web part, with front-loading · Tulu3 SFT
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 27 • 8

Note MixtureVitae web part, with front-loading · Tulu3 DPO
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k

Text Generation • 2B • Updated Jul 27 • 17

Note MixtureVitae web part, with front-loading · extended to 16K
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 27 • 15

Note MixtureVitae web part, with front-loading · 16K, Tulu3 SFT
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 27 • 13

Note MixtureVitae web part, with front-loading · 16K, Tulu3 DPO
ali-elganzory/1.7b-Comma0.1-300BT-WithChatTemplate

Text Generation • 2B • Updated Jul 28 • 10

Note Comma0.1, without front-loading · base
ali-elganzory/1.7b-Comma0.1-300BT-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 8

Note Comma0.1, without front-loading · Tulu3 SFT
ali-elganzory/1.7b-Comma0.1-300BT-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 7

Note Comma0.1, without front-loading · Tulu3 DPO
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k

Text Generation • 2B • Updated Apr 27 • 38

Note Comma0.1, without front-loading · extended to 16K
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Apr 13 • 29

Note Comma0.1, without front-loading · 16K, Tulu3 SFT
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Apr 13 • 32

Note Comma0.1, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096

2B • Updated 25 days ago • 22

Note Comma0.1, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated 27 days ago • 302

Note Comma0.1, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated 26 days ago • 52

Note Comma0.1, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k

2B • Updated 24 days ago • 44

Note Comma0.1, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated 24 days ago • 53

Note Comma0.1, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated 23 days ago • 33

Note Comma0.1, with front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096

Text Generation • 2B • Updated Jul 28 • 11

Note Nemotron-CC-HQ, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 18

Note Nemotron-CC-HQ, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 17

Note Nemotron-CC-HQ, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k

2B • Updated Aug 20 • 9

Note Nemotron-CC-HQ, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 20 • 19

Note Nemotron-CC-HQ, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 21 • 21

Note Nemotron-CC-HQ, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096

2B • Updated Aug 20 • 14

Note Nemotron-CC-HQ, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 20 • 13

Note Nemotron-CC-HQ, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 21 • 13

Note Nemotron-CC-HQ, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k

2B • Updated Aug 20 • 9

Note Nemotron-CC-HQ, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 20 • 33

Note Nemotron-CC-HQ, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 21 • 16

Note Nemotron-CC-HQ, with front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096

Text Generation • 2B • Updated Jul 28 • 13 • 1

Note FineWeb-Edu, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 16

Note FineWeb-Edu, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 13

Note FineWeb-Edu, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-4096-longsft_16k

Text Generation • 2B • Updated Apr 27 • 22 • 1

Note FineWeb-Edu, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-longsft_16k-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 32

Note FineWeb-Edu, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-longsft_16k-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Jul 28 • 23

Note FineWeb-Edu, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096

2B • Updated Aug 20 • 6

Note FineWeb-Edu, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 20 • 11

Note FineWeb-Edu, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated

Text Generation • 2B • Updated Aug 21 • 11

Note FineWeb-Edu, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k

2B • Updated Aug 20 • 7

Note FineWeb-Edu, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontam

Text Generation • 2B • Updated Aug 20 • 15

Note FineWeb-Edu, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontam

Text Generation • 2B • Updated Aug 21 • 11

Note FineWeb-Edu, with front-loading · 16K, Tulu3 DPO
ontocord/1.7b-Comma0.1-300BT

2B • Updated Apr 25 • 9

Note Comma0.1, without front-loading · base (language suite)
permissive-post-train/evaluation-records

Updated 3 days ago • 216

Note The evaluation records of every checkpoint here

Upvote