4-bit examples looking good.
Try it with the glue on! (example/model is without LoRa.)
!!NSFW!! - Erotica Writing Example - !!NSFW!!
A 11b Mistral model, based on the NeverSleep recipe.
Recipe
slices
sources:
- model: NeverSleep/Noromaid-7b-v0.1.1
- layer_range: [0, 24]
sources:
- model: chargoddard/loyal-piano-m7
- layer_range: [8, 32]
merge_method: passthrough
dtype: bfloat16
- Downloads last month
- 9