Eden-L3.3-70b-0.1

This is a merge of pre-trained language models created using mergekit.

For now, quant of this model can be found here: https://huggingface.co/bruhzair/prototype-0.4x197-Q6_K

Experimental and largely untested. Goal of the model is to be decent at creative writing/storytelling while also being smart. Was surprised to see its ability to flesh out a story with minimal information provided. Not sure how to explain it, but its definitely showing its smarts in terms of keeping track of small details, which can be a big plus foreshadowing. It can show some slop, but DRY and XTC helps a lot when enabled. It is not completely uncensored, but has shown some ability to produce extreme content at lower temps (.9 - .99). At higher temps (1.1) it will politley refuse providing info. I've had some success working around this by modifying the response to remove the refusal.

Llama 3 template

Recommended settings to start:

Temp: 1.0
Min P: 0.015

Merge Details

Merge Method

This model was merged using the Karcher Mean merge method.

Models Merged

The following models were included in the merge:

• Sao10K--L3.3-70B-Euryale-v2.3
• tdrussell--Llama-3-70B-Instruct-Storywriter
• nvidia--Llama-3.1-Nemotron-70B-Instruct-HF
• deepcogito--cogito-v1-preview-llama-70B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: /workspace/cache/models--nvidia--Llama-3.1-Nemotron-70B-Instruct-HF/snapshots/031d4042f36adc1a52cca51b331d25cbe3cf1022
  - model: /workspace/cache/models--deepcogito--cogito-v1-preview-llama-70B/snapshots/1d624e2293b5b35f9cfd2349f8e02c7ebf32ca83
  - model: /workspace/cache/models--tdrussell--Llama-3-70B-Instruct-Storywriter/snapshots/19be2a7c6382a9150e126cf144e2b2964e700d3c
  - model: /workspace/cache/models--Sao10K--L3.3-70B-Euryale-v2.3/snapshots/e5737724a37ae00926e95acf663ca73d430dc8ad
merge_method: karcher
parameters:
  max_iter: 10000
  tol: 1e-6
tokenizer:
  source: /workspace/cache/models--nvidia--Llama-3.1-Nemotron-70B-Instruct-HF/snapshots/031d4042f36adc1a52cca51b331d25cbe3cf1022
int8_mask: true
dtype: bfloat16