---
language: en
tags:
- mlx
library_name: mlx
pipeline_tag: text-generation
base_model:
- talkie-lm/talkie-1930-13b-base
---

## Model

A lightly post-trained version of [talkie-lm/talkie-1930-13b-base](https://huggingface.co/talkie-lm/talkie-1930-13b-base) via some initial SFT on an elicited persona followed by KTO on Claude-preferred responses. Distinct from the official instruction tune in that it is instructed to play the character of an intelligent machine, tuned to have slightly more modern-day preferences (so it may adopt the views of a 1930s progressive), and finally differs in its chat template, which forgoes XML to instead present itself as a transcript/play.

Recommended sampling settings are `temp=0.5, min_p=0.05, top_k=40, repetition_penalty=1.2, repetition_context_size=128`. Like the base model, it has a max context size of 2048. It additionally retians the (limited) few shot learning ability of the base model - going from 7.73% GSM8K at 1-shot to 11.30% at 2-shot to 12.36% at 4-shot.

The seed prompts used for initial persona elicitationcan be found [here](https://gist.github.com/N8python/00fda0195d4923908f6b1f0bd7337208) - each 'seed' was then continued by base talkie into a multi-turn conversation, the best of which formed the initial SFT round.

## Safety Disclaimer

This model is ancient and safety training is minimal. It may generate objectionable content, hateful content, or flat-up wrong content. Thus it is strictly for experimental purposes and I strongly recommend against its use in any user-facing application.

## Chat template

This model uses a standard `user` / `assistant` chat API surface, but renders
messages into TALKIE's play transcript format:

```python
messages = [
    {"role": "user", "content": "How are you?"},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
```

This produces:

```text
The following conversation took place between the HUMAN, and TALKIE - a mechanical mind imbued with the knowledge of the world and the ability to use human language - a "thinking machine". It is published here for the benefit of the public:

HUMAN:

How are you?

TALKIE:

```

## Generation stops

Stop generation when TALKIE starts a new speaker turn. For direct MLX
generation, this model treats both `<|endoftext|>` (`65535`) and the `HUM`
token (`56180`) as EOS. That makes bare `mlx_lm.generate` and
`mlx_lm.batch_generate` stop when the model begins to write the next `HUMAN:`
turn.

The model package also includes `generation_config.json` with these stop
strings for runtimes that support text stops:

```json
[
  "\n\nHUMAN:",
  "\nHUMAN:",
  "HUMAN:",
  "\n\nTALKIE:",
  "\nTALKIE:",
  "TALKIE:"
]
```

When serving through `mlx_lm.server`, pass the same strings as the request
`stop` field.