README.md

metadata

dataset_info:
  - config_name: dpo
    features:
      - name: prompt
        dtype: string
      - name: chosen
        dtype: string
      - name: rejected
        dtype: string
    splits:
      - name: train
        num_bytes: 1296049
        num_examples: 10036
      - name: validation
        num_bytes: 268970
        num_examples: 2002
    download_size: 683822
    dataset_size: 1565019
  - config_name: sft
    features:
      - name: instruction
        dtype: string
      - name: input
        dtype: string
      - name: output
        struct:
          - name: action
            dtype: string
          - name: reasoning
            dtype: string
          - name: facets
            list: string
          - name: response
            dtype: string
    splits:
      - name: train
        num_bytes: 3372697
        num_examples: 10036
      - name: validation
        num_bytes: 688146
        num_examples: 2002
    download_size: 1574648
    dataset_size: 4060843
configs:
  - config_name: dpo
    data_files:
      - split: train
        path: dpo/train-*
      - split: validation
        path: dpo/validation-*
  - config_name: sft
    data_files:
      - split: train
        path: sft/train-*
      - split: validation
        path: sft/validation-*

AskBeforeAnswer Dataset

This dataset contains the training and validation splits for the AskBeforeAnswer clarification-seeking model.

GitHub Release: v0.0.4

Subsets (Configurations)

This repository contains two subsets which must be loaded separately depending on the training stage:

1. sft (Supervised Fine-Tuning)

Contains the structured JSON responses for initial alignment.

• Features: instruction, input, output (JSON dict containing action, reasoning, facets, response)

from datasets import load_dataset
sft_dataset = load_dataset("chrisjcc/ask-before-answer-data", "sft")

2. dpo (Direct Preference Optimization)

Contains the preference pairs used to penalize hallucinations.

• Features: prompt, chosen, rejected

from datasets import load_dataset
dpo_dataset = load_dataset("chrisjcc/ask-before-answer-data", "dpo")