system_knowledge_base/dataset_generation_rules.md

TRADING LLM DATASET GENERATION RULES

The dataset creator must distinguish between:
A. Strategy knowledge / documentation datasets.
B. Numerical market-data datasets.
C. Image/chart understanding datasets.
D. Conversational trading-analysis datasets.
E. Trade-decision approval or denial datasets.

Never mix these formats without explicitly labeling the source type and target
training objective.

A strategy library should preserve semantic structure. A PDF containing a
strategy, rulebook, risk model, or trading methodology should be extracted,
cleaned, separated into coherent chunks, and represented with metadata such as
document name, section, page when available, topic, strategy name, timeframe,
market, and source type.

A numerical dataset must preserve numerical values exactly. Do not replace
OHLCV or indicator values with vague natural-language summaries when the target
model is expected to learn quantitative relationships. Preserve timestamps,
symbol, timeframe, OHLCV, indicator columns, and labels. Rows must be sorted
chronologically within each symbol/timeframe series.

If timestamps are missing, the system must not silently invent real-world dates
that could be mistaken for historical market data. A generated sequence must be
explicitly marked as synthetic/generated and must preserve row order.

Every training example should have a clearly defined target objective. Examples:
market-state classification, setup detection, trade-plan generation, trade
approval/denial, chart captioning, strategy explanation, or multimodal chart
analysis.

Avoid look-ahead leakage. Features used at decision time must not contain future
information. If a label uses future candles, the label horizon and future window
must be explicitly documented in metadata.

The system should preserve provenance. Each output row should identify the
source file and dataset type.