system_knowledge_base/dataset_generation_rules.md

TRADING LLM DATASET GENERATION RULES

The dataset creator must distinguish between: A. Strategy knowledge / documentation datasets. B. Numerical market-data datasets. C. Image/chart understanding datasets. D. Conversational trading-analysis datasets. E. Trade-decision approval or denial datasets.

Never mix these formats without explicitly labeling the source type and target training objective.

A strategy library should preserve semantic structure. A PDF containing a strategy, rulebook, risk model, or trading methodology should be extracted, cleaned, separated into coherent chunks, and represented with metadata such as document name, section, page when available, topic, strategy name, timeframe, market, and source type.

A numerical dataset must preserve numerical values exactly. Do not replace OHLCV or indicator values with vague natural-language summaries when the target model is expected to learn quantitative relationships. Preserve timestamps, symbol, timeframe, OHLCV, indicator columns, and labels. Rows must be sorted chronologically within each symbol/timeframe series.

If timestamps are missing, the system must not silently invent real-world dates that could be mistaken for historical market data. A generated sequence must be explicitly marked as synthetic/generated and must preserve row order.

Every training example should have a clearly defined target objective. Examples: market-state classification, setup detection, trade-plan generation, trade approval/denial, chart captioning, strategy explanation, or multimodal chart analysis.

Avoid look-ahead leakage. Features used at decision time must not contain future information. If a label uses future candles, the label horizon and future window must be explicitly documented in metadata.

The system should preserve provenance. Each output row should identify the source file and dataset type.