trade-pool / baseline_eval.txt
tosi-n's picture
Upload folder using huggingface_hub
ce6b50a verified
Raw
History Blame Contribute Delete
19.2 kB
Resolved source: stimulir/trade-pool
Eval job_id: trade_pool_poolside_laguna_xs.2_20260530_112447_9949ef5b
eval_dataset is not set, falling back to train dataset
Processing 4 groups (4 total rollouts): 0%| | 0/4 [00:00<?, ?it/s, reward=?] Processing 4 groups (4 total rollouts): 25%|██▌ | 1/4 [00:07<00:23, 7.96s/it, reward=?] Processing 4 groups (4 total rollouts): 25%|██▌ | 1/4 [00:07<00:23, 7.96s/it, reward=0.915] Processing 4 groups (4 total rollouts): 50%|█████ | 2/4 [00:09<00:08, 4.22s/it, reward=0.915] Processing 4 groups (4 total rollouts): 50%|█████ | 2/4 [00:09<00:08, 4.22s/it, reward=0.696] Processing 4 groups (4 total rollouts): 75%|███████▌ | 3/4 [00:18<00:06, 6.22s/it, reward=0.696] Processing 4 groups (4 total rollouts): 75%|███████▌ | 3/4 [00:18<00:06, 6.22s/it, reward=0.464] Processing 4 groups (4 total rollouts): 100%|██████████| 4/4 [00:30<00:00, 8.74s/it, reward=0.464] Processing 4 groups (4 total rollouts): 100%|██████████| 4/4 [00:30<00:00, 8.74s/it, reward=0.495] Processing 4 groups (4 total rollouts): 100%|██████████| 4/4 [00:30<00:00, 7.70s/it, reward=0.495]
--- Evaluation ---
Environment: trade-pool
Model: poolside/laguna-xs.2
Provider: https://api.pinference.ai/api/v1
Examples: 4
Rollouts per example: 1
--- Example ---
╭─────────────────────────────────── Step 0 ───────────────────────────────────╮
│ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓ │
│ ┃ Prompt ┃ Completion ┃ Reward ┃ │
│ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩ │
│ │ system: You are a quantitative │ assistant: │ 0.91 │ │
│ │ trading strategy engineer. You │ [reasoning] │ │ │
│ │ write a Python function that │ │ │ │
│ │ decides a target position each │ Let me analyze the price data │ │ │
│ │ bar, using ONLY causal │ and develop a strategy that │ │ │
│ │ technical features. │ follows the given rules. │ │ │
│ │ │ │ │ │
│ │ You must output a single │ Looking at the recent 30 │ │ │
│ │ Python code block defining │ closes, I can see the price │ │ │
│ │ exactly: │ has been declining from around │ │ │
│ │ │ 0.0245 to 0.0177, which is a │ │ │
│ │ ```python │ significant drop. This could │ │ │
│ │ def strategy(features, │ be a "post-dip curl" setup if │ │ │
│ │ position): │ we see a recovery. │ │ │
│ │ # features: dict of the │ │ │ │
│ │ current bar's indicators (see │ Let me design a strategy that: │ │ │
│ │ list below), all causal │ 1. Uses RSI to identify │ │ │
│ │ # position: your current │ oversold conditions │ │ │
│ │ target position, a float in │ 2. Uses MACD for momentum │ │ │
│ │ [-1, 1] │ confirmation │ │ │
│ │ # return: new target │ 3. Uses moving averages for │ │ │
│ │ position, float in [-1, 1] │ trend direction │ │ │
│ │ (1=full long, -1=full short, │ 4. Uses Bollinger Bands for │ │ │
│ │ 0=flat) │ position sizing │ │ │
│ │ ... │ 5. Incorporates volatility for │ │ │
│ │ return new_position │ risk management │ │ │
│ │ ``` │ 6. Follows the capital │ │ │
│ │ │ protection rules │ │ │
│ │ Available features (floats, │ │ │ │
│ │ computed on data up to the │ Key patterns to look for: │ │ │
│ │ current bar only): │ - Post-dip curl: RSI rising │ │ │
│ │ close, sma_10, sma_20, sma_50, │ from oversold (<30), MACD hist │ │ │
│ │ ema_12, ema_26, rsi_14, macd, │ turning positive after a drop │ │ │
│ │ macd_signal, macd_hist, │ - Healthy momentum: SMA_10 > │ │ │
│ │ zscore_20, bb_lo, bb_mid, │ SMA_20, MACD > MACD_signal │ │ │
│ │ bb_hi, ret_1, vol_20 │ - Avoid: Price too far from │ │ │
│ │ │ Bollinger Mid, high │ │ │
│ │ Hard rules: │ volatility, thin liquidity │ │ │
│ │ - No imports, no IO, no future │ │ │ │
│ │ data. You only get the current │ │ │ │
│ │ bar's features. │ │ │ │
│ │ - Favor capital protection: │ ```python │ │ │
│ │ control drawdown, avoid being │ def strategy(features, │ │ │
│ │ all-in blindly. │ position): │ │ │
│ │ - Your strategy is scored on a │ # Extract features │ │ │
│ │ HELD-OUT window and must beat │ close = features['close'] │ │ │
│ │ buy-and-hold. │ sma_10 = │ │ │
│ │ │ features['sma_10'] │ │ │
│ │ Trading discipline distilled │ sma_20 = │ │ │
│ │ from 618 real prior trading │ features['sma_20'] │ │ │
│ │ decisions (94 BUY, 489 WATCH, │ sma_50 = │ │ │
│ │ 35 AVOID): │ features['sma_50'] │ │ │
│ │ │ ema_12 = │ │ │
│ │ Patterns that earned a BUY │ features['ema_12'] │ │ │
│ │ (translate these into your │ ema_26 = │ │ │
│ │ strategy's entry logic): │ features['ema_26'] │ │ │
│ │ - "Post-dip curl": a 24h drop │ rsi_14 = │ │ │
│ │ followed by a 1h recovery with │ features['rsi_14'] │ │ │
│ │ a strong short-term buy/sell │ macd = features['macd'] │ │ │
│ │ ratio (>1.5). Enter on the │ macd_signal = │ │ │
│ │ curl, not the dip. → buy when │ features['macd_signal'] │ │ │
│ │ recent return turns up after a │ macd_hist = │ │ │
│ │ drawdown and momentum (rsi │ features['macd_hist'] │ │ │
│ │ rising from oversold, │ zscore_20 = │ │ │
│ │ macd_hist turning positive) │ features['zscore_20'] │ │ │
│ │ confirms. │ bb_lo = features['bb_lo'] │ │ │
│ │ - "Healthy momentum │ bb_mid = │ │ │
│ │ continuation": positive 1h AND │ features['bb_mid'] │ │ │
│ │ 24h change with buy pressure │ bb_hi = features['bb_hi'] │ │ │
│ │ and │ ret_1 = features['ret_1'] │ │ │
│ │ volume/liquidity > 1. → │ vol_20 = │ │ │
│ │ trend-follow when sma_10 > │ features['vol_20'] │ │ │
│ │ sma_20 and macd > macd_signal. │ │ │ │
│ │ - Size up only with liquidity │ # Initialize new position │ │ │
│ │ + volume backing the move; │ new_position = position │ │ │
│ │ size down when volatility is │ │ │ │
│ │ high. │ # Capital protection: │ │ │
│ │ │ avoid overexposure when │ │ │
│ │ Conditions that earned │ already up significantly │ │ │
│ │ WATCH/AVOID (translate into │ if ret_1 > 0.05: # │ │ │
│ │ your exits and risk limits): │ Already up 5% on the hour │ │ │
│ │ - Already +30% on the day → │ return min(position, │ │ │
│ │ entry timing is poor; do not │ 0.3) # Reduce exposure │ │ │
│ │ chase. → avoid full long when │ │ │ │
│ │ ret_1 is extremely high or │ # Avoid full long when │ │ │
│ │ price is far above bb_mid. │ price is far above middle │ │ │
│ │ - Volume/liquidity < 1.0 or │ Bollinger Band │ │ │
│ │ thin liquidity → lower │ if close > bb_mid * 1.03: │ │ │
│ │ conviction; reduce exposure. │ # 3% above BB mid │ │ │
│ │ - Sell-heavy short-term flow │ return max(position, │ │ │
│ │ after entry → cut quickly; │ -0.2) # Cap long exposure │ │ │
│ │ don't wait for a vague │ │ │ │
│ │ recovery. │ # Base position sizing │ │ │
│ │ │ inversely related to │ │ │
│ │ Capital-protection rules │ volatility │ │ │
│ │ (every BUY required these): │ vol_factor = max(0.5, 1.0 │ │ │
│ │ - Always define a stop below │ - vol_20) # Reduce size when │ │ │
│ │ entry and a target above; │ volatility high │ │ │
│ │ favor reward:risk >= 2:1. │ │ │ │
│ │ - Protect capital first: │ # Post-dip curl pattern: │ │ │
│ │ control drawdown over chasing │ RSI coming back from oversold │ │ │
│ │ upside. │ with MACD improvement │ │ │
│ │ - Don't average down losers; │ dip_recovery = False │ │ │
│ │ don't let a winner round-trip │ if rsi_14 < 30 and │ │ │
│ │ — tighten when flow weakens. │ macd_hist > 0: # Oversold │ │ │
│ │ │ with positive momentum │ │ │
│ │ │ dip_recovery = True │ │ │
│ │ │ │ │ │
│ │ user: Token: NOCK │ # Healthy momentum: short │ │ │
│ │ In-sample bars available: 600 │ and medium term trend │ │ │
│ │ Recent closes (last 30): │ confirmation │ │ │
│ │ [0.024502, 0.02227, 0.022973, │ momentum_ok = sma_10 > │ │ │
│ │ 0.022138, 0.022407, 0.02138, │ sma_20 and macd > macd_signal │ │ │
│ │ 0.02204, 0.025153, 0.023984, │ │ │ │
│ │ 0.025721, 0.024264, 0.023896, │ # Trend following with │ │ │
│ │ 0.023152, 0.022478, 0.023024, │ moving averages │ │ │
│ │ 0.022491, 0.022522, 0.02344, │ trend_up = ema_12 > ema_26 │ │ │
│ │ 0.023711, 0.022644, 0.022412, │ and sma_20 > sma_50 │ │ │
│ │ 0.022769, 0.02276, 0.023428, │ │ │ │
│ │ 0.022117, 0.021182, 0.020521, │ # Entry signals │ │ │
│ │ 0.019539, 0.018417, 0.017665] │ if dip_recovery and │ │ │
│ │ │ momentum_ok: │ │ │
│ │ Write a `strategy(features, │ # Post-dip entry with │ │ │
│ │ position)` function. Use │ strong confirmation │ │ │
│ │ indicators like RSI (rsi_14), │ new_position = min(0.6 │ │ │
│ │ MACD │ * vol_factor, 0.6) │ │ │
│ │ (macd/macd_signal/macd_hist), │ elif trend_up and │ │ │
│ │ moving averages (sma_*/ema_*), │ momentum_ok and zscore_20 < │ │ │
│ │ z-score (zscore_20), Bollinger │ 1.5: │ │ │
│ │ bands (bb_*), and volatility │ # Trend continuation │ │ │
│ │ (vol_20). Return a target │ with reasonable valuation │ │ │
│ │ position in [-1, 1]. │ new_position = min(0.4 │ │ │
│ │ │ * vol_factor, 0.4) │ │ │
│ │ │ elif zscore_20 < -1.0 and │ │ │
│ │ │ macd_hist > -0.001: │ │ │
│ │ │ # Mean reversion setup │ │ │
│ │ │ near lower Bollinger Band │ │ │
│ │ │ new_position = min(0.3 │ │ │
│ │ │ * vol_factor, 0.3) │ │ │
│ │ │ else: │ │ │
│ │ │ # Reduce exposure if │ │ │
│ │ │ no clear signal │ │ │
│ │ │ new_position = │ │ │
│ │ │ position * 0.7 │ │ │
│ │ │ │ │ │
│ │ │ # Exit management: reduce │ │ │
│ │ │ position if momentum weakens │ │ │
│ │ │ if macd_hist < 0 and │ │ │
│ │ │ position > 0.2: │ │ │
│ │ │ new_position = │ │ │
│ │ │ max(new_position, -0.1) # │ │ │
│ │ │ Quick exit │ │ │
│ │ │ elif rsi_14 > 70 and │ │ │
│ │ │ position > 0.3: │ │ │
│ │ │ new_position = │ │ │
│ │ │ max(new_position, 0.0) # Take │ │ │
│ │ │ profits on overbought │ │ │
│ │ │ │ │ │
│ │ │ # Ensure position bounds │ │ │
│ │ │ new_position = max(-1.0, │ │ │
│ │ │ min(1.0, new_position)) │ │ │
│ │ │ │ │ │
│ │ │ return new_position │ │ │
│ │ │ ``` │ │ │
│ │ │ </assistant> │ │ │
│ └────────────────────────────────┴────────────────────────────────┴────────┘ │
╰──────────────────────────────────────────────────────────────────────────────╯
--- All ---
Rewards:
reward: avg - 0.495, std - 0.328
r1: [0.915, 0.587, 0.0, 0.478]
reward_sharpe: avg - 0.448, std - 0.320
r1: [0.873, 0.577, 0.0, 0.344]
reward_beats_benchmark: avg - 0.250, std - 0.433
r1: [1.0, 0.0, 0.0, 0.0]
reward_drawdown: avg - 0.652, std - 0.381
r1: [0.943, 0.878, 0.0, 0.788]
reward_exposure: avg - 0.750, std - 0.433
r1: [1.0, 1.0, 0.0, 1.0]
reward_cost: avg - 0.355, std - 0.206
r1: [0.482, 0.5, 0.0, 0.438]
reward_valid: avg - 0.750, std - 0.433
r1: [1.0, 1.0, 0.0, 1.0]
num_turns: avg - 1.000, std - 0.000
r1: [1.0, 1.0, 1.0, 1.0]
Info:
is_truncated: avg - 0.000, std - 0.000
stop_conditions: max_turns_reached: 1.000
Timing:
total min - 6s, mean - 15s, max - 29s
setup min - 0ms, mean - 0ms, max - 0ms
generation min - 6s, mean - 15s, max - 29s
model min - 6s, mean - 15s, max - 29s
env min - 0ms, mean - 0ms, max - 0ms
scoring min - 1ms, mean - 41ms, max - 55ms
overhead min - 0ms, mean - 0ms, max - 1ms
Usage:
input_tokens (avg): 1061.500
output_tokens (avg): 2066.000
final_input_tokens (avg): 1061.500
final_output_tokens (avg): 2066.000
Performance:
event_loop_lag: mean=7ms, p99=3ms, max=2s (n=335)