Hugging Face's logo

Spaces:
bawsi99
/
indicators-env
Sleeping

App Files Community
indicators-env / README.md
bawsi99's picture
bawsi99
docs: update README and submission_summary to v4.1
147e9ed
preview code
|
Raw
History Blame Contribute Delete
7.3 kB
metadata 
title: IndicatorsEnv
emoji: 📈
colorFrom: indigo
colorTo: blue
sdk: docker
pinned: true
tags:
  - openenv
  - reinforcement-learning
  - finance
  - stock-market
  - technical-analysis
  - rl-environment
license: mit

IndicatorsEnv v4.1 — Multi-stock Portfolio MDP

An OpenEnv-compatible RL environment where agents learn within-sector relative alpha on real NSE (India) equities — with genuine portfolio capital dynamics.

Environment Description

IndicatorsEnv simulates the task a quantitative portfolio manager performs: given 3 stocks from the same sector, identify which has the strongest relative momentum signal, declare a direction, and set a conviction level. Reward is the market-neutral alpha the agent captures — the chosen stock's return minus the sector average.

Why Multi-stock + Portfolio State

Single-stock environments collapse to contextual bandits once you remove multi-step structure. IndicatorsEnv v4.1 solves this with two sources of genuine MDP state:

  1. Signal history — RSI trends, price momentum, and past pick outcomes accumulate across steps. The agent can detect multi-week divergence patterns.
  2. Portfolio capital — current_holding, capital, and drawdown are in every observation. Switching holdings costs 0.1% of capital. This creates a real exploration-exploitation tradeoff: staying in a position avoids transaction cost; switching to a better signal costs capital.

A random policy earns ≈ 0 expected reward (market-neutral by construction). A skilled policy earns consistently positive alpha.

Action Space 

{
  "stock":     "HDFCBANK",
  "direction": "Bullish",
  "conviction": 0.8
}
Field Type Description
stock string NSE symbol to trade, or "NONE" to pass this step
direction string (enum) Bullish (long alpha), Bearish (short alpha), or NONE
conviction float [0, 1] Kelly fraction — fraction of virtual wealth wagered

Passing (NONE) skips the step with zero reward and preserves the current holding at no cost. Switching to a different stock from the one held last step costs 0.1% of current capital.

Observation Space 

{
  "step": 3,
  "max_steps": 10,
  "term": "MEDIUM",
  "sector": "banking",
  "available_stocks": ["HDFCBANK", "ICICIBANK", "AXISBANK"],
  "stocks": {
    "HDFCBANK":  { "rsi_14": 61.2, "rsi_trend": "up",   "price_momentum_pct": 2.4, "indicators": {...} },
    "ICICIBANK": { "rsi_14": 54.8, "rsi_trend": "flat", "price_momentum_pct": 0.8, "indicators": {...} },
    "AXISBANK":  { "rsi_14": 48.1, "rsi_trend": "down", "price_momentum_pct": -1.1, "indicators": {...} }
  },
  "signal_history": [
    { "step": 1, "picked_stock": "HDFCBANK", "direction": "Bullish", "correct": true,  "alpha_pct": 1.8, "reward": 0.63 },
    { "step": 2, "picked_stock": "NONE",     "direction": "NONE",    "correct": null,  "alpha_pct": 0.0, "reward": 0.0  }
  ],
  "current_holding": "HDFCBANK",
  "capital": 1.0126,
  "drawdown": 0.0,
  "macro": null
}

Per-stock Indicator Suite (25+ signals)

Category Indicators
Moving Averages SMA(20/50/200), EMA(20/50), Golden/Death Cross
Momentum RSI(14) + trend, MACD(12/26/9) + crossover, Stochastic(14/3)
Volatility Bollinger Bands (%, width, squeeze), ATR(14), volatility regime
Trend ADX(14), +DI/-DI, trend strength
Volume OBV, VWAP, MFI(14), CMF(20), A/D Line, volume ratio
Levels Pivot Points (R2/R1/P/S1/S2)

Tasks

Task 1: Short-term Relative Alpha (Easy)

  • Steps: 5 (1 trading day apart = 1 week)
  • GT: 1-day forward return, ±0.3% threshold
  • Portfolio: capital tracked, 5% drawdown terminates episode early
  • Grader: Directional accuracy on active (non-NONE) steps

Task 2: Medium-term Relative Alpha (Medium)

  • Steps: 10 (5 trading days apart = 10 weeks)
  • GT: 5-day forward return, ±1.5% threshold
  • Portfolio: capital tracked, 10% drawdown terminates episode early
  • Grader: Weighted accuracy — Bearish/Neutral correct = 1.5× weight; participation bonus

Task 3: Long-term Risk-Constrained Alpha (Hard)

  • Steps: 15 (20 trading days apart = 15 months)
  • GT: 20-day forward return, ±2.5% threshold
  • Portfolio: capital tracked, 15% drawdown terminates episode early; macro context (NIFTY50 trend, market regime) added
  • Grader: Conviction-calibrated accuracy — correct + conviction ≥ 0.7 → 1.0; overconfident wrong → −0.1

Reward Function 

alpha          = chosen_stock_period_return − sector_average_return
direction_sign = +1 (Bullish) or −1 (Bearish)
raw_reward     = alpha × direction_sign × conviction × 50
tx_cost        = 0.001 × capital  (if switching from a different held stock, else 0)
reward         = clamp(raw_reward − tx_cost, −1.5, 1.5)

Example (medium task, step 3):

  • HDFCBANK returns +3.5%, sector average +1.5%
  • alpha = +2.0%, direction = Bullish, conviction = 0.8
  • No switch (was already holding HDFCBANK): tx_cost = 0
  • reward = 0.02 × 1 × 0.8 × 50 = 0.80

Market-neutral property: Because reward is relative to the sector average, a random policy earns ≈ 0 in expectation. Broad market moves cancel out. The agent profits only from correctly identifying within-sector outperformers.

Kelly conviction: Higher conviction amplifies both gains and losses, incentivizing calibrated confidence rather than uniform maximum bets.

Episode Structure 

reset()  → sector=banking  stocks=[HDFCBANK, ICICIBANK, AXISBANK]  capital=1.0  holding=NONE

step 1   → pick HDFCBANK Bullish 0.7   reward=+0.63  done=False  capital=1.009  tx_cost=0.0
step 2   → NONE                         reward=0.00   done=False  capital=1.009  tx_cost=0.0
step 3   → pick ICICIBANK Bullish 0.8   reward=+0.41  done=False  capital=1.017  tx_cost=0.001  ← switch cost
step 4   → pick ICICIBANK Bullish 0.6   reward=-0.15  done=False  capital=1.008  tx_cost=0.0   ← same stock
step 5   → pick HDFCBANK Bearish 0.5    reward=+0.22  done=True   capital=1.012  tx_cost=0.001  ← switch cost

Baseline Scores (Random Agent, seed=42)

Task Steps per run Expected score
Short-term Direction 50 (10 eps × 5 steps) ~0.33
Medium-term Direction 100 (10 eps × 10 steps) ~0.33
Long-term Conviction ≤150 (10 eps × ≤15 steps) ~0.10

Setup & Usage

Local 

pip install -r requirements-server.txt
cd env/
python indicators_env.py   # Runs on http://localhost:7860

Docker 

docker build -t indicators-env .
docker run -p 7860:7860 indicators-env

Inference Agent 

export API_BASE_URL=https://router.huggingface.co/v1/
export MODEL_NAME=meta-llama/Llama-3.2-1B-Instruct
export HF_TOKEN=hf_your_token

python inference.py --env_url http://localhost:7860 --n_episodes 5

Links