finish training ppo rnn pnl/active return multi-scale features combos

progress ppo rnn pnl/active return multi-scale features combos

Merge branch 'main' of hf.co:bobotsalos/rnn_crypto

progress ppo rnn pnl/active return multi-scale feature combos

ppo rnn pnl/active reward base momentum features no double actor hidden state advancing + no floor on buys with not enough cash

ppo rnn pnl/active reward base momentum features

fix old model

ppo rnn with normalized momentum features

ppo rnn strategy reward base momentum features grad_clip 0.5

ppo rnn pnl reward with new momentum features

ppo rnn with cidl features no cyuclical, no adv normalization, and with ent coef

ppo rnn with no cyclical features + with cidl features not normalized

ppo rnn cidl features with only week sin/cos sharpe reward + cidl features different window lengths and envs

agents ppo rnn cidl features with only week sin/cos

ppo rnn with bootstrap last value on episode end and 1k window length: 1) active return 3e-4 pi lr, 2) active return penalized turnovers 3e-4 pi lr, 3) sharpe based reward

ppo with bootstrap V value at episode boundaries not zero

finish with ppo rnn with sharpe ratio reward base/cidl features

trained ppo 5 versions adv normalization, rew norm w/ critic grad clip 1.0, rew norm w/ vf_lr 1e-5, vf_lr 1e-5, clip_ratio 0.1

progress (1/5) train ppo rnn with sharpe ratio reward base/cidl features

ppo active return 1e-3 pi_lr rewards std advantage normalization

ppo rnn active return pi_lr 3e-4 or 1e-3

initial commit