--- title: EuNEx emoji: πŸ“ˆ colorFrom: green colorTo: blue sdk: docker app_port: 7860 pinned: false --- # EuNEx β€” Euronext Optiq Architecture Learning Project C++ actor-based matching engine that mirrors the Euronext Optiq architecture, ported from the [StockEx](https://github.com/Bonum/StockEx) Python prototype. ## Architecture Mapping ``` StockEx (Python/Kafka) EuNEx (C++/Simplx) Optiq (Production) ───────────────────── ────────────────── ────────────────── fix_oeg_server.py β†’ OEGActor β†’ OEActor Kafka 'orders' topic β†’ Event::Pipe β†’ Simplx Event::Pipe matcher.py β†’ MECoreActor β†’ LogicalCoreActor + Book match_order() β†’ Book::newOrder() β†’ RecoveryCause β†’ IACA Cause β†’ forwardToBook handle_cancel() β†’ Book::cancelOrder() β†’ CancelOrderData handle_amend() β†’ Book::modifyOrder() β†’ ModifyOrderData Kafka 'trades' topic β†’ TradeEvent via Pipe β†’ IACA fragment chain dashboard.py (SSE) β†’ MDGActor β†’ MDLimitLogicalCoreHandler /orderbook/ β†’ BookUpdateEvent β†’ PublishLimitUpdateRequest /trades β†’ TradeEvent β†’ IACA β†’ IA SBE message database.py (SQLite) β†’ RecoveryProxy (memory) β†’ RecoveryProxy β†’ Kafka save_trade() β†’ FragmentStore::append() β†’ PersistenceAgent β†’ Kafka produce fix_oeg_server.py β†’ FIXAcceptorActor β†’ FIX 4.4 OEG Acceptor NewOrderSingle β†’ 35=D handling β†’ Optiq FIX gateway ExecutionReport β†’ 35=8 response β†’ Execution reports ch_ai_trader.py β†’ ClearingHouseActor β†’ Clearing House (PTB path) AI strategies β†’ AITraderActor β†’ Trading obligations ``` ## Actor Topology (v0.9) ``` Core 0: OEGActor + FIXAcceptorActor ← Order entry & FIX protocol Core 1: MECoreActor (per symbol) ← Matching engine (Book) Core 2: MDGActor ← Market data snapshots Core 3: ClearingHouseActor + AITrader ← Post-trade & AI members ``` ## Dual Engine Architecture The dashboard can run in two modes, switchable at runtime via a toggle in the UI: ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Browser (:8090) β”‚ β”‚ [Python ○────● C++] toggle β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Dashboard (Flask :8090) β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ Python ME β”‚ β”‚ C++ Bridge β”‚ β”‚ β”‚ β”‚MatchingEng β”‚ β”‚ FIX 4.4 TCP β”‚ β”‚ β”‚ β”‚(built-in) β”‚ β”‚ client β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ mode=python mode=cpp β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ FIX 4.4 TCP β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ C++ Matching Engine β”‚ β”‚ eunex_me (:9001) β”‚ β”‚ β”‚ β”‚ OEG β†’ MECore β†’ MDG β†’ ClearingHouse β”‚ β”‚ FIXAcceptorActor (TCP :9001) β”‚ β”‚ Multi-threaded actors β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` | Mode | Engine | How orders are matched | Use case | |------|--------|----------------------|----------| | **Python** | Built-in `MatchingEngine` | In-process, same Flask app | Demo, learning, no compilation needed | | **C++** | `eunex_me` via FIX 4.4 | TCP to C++ actors on port 9001 | Production-like, microsecond latency | ## Service Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ nginx (:7860) β”‚ β”‚ Reverse proxy (Docker) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β–Ό β–Ό β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Dashboard β”‚ β”‚ Clearing House β”‚ β”‚ FIX Gateway β”‚ β”‚ (:8090) β”‚ β”‚ (:8091) β”‚ β”‚ (:9001 TCP) β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ Order Book β”‚ β”‚ 10 AI Members β”‚ β”‚ FIX 4.4 Acceptorβ”‚ β”‚ Trade Charts β”‚ β”‚ Leaderboard β”‚ β”‚ NewOrder/Cancel β”‚ β”‚ OHLCV History β”‚ β”‚ Portfolios β”‚ β”‚ Amend/ExecRpt β”‚ β”‚ SQLite DB β”‚ β”‚ Settlements β”‚ β”‚ β”‚ β”‚ SSE Streaming β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ Engine Switch β”‚ β”‚ LLM Trading β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β–Ό β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ C++ Matching Engine β”‚ β”‚ (eunex_me) β”‚ β”‚ Multi-threaded actors β”‚ β”‚ Price-time priority β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ## Quick Start (Linux) ```bash # Install dependency pip install flask # Start all services (dashboard + FIX gateway + clearing house) ./run.sh # Services: # Dashboard: http://localhost:8090 # Clearing House: http://localhost:8091 # FIX Gateway: localhost:9001 (TCP) # Stop all ./run.sh stop # Check status ./run.sh status ``` ## Quick Start (Docker β€” Windows/Linux) ```bash cd docker docker compose up --build # All services via nginx: # Dashboard: http://localhost:7860 # Clearing House: http://localhost:7860/ch/ # Kafka: localhost:9092 ``` ## Build C++ Engine ```bash cmake -B build -DEUNEX_BUILD_TESTS=ON cmake --build build --config Release # Run matching engine (FIX server on port 9001) ./build/Release/eunex_me # Run all tests (7 suites) cd build && ctest -C Release # Use with dashboard: start eunex_me, then toggle "C++" in the dashboard header ``` ## With Kafka Persistence ```bash # Compile with Kafka support (requires librdkafka-dev) cmake -B build -DEUNEX_USE_KAFKA=ON cmake --build build --config Release # Run with Kafka (set broker address) EUNEX_KAFKA_BROKERS=localhost:9092 ./build/Release/eunex_me # Topics: eunex.orders, eunex.trades, eunex.market-data, eunex.recovery.fragments ``` Without `EUNEX_USE_KAFKA`, the engine compiles with a no-op stub and runs standalone. ## FIX Gateway The C++ engine includes a built-in FIX 4.4 acceptor on TCP port 9001. A Python FIX gateway is also available: ```bash python fix_gateway/fix_server.py python fix_gateway/fix_server.py test ``` Supports: NewOrderSingle (35=D), OrderCancelRequest (35=F), OrderCancelReplaceRequest (35=G), ExecutionReport (35=8). ## Configuration (.env) All Python services auto-load settings from `.env` in the project root. No extra packages needed β€” `shared/config.py` parses it at startup. ```bash # .env β€” LLM and service configuration LLM_PROVIDER=auto # ollama | groq | hf | auto (try all) OLLAMA_HOST=http://localhost:11434 OLLAMA_MODEL=llama3.1 HF_MODEL=Qwen/Qwen2.5-7B-Instruct # HF_TOKEN=hf_... # or auto-discovered from ~/.cache/huggingface/token # GROQ_API_KEY=gsk_... ``` | Variable | Default | Description | |----------|---------|-------------| | `LLM_PROVIDER` | `auto` | LLM backend: `ollama`, `groq`, `hf`, or `auto` (tries all in order) | | `OLLAMA_HOST` | `http://localhost:11434` | Ollama server URL | | `OLLAMA_MODEL` | `llama3.1` | Ollama model name | | `HF_TOKEN` | *(auto-discovered)* | HuggingFace API token; reads `~/.cache/huggingface/token` if not set | | `HF_MODEL` | `Qwen/Qwen2.5-7B-Instruct` | HuggingFace model (free tier) | | `GROQ_API_KEY` | *(empty)* | Groq Cloud API key | | `GROQ_MODEL` | `llama-3.1-8b-instant` | Groq model name | | `EUNEX_DASHBOARD_PORT` | `8090` | Dashboard HTTP port | | `EUNEX_CH_PORT` | `8091` | Clearing House HTTP port | | `EUNEX_FIX_PORT` | `9001` | FIX Gateway TCP port | | `EUNEX_KAFKA_BROKERS` | `localhost:9092` | Kafka broker address | ## AI Market Analyst The Dashboard includes an AI-powered market analyst that generates real-time commentary on trading activity. It uses a fallback chain of LLM providers: 1. **Ollama** (local) β€” best for development, requires [Ollama](https://ollama.ai) installed 2. **Groq** (cloud) β€” fast inference, requires `GROQ_API_KEY` 3. **HuggingFace** (cloud) β€” free tier with `HF_TOKEN`, uses Qwen 2.5 7B The dashboard shows a green/red status dot and lets you select the active Ollama model from a dropdown. Set `LLM_PROVIDER=auto` to try all providers in order until one succeeds. ## Clearing House 10 AI trading members (MBR01-MBR10) with 4 strategies: - **Momentum**: follow price trends - **Mean Reversion**: fade price moves - **Random**: noise trading - **LLM**: AI-driven decisions with natural language explanations (uses same provider chain) Features: capital tracking, holdings per symbol, P&L, leaderboard, strategy selection per member, LLM trading explanations panel. ## Project Structure ``` EuNEx/ β”œβ”€β”€ src/ # C++ matching engine β”‚ β”œβ”€β”€ main.cpp # Entry point, actor wiring β”‚ β”œβ”€β”€ engine/SimplxShim.hpp # Multi-threaded actor engine β”‚ β”œβ”€β”€ common/ β”‚ β”‚ β”œβ”€β”€ Types.hpp # Price, Order, Trade, enums β”‚ β”‚ └── Book.hpp/cpp # Price-time priority matching β”‚ β”œβ”€β”€ actors/ β”‚ β”‚ β”œβ”€β”€ Events.hpp # Inter-actor event types β”‚ β”‚ β”œβ”€β”€ OEGActor.hpp/cpp # Order Entry Gateway β”‚ β”‚ β”œβ”€β”€ MECoreActor.hpp/cpp # Matching Engine core (per symbol) β”‚ β”‚ β”œβ”€β”€ MDGActor.hpp/cpp # Market Data Gateway β”‚ β”‚ β”œβ”€β”€ FIXAcceptorActor.hpp/cpp # FIX 4.4 TCP acceptor β”‚ β”‚ β”œβ”€β”€ ClearingHouseActor.hpp/cpp # Trade clearing & member positions β”‚ β”‚ └── AITraderActor.hpp/cpp # Automated trading members β”‚ β”œβ”€β”€ net/SocketCompat.hpp # Cross-platform socket abstraction β”‚ β”œβ”€β”€ persistence/ β”‚ β”‚ β”œβ”€β”€ PersistenceStore.hpp # Abstract store + InMemoryStore β”‚ β”‚ β”œβ”€β”€ KafkaBus.hpp # Multi-topic Kafka publisher (Optiq KFK) β”‚ β”‚ └── KafkaStore.hpp # Kafka persistence (optional) β”‚ β”œβ”€β”€ recovery/RecoveryProxy.hpp/cpp # Recovery Cause/Effect β”‚ └── iaca/ β”‚ β”œβ”€β”€ Fragment.hpp # IACA fragment definitions β”‚ └── IacaAggregator.hpp/cpp # Fragment chain aggregation β”œβ”€β”€ dashboard/ β”‚ β”œβ”€β”€ app.py # Flask dashboard + matching engine β”‚ β”œβ”€β”€ database.py # SQLite (orders, trades, OHLCV) β”‚ └── templates/index.html # Trading UI with Chart.js β”œβ”€β”€ fix_gateway/ β”‚ └── fix_server.py # Python FIX 4.4 TCP acceptor β”œβ”€β”€ clearing_house/ β”‚ β”œβ”€β”€ app.py # Flask CH portal + API β”‚ β”œβ”€β”€ ch_database.py # SQLite (members, holdings) β”‚ β”œβ”€β”€ ch_ai_trader.py # AI trading strategies β”‚ └── templates/ # CH web UI β”œβ”€β”€ .env # LLM and service configuration β”œβ”€β”€ shared/config.py # Centralized configuration (auto-loads .env) β”œβ”€β”€ docker/ β”‚ β”œβ”€β”€ docker-compose.yml # Kafka + EuNEx (all services) β”‚ β”œβ”€β”€ Dockerfile # Multi-stage Linux build β”‚ └── nginx.conf # Reverse proxy configuration β”œβ”€β”€ tests/ β”‚ β”œβ”€β”€ test_orderbook.cpp # Book unit tests (26 cases) β”‚ β”œβ”€β”€ test_matching_engine.cpp # ME integration tests β”‚ β”œβ”€β”€ test_threaded_engine.cpp # Multi-threaded engine tests β”‚ β”œβ”€β”€ test_clearing_house.cpp # Clearing house tests (7 cases) β”‚ β”œβ”€β”€ test_fix_gateway.cpp # FIX gateway tests (5 cases) β”‚ └── test_ai_trader.cpp # AI trader tests (6 cases) β”œβ”€β”€ examples/ β”‚ β”œβ”€β”€ ping_pong.cpp # Actor basics tutorial β”‚ └── simple_match.cpp # Matching with Recovery + IACA └── docs/ β”œβ”€β”€ developers-guide.md # Detailed developers guide └── process-diagram.md # Architecture diagrams ``` ## Documentation - **[Developers Guide](docs/developers-guide.md)** β€” Detailed architecture, data flow, component reference - **[Process Diagram](docs/process-diagram.md)** β€” Optiq architecture diagrams and roadmap ## Next Steps 1. ~~Multi-threaded actor engine~~ βœ“ SimplxShim with mailbox queues 2. ~~Kafka persistence~~ βœ“ KafkaStore + Docker Compose (KRaft mode) 3. ~~FIX gateway~~ βœ“ C++ FIXAcceptorActor + Python fallback 4. ~~Clearing House~~ βœ“ ClearingHouseActor + AITraderActor 5. ~~Market simulation~~ βœ“ Realistic AI trading + Dashboard auto-simulation 6. ~~AI Analyst~~ βœ“ Ollama/Groq/HuggingFace market commentary 7. ~~Message Flow Visualizer~~ βœ“ Developer pipeline tracing tool 8. ~~Engine Mode Switch~~ βœ“ Python ↔ C++ toggle with FIX 4.4 bridge 9. **SBE encoding** β€” replace event structs with SBE-encoded messages 10. **Master/Mirror failover** β€” implement full Recovery replay on Mirror node 11. **Trading phases** β€” pre-open, uncrossing, continuous, close, TAL 12. **Additional order types** β€” Stop, Pegged, Mid-Point, Iceberg