| # EuNEx Developers Guide |
|
|
| **Version 0.9.5** | Euronext Optiq-Modeled Exchange Simulator |
|
|
| --- |
|
|
| ## Table of Contents |
|
|
| 1. [Overview](#1-overview) |
| 2. [Architecture](#2-architecture) |
| 3. [Optiq Naming Alignment](#3-optiq-naming-alignment) |
| 4. [Actor Topology](#4-actor-topology) |
| 5. [Data Flow](#5-data-flow) |
| 6. [Core Components](#6-core-components) |
| 7. [Event System](#7-event-system) |
| 8. [Order Book & Matching](#8-order-book--matching) |
| 9. [Recovery & IACA](#9-recovery--iaca) |
| 10. [Kafka Bus](#10-kafka-bus) |
| 11. [FIX Protocol Gateway](#11-fix-protocol-gateway) |
| 12. [Clearing House](#12-clearing-house) |
| 13. [AI Trading Members](#13-ai-trading-members) |
| 14. [Market Simulation](#14-market-simulation) |
| 15. [AI Market Analyst](#15-ai-market-analyst) |
| 16. [Developer Message Flow Visualizer](#16-developer-message-flow-visualizer) |
| 17. [Engine Mode Switch](#17-engine-mode-switch) |
| 18. [Project Structure](#18-project-structure) |
| 19. [Build & Test](#19-build--test) |
| 20. [Configuration](#20-configuration) |
| 21. [Extending EuNEx](#21-extending-eunex) |
|
|
| --- |
|
|
| ## 1. Overview |
|
|
| EuNEx (Euronext Exchange Simulator) is a C++20 actor-based matching engine that mirrors the Euronext Optiq production architecture. It implements the full order lifecycle: entry, validation, matching, market data dissemination, trade clearing, and FIX protocol connectivity. |
|
|
| ``` |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β EuNEx v0.9.5 β |
| β β |
| β FIX 4.4 Clients βββΊ OEG βββΊ ME (per symbol) βββΊ MDG β |
| β β β β |
| β βββ Exec Reports βββββ β |
| β β β |
| β Clearing House β |
| β β β |
| β AI Traders (x10) β |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| **Key design principles:** |
| - Actor-per-core isolation (no shared mutable state in the hot path) |
| - Lock-free `Event::Pipe` for cross-actor communication |
| - Fixed-point pricing (8 decimal places, `PRICE_SCALE = 10^8`) |
| - Price-time priority matching with multi-level sweep |
| - Master/Mirror recovery gating for high availability |
|
|
| --- |
|
|
| ## 2. Architecture |
|
|
| ### High-Level System Diagram |
|
|
| ``` |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β EXTERNAL CLIENTS β |
| β Browser / FIX 4.4 / Direct API / REST β |
| ββββββββ¬βββββββββββββββββββ¬ββββββββββββββββββββββββββββββββ¬ββββββββββββββββββββ |
| β β β |
| HTTP :7860 TCP :9001 Direct API |
| β β β |
| ββββββββͺβββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β PRESENTATION LAYER (Python / nginx) β |
| β β |
| β ββββββββββββββββββββββββ ββββββββββββββββββββββββ β |
| β β nginx (:7860) β β β β |
| β β reverse proxy β β β β |
| β β / β :8090 β β β β |
| β β /ch/ β :8091 β β β β |
| β βββββ¬βββββββββββ¬ββββββββ β β β |
| β β β β β β |
| β βΌ βΌ β β β |
| β ββββββββββββ ββββββββββββ β β β |
| β βDashboard β βClearing β β β β |
| β β (:8090) β βHouse UI β β β β |
| β β β β (:8091) β β β β |
| β β Order β β β β β β |
| β β Book β β Leader- β β β β |
| β β Charts β β board β β β β |
| β β OHLCV β β Holdings β β β β |
| β β SSE β β P&L β β β β |
| β ββββββ¬ββββββ ββββββ¬ββββββ β β β |
| β β β β β β |
| β getSnapshot() getLeaderboard() β β |
| β getRecentTrades() (thread-safe reads) β β |
| βββββββββͺβββββββββββββͺβββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β β β β |
| ββββββββββββββ΄βββββββββ β |
| β β |
| Thread-safe C++ API TCP :9001 |
| β β |
| ββββββββββββββββββββββͺβββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββ |
| β CORE 0 β Gateway β |
| β ββββββββββββββββββββββββββββ ββββββββββββββββββββββββββββ β |
| β β FIXAcceptorActor ββββββ€ OEGActor ββββββ€ β |
| β β βββββΊβ β β |
| β β β’ TCP accept loop β β β’ Session validation β β |
| β β β’ FIX 4.4 parse/build β β β’ Symbol routing β β |
| β β β’ Logon/Logout/HB β β β’ Exec report fanout β β |
| β β β’ D, F, G β Events β β β’ Cancel, Modify β β |
| β ββββββββββββββββββββββββββββ ββββββββββββ¬ββββββββββββββββ β |
| βββββββββββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββ |
| β NewOrderEvent |
| β CancelOrderEvent |
| β ModifyOrderEvent |
| β (Event::Pipe) |
| βΌ |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β CORE 1 β Matching Engine (one MECoreActor per symbol) β |
| β β |
| β βββββββββββββββββ βββββββββββββββββ βββββββββββββββββ βββββββββββββββββ β |
| β β MECoreActor β β MECoreActor β β MECoreActor β β MECoreActor β β |
| β β AAPL (sym=1) β β MSFT (sym=2) β β GOOGL (sym=3) β β EURO50 (sym=4)β β |
| β β β β β β β β β β |
| β β βββββββββββ β β βββββββββββ β β βββββββββββ β β βββββββββββ β β |
| β β β Book β β β β Book β β β β Book β β β β Book β β β |
| β β β (bids) β β β β (bids) β β β β (bids) β β β β (bids) β β β |
| β β β (asks) β β β β (asks) β β β β (asks) β β β β (asks) β β β |
| β β βββββββββββ β β βββββββββββ β β βββββββββββ β β βββββββββββ β β |
| β βββββββββ¬ββββββββ βββββββββ¬ββββββββ βββββββββ¬ββββββββ βββββββββ¬ββββββββ β |
| β β β β β β |
| β ββββββββββ¬βββββββββ΄βββββββββ¬βββββββββ΄βββββββββ¬βββββββββ β |
| βββββββββββββββββββββͺβββββββββββββββββͺβββββββββββββββββͺββββββββββββββββββββββββ |
| β β β |
| ExecReportβ TradeEventβ BookUpdateβ |
| (β OEG) β (β MDG) β (β MDG) β |
| β β β |
| βββββββββββββββββββββͺβββββββββββββββββͺβββββββββββββββββͺββββββββββββββββββββββββ |
| β KAFKA BUS (optional, Optiq KFK) β β β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β KafkaBus (shared by all MECoreActors) β β |
| β β β β |
| β β β’ eunex.orders β raw Order on entry β β |
| β β β’ eunex.trades β Trade on each fill β β |
| β β β’ eunex.market-data β BBO snapshot on book update β β |
| β β β’ eunex.recovery.fragments β recovery data β β |
| β β β β |
| β β Enabled: EUNEX_USE_KAFKA=ON + EUNEX_KAFKA_BROKERS env var β β |
| β β Disabled: no-op stub compiles in (default) β β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β β β |
| βββββββββββββββββββββͺβββββββββββββββββͺβββββββββββββββββͺββββββββββββββββββββββββ |
| β CORE 2 β Market Data βΌ βΌ β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β MDGActor β β |
| β β β β |
| β β β’ BBO snapshots per symbol (bestBid, bestAsk, depths) β β |
| β β β’ Recent trade history (last 200 trades) β β |
| β β β’ Trade count, volume, last price per symbol β β |
| β β β’ Thread-safe reads for Python dashboard bridge β β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β |
| TradeEvent |
| (β Clearing) |
| β |
| βββββββββββββββββββββββββββββββββββββββββββββββͺββββββββββββββββββββββββββββββββ |
| β CORE 3 β Post-Trade & AI βΌ β |
| β βββββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββ β |
| β β ClearingHouseActor β β AITraderActor β β |
| β β β β β β |
| β β β’ Session β Member map β β β’ 10 members (MBR01-10) β β |
| β β β’ Capital tracking β β β’ 3 strategies: β β |
| β β β’ Holdings per symbol β β momentum, mean_revert, β β |
| β β β’ P&L calculation β β random β β |
| β β β’ Leaderboard (sorted) β β β’ Periodic order bursts β β |
| β β β’ Thread-safe for bridge β β β’ BBO + price history β β |
| β βββββββββββββββββββββββββββββββ ββββββββββββββββ¬βββββββββββββββ β |
| β β β |
| β NewOrderEvent β |
| β (β OEG β ME) β |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| --- |
|
|
| ## 3. Optiq Naming Alignment |
|
|
| EuNEx components are named to match Euronext Optiq production terminology: |
|
|
| ``` |
| Optiq Production EuNEx Class File |
| βββββββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββββββββββ |
| OEActor (OE Gateway) ββββΊ OEGActor src/actors/OEGActor.hpp |
| LogicalCoreActor + Book ββββΊ MECoreActor src/actors/MECoreActor.hpp |
| Book (order book engine) ββββΊ Book src/common/Book.hpp |
| MDLimitLogicalCoreHandler ββββΊ MDGActor src/actors/MDGActor.hpp |
| OEG FIX Gateway ββββΊ FIXAcceptorActor src/actors/FIXAcceptorActor.hpp |
| Clearing House (PTB path) ββββΊ ClearingHouseActor src/actors/ClearingHouseActor.hpp |
| Member Trading Bots ββββΊ AITraderActor src/actors/AITraderActor.hpp |
| RecoveryProxy ββββΊ RecoveryProxy src/recovery/RecoveryProxy.hpp |
| IacaAggregatorActor ββββΊ IacaAggregator src/iaca/IacaAggregator.hpp |
| ``` |
|
|
| **Why these names?** |
| - **OEG** = Order Entry Gateway (the Optiq front-end for all order flow) |
| - **ME** = Matching Engine (the core `Book` + `LogicalCoreActor` in Optiq) |
| - **MDG** = Market Data Gateway (publishes BBO, trades, depth to consumers) |
| - **FIXAcceptor** = Optiq's FIX protocol acceptor component within OEG |
|
|
| --- |
|
|
| ## 4. Actor Topology |
|
|
| ### Core Affinity Layout |
|
|
| ``` |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β PRESENTATION (Python) β |
| β β |
| β ββ nginx :7860 βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β / β Dashboard :8090 /ch/ β Clearing House UI :8091 β β |
| β ββββββββββββ¬βββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββ β |
| β β β β |
| β ββββββββββββΌββββββββββββββββββ ββββββββββΌββββββββββββββββββββββββ β |
| β β Dashboard (Flask :8090) β β Clearing House UI (Flask :8091)β β |
| β β β’ Order Book, Charts β β β’ Leaderboard, P&L, Holdings β β |
| β β β’ OHLCV, SSE streaming β β β’ Member portfolios β β |
| β β β’ SQLite for history β β β β |
| β ββββββββββββ¬ββββββββββββββββββ ββββββββββ¬ββββββββββββββββββββββββ β |
| β β getSnapshot() β getLeaderboard() β |
| β β getRecentTrades() β (thread-safe reads) β |
| β β (thread-safe reads) β β |
| βββββββββββββββΌββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ |
| β β |
| βΌ βΌ |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β C++ ACTOR ENGINE β |
| β β |
| β ββ CPU Core 0 ββββββββββββββββββ ββ CPU Core 1 βββββββββββββββββββββ β |
| β β β β β β |
| β β OEGActor β β MECoreActor (AAPL, sym=1) β β |
| β β β’ Symbol β Book routing β β MECoreActor (MSFT, sym=2) β β |
| β β β’ Exec report fanout β β MECoreActor (GOOGL, sym=3) β β |
| β β β β MECoreActor (EURO50, sym=4) β β |
| β β FIXAcceptorActor β β β β |
| β β β’ TCP :9001 β β Each owns a Book instance β β |
| β β β’ FIX 4.4 protocol β β KafkaBus* β Kafka topics β β |
| β β β β No locks in matching path β β |
| β βββββββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββ β |
| β β |
| β ββ CPU Core 2 ββββββββββββββββββ ββ CPU Core 3 βββββββββββββββββββββ β |
| β β β β β β |
| β β MDGActor β β ClearingHouseActor β β |
| β β β’ BBO per symbol β β β’ 10 members, capital, P&L β β |
| β β β’ Trade history β β β’ Session β Member mapping β β |
| β β β’ Snapshot queries β β β β |
| β β βββ Dashboard reads here β β AITraderActor β β |
| β β β β β’ 10 AI members β β |
| β β β β β’ Momentum / MeanRev / Rand β β |
| β βββββββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββββββ β |
| β β |
| β ββ KafkaBus (optional) βββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β eunex.orders β eunex.trades β eunex.market-data β eunex.recovery β β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| ### Event::Pipe Connections |
|
|
| ``` |
| FIXAcceptorActor ββNewOrderEventβββββββΊ OEGActor ββNewOrderEventβββΊ MECoreActor |
| ββCancelOrderEventβββΊ |
| ββModifyOrderEventβββΊ |
| |
| MECoreActor ββExecReportEventβββΊ OEGActor ββExecReportEventβββΊ FIXAcceptorActor |
| ββExecReportEventβββΊ AITraderActor |
| |
| MECoreActor ββTradeEventβββββββΊ MDGActor |
| ββBookUpdateEventββΊ MDGActor |
| ββTradeEventβββββββΊ ClearingHouseActor |
| ββpublishOrder()βββΊ KafkaBus β eunex.orders (if Kafka enabled) |
| ββpublishTrade()βββΊ KafkaBus β eunex.trades (if Kafka enabled) |
| ββpublishMarketData()βΊ KafkaBus β eunex.market-data (if Kafka enabled) |
| |
| AITraderActor ββNewOrderEventβββΊ OEGActor (via pipe) |
| |
| Dashboard (Python) ββgetSnapshot()βββββββΊ MDGActor (thread-safe read) |
| ββgetRecentTrades()βββΊ MDGActor (thread-safe read) |
| Clearing House UI ββgetLeaderboard()ββββΊ ClearingHouseActor (thread-safe read) |
| ``` |
|
|
| --- |
|
|
| ## 5. Data Flow |
|
|
| ### Order Lifecycle (New Limit Order) |
|
|
| ``` |
| ββββββββββββ FIX 35=D ββββββββββββββββββββ NewOrderEvent ββββββββββββββββ |
| β Client β βββββββββββββββΊ β FIXAcceptorActor β βββββββββββββββββΊ β OEGActor β |
| β (FIX) β β parse FIX tags β β route by sym β |
| ββββββββββββ β map symbol string β ββββββββ¬ββββββββ |
| β to SymbolIndex_t β β |
| βββββββββββββββββββββ β |
| NewOrderEvent |
| (Event::Pipe) |
| β |
| βΌ |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β MECoreActor β |
| β β |
| β 1. KafkaBus.publishOrder() β eunex.orders * β |
| β 2. RecoveryProxy.cause() β persist fragment β |
| β 3. Book.newOrder() β price-time matching β |
| β 4. For each fill: β |
| β a. Trade generated (buyer + seller session) β |
| β b. TradeEvent β MDGActor β |
| β c. TradeEvent β ClearingHouseActor β |
| β d. KafkaBus.publishTrade() β eunex.trades * β |
| β 5. ExecReportEvent β OEGActor (ack/fill/reject) β |
| β 6. BookUpdateEvent β MDGActor (new BBO) β |
| β 7. KafkaBus.publishMarketData() β eunex.md * β |
| β 8. IACA fragments β IacaAggregator β |
| β β |
| β * only when EUNEX_KAFKA_BROKERS is set β |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β β |
| ExecReportEvent TradeEvent |
| β β |
| βΌ βΌ |
| βββββββββββββββββββββ ββββββββββββββββββββββββ |
| β OEGActor β β ClearingHouseActor β |
| β fanout to: β β β’ map sessionβmemberβ |
| β β’ FIXAcceptor β β β’ update capital β |
| β β’ AITrader β β β’ update holdings β |
| ββββββββββ¬βββββββββββ ββββββββββββββββββββββββ |
| β |
| FIX 35=8 (ExecReport) |
| β |
| βΌ |
| βββββββββββββββββββββ |
| β Client β |
| β receives fill β |
| βββββββββββββββββββββ |
| ``` |
|
|
| ### Market Data Flow |
|
|
| ``` |
| MECoreActor ββTradeEventβββββββββββββββΊ MDGActor |
| ββBookUpdateEventβββββββββββΊ β |
| ββpublishTrade()βββββββββββββΊ KafkaBus β eunex.trades * |
| ββpublishMarketData()ββββββββΊ KafkaBus β eunex.market-data * |
| β |
| βΌ |
| ββββββββββββββββββββββββ |
| β MarketDataSnapshot β |
| β per SymbolIndex_t: β |
| β β |
| β β’ bestBid, bestAsk β |
| β β’ lastTradePrice β |
| β β’ lastTradeQty β |
| β β’ totalBidQty β |
| β β’ totalAskQty β |
| β β’ tradeCount β |
| β β’ updateTime (ns) β |
| β β |
| β recentTrades[] β |
| β (last 200 trades) β |
| ββββββββββββββββββββββββ |
| β |
| getSnapshot() / getRecentTrades() |
| (thread-safe, mutex-protected) |
| β |
| βββββββββββββββββ΄ββββββββββββββββ |
| β β |
| βΌ βΌ |
| βββββββββββββββββββββ βββββββββββββββββββββββββ |
| β Dashboard :8090 β β Clearing House :8091 β |
| β (Flask + SSE) β β (Flask) β |
| β β β β |
| β Order Book view β β getLeaderboard() β |
| β Trade charts β β Member P&L, holdingsβ |
| β OHLCV history β β β |
| βββββββββββββββββββββ βββββββββββββββββββββββββ |
| β β |
| βββββββββββββ¬ββββββββββββββββββββ |
| β |
| nginx :7860 |
| (reverse proxy) |
| β |
| βΌ |
| Browser |
| |
| * KafkaBus publishes only when EUNEX_KAFKA_BROKERS is set at runtime |
| ``` |
|
|
| --- |
|
|
| ## 6. Core Components |
|
|
| ### 6.1 SimplxShim β Actor Framework (`src/engine/SimplxShim.hpp`) |
|
|
| The actor engine emulates the [Tredzone Simplx](https://github.com/Tredzone/simplx) framework API: |
|
|
| | Simplx Concept | EuNEx Implementation | |
| |----------------------|-----------------------------------------------| |
| | `Actor` | Base class with event handlers, mailbox | |
| | `Event::Pipe` | Lock-free cross-actor channel | |
| | `ActorId` | `{id: uint64, coreId: uint8}` | |
| | `Engine` | Multi-threaded scheduler with core affinity | |
| | `Callback` | Timer-like periodic invocation | |
| | `AsyncService` | Service locator pattern | |
|
|
| **Two operating modes:** |
| - **Synchronous** (tests): events delivered inline, single thread |
| - **Threaded** (`Engine::runFor`): per-core OS threads, mailbox queues |
|
|
| ``` |
| Synchronous Mode (unit tests) Threaded Mode (Engine::runFor) |
| βββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββ |
| |
| ActorA.pipe.push<Event>() ActorA.pipe.push<Event>() |
| β β |
| βΌ βΌ |
| ActorB.onEvent() β inline Mailbox[coreB].enqueue(Ξ») |
| β |
| Core B thread: |
| mbox.waitAndDrain() |
| β |
| βΌ |
| ActorB.onEvent() |
| ``` |
|
|
| ### 6.2 Types (`src/common/Types.hpp`) |
|
|
| ```cpp |
| Price_t = int64_t // Fixed-point, PRICE_SCALE = 100'000'000 |
| Quantity_t = uint64_t |
| OrderId_t = uint64_t // Exchange-assigned order ID |
| ClOrdId_t = uint64_t // Client order ID |
| SymbolIndex_t = uint32_t // Instrument identifier |
| TradeId_t = uint64_t |
| SessionId_t = uint16_t // Client session identifier |
| MemberId_t = uint16_t // Clearing member identifier |
| Timestamp_ns = uint64_t // Nanosecond timestamp |
| ``` |
|
|
| **Price conversion:** |
| ```cpp |
| Price_t px = toFixedPrice(150.25); // β 15'025'000'000 |
| double d = toDouble(px); // β 150.25 |
| ``` |
|
|
| ### 6.3 Order & Trade Structs |
|
|
| ``` |
| Order (88 bytes, packed) Trade (72 bytes, packed) |
| βββββββββββββββββββββββ βββββββββββββββββββββββ |
| orderId : uint64 tradeId : uint64 |
| clOrdId : uint64 symbolIdx : uint32 |
| symbolIdx : uint32 price : int64 |
| side : uint8 (Buy=1,Sell=2) quantity : uint64 |
| ordType : uint8 (Market=1,Lim=2) buyOrderId : uint64 |
| tif : uint8 (Day/GTC/IOC/FOK) sellOrderId : uint64 |
| price : int64 buyClOrdId : uint64 |
| quantity : uint64 sellClOrdId : uint64 |
| remainingQty : uint64 matchTime : uint64 |
| entryTime : uint64 buySessionId : uint16 |
| sessionId : uint16 sellSessionId : uint16 |
| status : uint8 |
| ``` |
|
|
| --- |
|
|
| ## 7. Event System |
|
|
| Events flow between actors via `Event::Pipe`. Each event struct inherits `tredzone::Actor::Event`: |
|
|
| ``` |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β EVENT CATALOG β |
| β β |
| β NewOrderEvent OEG β ME New order submission β |
| β CancelOrderEvent OEG β ME Cancel resting order β |
| β ModifyOrderEvent OEG β ME Modify price/qty (cancel-replace) β |
| β ExecReportEvent ME β OEG Ack, fill, partial, reject β |
| β TradeEvent ME β MDG/CH Trade execution β |
| β BookUpdateEvent ME β MDG/AI BBO + depth snapshot β |
| β RecoveryFragmentEvent ME β Persist Recovery persistence β |
| β β |
| β Direction Key: β |
| β OEG = OEGActor ME = MECoreActor MDG = MDGActor β |
| β CH = ClearingHouse AI = AITraderActor β |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| ### Event::Pipe Usage Pattern |
|
|
| ```cpp |
| // In MECoreActor constructor: |
| oePipe_ = Event::Pipe(*this, oeGatewayId); // pipe to OEG |
| mdPipe_ = Event::Pipe(*this, marketDataId); // pipe to MDG |
| |
| // Pushing an event (lock-free, zero-copy in same-core mode): |
| oePipe_.push<ExecReportEvent>(report, sessionId); |
| mdPipe_.push<TradeEvent>(trade); |
| ``` |
|
|
| --- |
|
|
| ## 8. Order Book & Matching |
|
|
| ### Book (`src/common/Book.hpp`) |
|
|
| The `Book` class implements price-time priority matching: |
|
|
| ``` |
| BIDS (sorted descending) ASKS (sorted ascending) |
| ββββββββββββββββββββββββ ββββββββββββββββββββββββ |
| |
| Price β Orders (FIFO) Price β Orders (FIFO) |
| ββββββββββΌββββββββββββββ ββββββββββΌββββββββββββββ |
| 155.00 β [100] [50] 157.00 β [200] |
| 154.50 β [200] 158.00 β [100] [150] |
| 154.00 β [75] [100] [25] 159.00 β [300] |
| 153.00 β [500] 160.00 β [50] |
| |
| std::map<Price_t, std::map<Price_t, |
| vector<Order>, vector<Order>, |
| std::greater<>> std::less<>> |
| ``` |
|
|
| **Matching algorithm:** |
|
|
| ``` |
| Incoming BUY @ 158.00, qty=250 |
| ββββββββββββββββββββββββββββββ |
| |
| Step 1: Match vs best ask 157.00 (qty=200) |
| β Trade: 200 @ 157.00 |
| β Ask level 157.00 exhausted |
| |
| Step 2: Match vs next ask 158.00 (qty=100 + 150) |
| β Trade: 50 @ 158.00 (from first order, partial) |
| β Remaining incoming qty = 0 β DONE |
| |
| Step 3: Incoming fully filled |
| β ExecReport: status=Filled, filledQty=250 |
| ``` |
|
|
| **Order types:** |
| - `Limit`: rests on book if no match; price-time priority |
| - `Market`: sweeps all levels; never rests (IOC behavior) |
|
|
| **Time-in-force:** |
| - `Day`: rests until end of session |
| - `IOC`: fill what's available, cancel remainder |
| - `FOK`: fill all or reject entirely |
|
|
| ### MECoreActor (`src/actors/MECoreActor.hpp`) |
|
|
| ``` |
| NewOrderEvent βββΊ MECoreActor.onEvent() |
| β |
| βββ RecoveryProxy.cause(persistenceId, order, Ξ») |
| β βββ Fragment persisted to store |
| β |
| βββ book_.newOrder(order, onTrade, onExec) |
| β βββ matchBuy() / matchSell() |
| β β βββ for each fill: onTrade(trade) |
| β βββ onExec(report) |
| β |
| βββ oePipe_.push<ExecReportEvent>(report, session) |
| βββ mdPipe_.push<TradeEvent>(trade) [per fill] |
| βββ mdPipe_.push<BookUpdateEvent>(snapshot) [if book changed] |
| βββ chPipe_.push<TradeEvent>(trade) [per fill, if CH wired] |
| ``` |
|
|
| --- |
|
|
| ## 9. Recovery & IACA |
|
|
| ### RecoveryProxy (`src/recovery/RecoveryProxy.hpp`) |
|
|
| ``` |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β RecoveryProxy (per MECoreActor) β |
| β β |
| β cause(persistenceId, payload, callback) β |
| β β persist Fragment to FragmentStore β |
| β β callback returns nextCount (# children expected) β |
| β β always executes (Master AND Mirror) β |
| β β |
| β effect(fn, args...) β |
| β β executes fn only on MASTER β |
| β β side effects: send ExecReports, publish trades β |
| β β |
| β recoveryEffect(fn, args...) β |
| β β executes fn only on MIRROR (during failover replay) β |
| β β |
| β ββββββββββββββββββ ββββββββββββββββββ β |
| β β MASTER β β MIRROR β β |
| β β cause() β β ββββββββΊ β cause() β β β |
| β β effect() β β Fragment β effect() β β β |
| β β recEff() β β Store β recEff() β β β |
| β ββββββββββββββββββ (shared) ββββββββββββββββββ β |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| ### IACA Fragment Chains (`src/iaca/IacaAggregator.hpp`) |
|
|
| ``` |
| Chain Completion Algorithm: |
| ββββββββββββββββββββββββββ |
| |
| Fragment 1 (ROOT: NEW_ORDER_BUY) nextCount = 2 |
| βββ Fragment 2 (ACK_DATA) nextCount = 0 |
| βββ Fragment 3 (TRADE_DATA) nextCount = 0 |
| |
| Completion check: |
| sum(nextCount) = 2 + 0 + 0 = 2 |
| total fragments = 3 |
| complete when: sum == total - 1 β 2 == 3 - 1 β |
| |
| On completion: |
| β NewOrderHandler fires |
| β Generates IA SBE message (future: IDS/PTB/SATURN) |
| ``` |
|
|
| --- |
|
|
| ## 10. Kafka Bus |
|
|
| ### Overview |
|
|
| The Kafka Bus (`src/persistence/KafkaBus.hpp`) mirrors the Optiq KFK (Kafka Bus) that connects the Matching Engine to downstream consumers: MDG, PTB, Clearing, IDS, and SATURN. |
|
|
| When `EUNEX_USE_KAFKA` is enabled at compile time and `EUNEX_KAFKA_BROKERS` is set at runtime, the bus publishes events to Kafka topics in real time. When disabled, a no-op stub compiles in its place so the engine runs standalone. |
|
|
| ### Topics |
|
|
| ``` |
| Topic Content Key |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| eunex.orders Raw Order structs symbolIdx |
| eunex.trades Trade structs symbolIdx |
| eunex.market-data BBO snapshots symbolIdx |
| eunex.recovery.fragments Recovery fragments originId:originKey |
| eunex.control Control messages (reserved) |
| ``` |
|
|
| ### Architecture |
|
|
| ``` |
| MECoreActor (per symbol) |
| β |
| ββ onEvent(NewOrderEvent) |
| β ββ kafka_->publishOrder(order) β eunex.orders |
| β ββ onTrade callback: |
| β β ββ kafka_->publishTrade(trade) β eunex.trades |
| β β ββ mdPipe_ / chPipe_ (actors) |
| β ββ publishBookUpdate() |
| β ββ kafka_->publishMarketData(...) β eunex.market-data |
| β |
| ββ KafkaBus* kafka_ (nullptr when disabled) |
| ``` |
|
|
| ### Compile-Time Toggle |
|
|
| ```cmake |
| cmake .. -DEUNEX_USE_KAFKA=ON # requires librdkafka-dev |
| cmake .. -DEUNEX_USE_KAFKA=OFF # no-op stub (default) |
| ``` |
|
|
| ### Runtime Configuration |
|
|
| Set `EUNEX_KAFKA_BROKERS` environment variable: |
|
|
| ```bash |
| export EUNEX_KAFKA_BROKERS=kafka:9092 |
| ./eunex_me |
| ``` |
|
|
| The engine prints Kafka connection status at startup and publishes cumulative stats (orders, trades, market-data messages) every 10 rounds. |
|
|
| ### Docker Deployment |
|
|
| `docker/docker-compose.yml` runs Kafka in KRaft mode (no ZooKeeper) using `apache/kafka:3.9.0`, creates all topics via `kafka-init`, then starts the engine with `EUNEX_KAFKA_BROKERS=kafka:9092`. |
|
|
| ```bash |
| cd docker |
| docker compose up --build |
| ``` |
|
|
| --- |
|
|
| ## 11. FIX Protocol Gateway |
|
|
| ### FIXAcceptorActor (`src/actors/FIXAcceptorActor.hpp`) |
|
|
| ``` |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β FIXAcceptorActor (TCP :9001) β |
| β β |
| β ββ Accept Thread βββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β while(running): β β |
| β β sock = accept(listenSock) β β |
| β β create FIXSession { sock, sessionId, senderCompId } β β |
| β β spawn client recv thread β β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β |
| β ββ Client Recv Thread (per session) ββββββββββββββββββββββββββββββββ β |
| β β while(loggedOn): β β |
| β β data = recv(sock) β β |
| β β msgs = parseFIXMessages(data) β β |
| β β for msg in msgs: β β |
| β β switch(msg[35]): β β |
| β β "A" β handleLogon() β β |
| β β "D" β handleNewOrderSingle() β push NewOrderEvent β OEG β β |
| β β "F" β handleCancelRequest() β push CancelOrderEvent β β |
| β β "G" β handleCancelReplace() β push ModifyOrderEvent β β |
| β β "5" β logout β β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β |
| β ββ Actor Thread (onEvent) ββββββββββββββββββββββββββββββββββββββββββ β |
| β β onEvent(ExecReportEvent): β β |
| β β find session by sessionId β β |
| β β build FIX ExecutionReport (35=8) β β |
| β β send() over TCP to client β β |
| β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| **Supported FIX messages:** |
|
|
| | Tag 35 | Message | Direction | |
| |--------|-----------------------------|-----------| |
| | A | Logon | Inbound | |
| | 5 | Logout | Inbound | |
| | 0 | Heartbeat | Inbound | |
| | D | NewOrderSingle | Inbound | |
| | F | OrderCancelRequest | Inbound | |
| | G | OrderCancelReplaceRequest | Inbound | |
| | 8 | ExecutionReport | Outbound | |
|
|
| **Symbol mapping:** |
|
|
| | Symbol String | SymbolIndex_t | |
| |---------------|---------------| |
| | AAPL | 1 | |
| | MSFT | 2 | |
| | GOOGL | 3 | |
| | TSLA | 4 | |
| | NVDA | 5 | |
| | AMD | 6 | |
| | ENX | 7 | |
| |
| --- |
| |
| ## 12. Clearing House |
| |
| ### ClearingHouseActor (`src/actors/ClearingHouseActor.hpp`) |
| |
| ``` |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β ClearingHouseActor β |
| β β |
| β Session-to-Member Mapping: β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β SessionId 100-109 βββΊ MemberId 1-10 (FIX gateway clients) β β |
| β β SessionId 200-209 βββΊ MemberId 1-10 (AI traders) β β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β |
| β On TradeEvent: β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β 1. Resolve buySessionId β buyMember β β |
| β β 2. Resolve sellSessionId β sellMember β β |
| β β 3. Buy side: β β |
| β β capital -= price Γ quantity β β |
| β β holdings[symbol].quantity += quantity β β |
| β β holdings[symbol].avgCost updated (weighted average) β β |
| β β 4. Sell side: β β |
| β β capital += price Γ quantity β β |
| β β holdings[symbol].quantity -= quantity β β |
| β β 5. Both sides: tradeCount++ β β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β |
| β Members (10): β |
| β ββββββββββ¬βββββββββββ¬ββββββββββββββββββ¬βββββββββ β |
| β β Name β Capital β Holdings β Trades β β |
| β ββββββββββΌβββββββββββΌββββββββββββββββββΌβββββββββ€ β |
| β β MBR01 β 100000.0 β {symβqty,cost} β 0 β β |
| β β MBR02 β 100000.0 β {symβqty,cost} β 0 β β |
| β β ... β ... β ... β ... β β |
| β β MBR10 β 100000.0 β {symβqty,cost} β 0 β β |
| β ββββββββββ΄βββββββββββ΄ββββββββββββββββββ΄βββββββββ β |
| β β |
| β getLeaderboard(): β |
| β sorted by capital descending β |
| β returns: memberId, name, capital, pnl, tradeCount, holdingCount β |
| β thread-safe (mutex) for Python bridge reads β |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
| |
| --- |
| |
| ## 13. AI Trading Members |
| |
| ### AITraderActor (`src/actors/AITraderActor.hpp`) |
| |
| ``` |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β AITraderActor β 10 automated trading members β |
| β β |
| β Members: β |
| β ββββββββββ¬ββββββββββββ¬βββββββββββββββββ¬βββββββββββββββ β |
| β β Name β SessionId β Strategy β Description β β |
| β ββββββββββΌββββββββββββΌβββββββββββββββββΌβββββββββββββββ€ β |
| β β MBR01 β 200 β Momentum β Follow trend β β |
| β β MBR02 β 201 β MeanReversion β Fade moves β β |
| β β MBR03 β 202 β Random β Noise trader β β |
| β β MBR04 β 203 β Momentum β β β |
| β β ... β ... β (rotates) β β β |
| β β MBR10 β 209 β Random β β β |
| β ββββββββββ΄ββββββββββββ΄βββββββββββββββββ΄βββββββββββββββ β |
| β β |
| β Strategy Details: β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β β β |
| β β MOMENTUM: β β |
| β β if last N prices trending up β BUY (follow the trend) β β |
| β β if last N prices trending down β SELL β β |
| β β price = BBO Β± small offset β β |
| β β β β |
| β β MEAN REVERSION: β β |
| β β if price > moving average β SELL (expect revert) β β |
| β β if price < moving average β BUY β β |
| β β price = BBO Β± small offset β β |
| β β β β |
| β β RANDOM: β β |
| β β random side (50/50), random qty (10-100) β β |
| β β price around midpoint with random spread β β |
| β β β β |
| β β LLM (Python CH only): β β |
| β β sends portfolio + market data to LLM β β |
| β β expects JSON response: symbol, side, quantity, price, reason β β |
| β β uses same provider chain as AI Analyst β β |
| β β falls back to random on LLM failure β β |
| β β explanations shown in CH dashboard panel β β |
| β β β β |
| β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β |
| β β |
| β Data Sources: β |
| β β’ BookUpdateEvent β BBO (bestBid, bestAsk) per symbol β |
| β β’ TradeEvent β price history (last 50 prices) per symbol β |
| β β’ ExecReportEvent β order fill confirmations β |
| β β |
| β Output: β |
| β β’ NewOrderEvent β OEGActor (via Event::Pipe) β |
| β β’ ~30s intervals via Actor::Callback β |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
| |
| --- |
| |
| ## 14. Market Simulation |
| |
| Both the C++ engine and the Python dashboard include market simulation to continuously generate orders and trades. |
| |
| ### C++ Engine (AITraderActor) |
| |
| The `AITraderActor` generates orders every ~3 seconds per round. Each of the 10 AI members picks a random symbol and applies its strategy (Momentum, Mean Reversion, or Random). All strategies use per-symbol **reference prices** matching real market levels: |
| |
| | Symbol | Reference Price | |
| |--------|----------------| |
| | AAPL | $154.00 | |
| | MSFT | $324.00 | |
| | GOOGL | $141.00 | |
| | TSLA | $375.00 | |
| | NVDA | $201.00 | |
| | AMD | $320.00 | |
| | ENX | β¬146.00 | |
| |
| When no BBO data is available yet, the fallback `submitOrder()` generates prices within Β±3 ticks of the reference price instead of random values. |
| |
| ### Python Dashboard (MarketSimulator) |
| |
| The dashboard runs a `MarketSimulator` thread that generates orders when the session status is `"active"`: |
| |
| ``` |
| Configuration (shared/config.py): |
| SIM_INTERVAL = 30s (env: EUNEX_SIM_INTERVAL) |
| SIM_ORDERS_PER_ROUND = 4 (env: EUNEX_SIM_ORDERS) |
| |
| Per round (every SIM_INTERVAL seconds): |
| For each of 7 symbols: |
| 1. Submit SIM_ORDERS_PER_ROUND limit orders near current mid price (Β±0.5%) |
| 2. Submit 1 crossing order that will match against the book |
| β Generates resting orders + at least 1 trade per symbol per round |
| ``` |
| |
| **Order flow:** |
| - Session "Start Day" β seeds initial orders (2 bid + 2 ask per symbol) |
| - Simulation thread wakes every 30s β submits ~35 orders β ~7 trades |
| - All trades are persisted to SQLite (trades table + OHLCV aggregation) |
| - Dashboard chart shows candlestick data from OHLCV buckets |
| |
| ### Seed Orders (Session Start) |
| |
| When the dashboard session starts, 4 orders are seeded per symbol to establish a market: |
| |
| ``` |
| For each symbol at startPrice: |
| Buy @ startPrice - 1.00, qty=100 |
| Buy @ startPrice - 0.50, qty=150 |
| Sell @ startPrice + 0.50, qty=200 |
| Sell @ startPrice + 1.00, qty=100 |
| ``` |
| |
| ### Daily Close Persistence |
| |
| When the session ends ("End Day"), closing prices are saved to the `daily_close` SQLite table: |
|
|
| ``` |
| daily_close table: |
| symbol TEXT, trade_date TEXT, close_price REAL, bid REAL, ask REAL, |
| volume INTEGER, trade_count INTEGER |
| PRIMARY KEY (symbol, trade_date) |
| ``` |
|
|
| On the next "Start Day", the engine loads the most recent closing prices from `daily_close` and uses them as seed prices instead of the hardcoded `startPrice` from config. This ensures price continuity across trading sessions. |
|
|
| **Flow:** |
| 1. Session "End Day" β `_save_closing_prices()` writes current snapshot to `daily_close` |
| 2. Next "Start Day" β `_seed_initial_orders()` calls `get_last_closing_prices()` from DB |
| 3. If closing prices exist β seed around those prices |
| 4. If no history β fall back to `startPrice` from `shared/config.py` |
|
|
| The simulator also loads closing prices on startup via `_load_closing_refs()`. |
|
|
| ### Ticker Tape |
|
|
| The dashboard displays a scrolling ticker tape below the header showing all 7 symbols with: |
| - Current price, percentage change (vs. previous price), volume |
| - Green/red arrows for up/down movement |
| - Auto-updates via SSE snapshot events |
|
|
| ### OHLCV Candlestick Chart |
|
|
| The price chart renders OHLCV data as candlestick bars using a custom Chart.js plugin: |
| - Green candles (close >= open), red candles (close < open) |
| - High/low wicks drawn as vertical lines |
| - Volume bars on secondary axis with matching colors |
| - Tooltip shows full OHLCV values per bar |
|
|
| --- |
|
|
| ## 15. AI Market Analyst |
|
|
| The dashboard includes an AI analyst powered by local or cloud LLM providers. |
|
|
| ### Provider Fallback Chain |
|
|
| ``` |
| Auto mode tries in order: |
| 1. Ollama (local) β POST {OLLAMA_HOST}/api/chat |
| 2. Groq (cloud) β POST api.groq.com/openai/v1/chat/completions |
| 3. HuggingFace β POST router.huggingface.co/v1/chat/completions |
| ``` |
|
|
| ### Environment Variables |
|
|
| | Variable | Default | Description | |
| |---------------|-------------------------------|--------------------------| |
| | OLLAMA_HOST | http://localhost:11434 | Ollama API endpoint | |
| | OLLAMA_MODEL | llama3.2:3b | Default Ollama model | |
| | GROQ_API_KEY | (empty) | Groq API key | |
| | GROQ_MODEL | llama-3.1-8b-instant | Default Groq model | |
| | HF_TOKEN | (empty) | HuggingFace token | |
| | HF_MODEL | Qwen/Qwen2.5-7B-Instruct | Default HF model | |
| |
| ### Prompt Template |
| |
| The analyst builds a market context prompt from current trades and order book: |
| |
| ``` |
| You are a concise financial market analyst for the EuNEx simulated exchange. |
| Time: HH:MM:SS | Session: ACTIVE |
| |
| Recent trades: |
| AAPL: 12 trade(s), range 153.50-154.50, vol 1200, last 154.10 |
| MSFT: 8 trade(s), range 323.80-324.20, vol 800, last 324.00 |
| ... |
| |
| Order book: |
| AAPL: Bid 153.80 / Ask 154.20 (spread 0.40) |
| ... |
| |
| In 3-4 sentences: activity level, notable moves, market sentiment. |
| ``` |
| |
| ### API Endpoints |
| |
| | Endpoint | Method | Description | |
| |-------------------|--------|--------------------------------| |
| | /ai/generate | POST | Trigger async LLM generation | |
| | /ai/insights | GET | Get cached insight history | |
| | /ai/config | GET | Provider availability/status | |
| | /ai/select | POST | Switch provider/model | |
| |
| Insights are broadcast via SSE `ai_insight` event and cached in memory (last 20). |
|
|
| --- |
|
|
| ## 16. Developer Message Flow Visualizer |
|
|
| The dashboard includes a "Message Flow" tab that traces the full order lifecycle through all system components. This is a developer tool for understanding and debugging the Optiq-modeled pipeline. |
|
|
| ### Pipeline Stages |
|
|
| ``` |
| βββββββ ββββββββ βββββββββ βββββββββ ββββββ ββββββ |
| β OEG β ββΊ β Book β ββΊ β Match β ββΊ β Trade β ββΊ β DB β ββΊ β CH β |
| βββββββ ββββββββ βββββββββ βββββββββ ββββββ ββββββ |
| Order Insert Fill Record SQLite Clear |
| Entry /Status Partial Trade Persist House |
| ``` |
|
|
| Each stage logs a timestamped message with detail text. Messages flow via SSE `msgflow` events for real-time display. |
|
|
| ### Implementation |
|
|
| The visualizer uses Python function patching to intercept the matching engine methods: |
| - `engine.submit_order()` β logs OEG (entry) + Book (status) + Match (fill) |
| - `engine.cancel_order()` β logs OEG (cancel) + Book (cancelled) |
| - `broadcast_event("trade", ...)` β logs Trade + DB + CH steps |
|
|
| ### API |
|
|
| | Endpoint | Method | Description | |
| |-------------------|--------|---------------------------------| |
| | /dev/messages | GET | Get recent message log (500 max)| |
|
|
| ### UI Features |
|
|
| - **Pipeline counter**: shows cumulative message count per stage |
| - **Live log**: new messages appear at top with highlight animation |
| - **Clear**: resets all counters and log entries |
|
|
| --- |
|
|
| ## 17. Engine Mode Switch |
|
|
| The dashboard supports two matching engine backends, switchable at runtime via a toggle in the header. |
|
|
| ### Architecture |
|
|
| ``` |
| ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β Dashboard (Flask :8090) β |
| β β |
| β βββββββββββββββββββββββββββββββ βββββββββββββββββββββββββββββββ β |
| β β Python MatchingEngine β β CppEngineBridge β β |
| β β (built-in, default) β β (FIX 4.4 TCP client) β β |
| β β β β β β |
| β β β’ In-process order book β β β’ Connects to :9001 β β |
| β β β’ Price-time priority β β β’ Sends NewOrderSingle(D) β β |
| β β β’ No compilation needed β β β’ Sends CancelRequest (F) β β |
| β β β’ SQLite persistence β β β’ Sends CancelReplace (G) β β |
| β β β’ Good for demos β β β’ Receives ExecReport (8) β β |
| β ββββββββββββββββββββββββββββββββ β β’ Logon/Logout/Heartbeat β β |
| β ββββββββββββββββ¬ββββββββββββββββ β |
| β _active_engine() β β |
| β returns engine or cpp_bridge β β |
| β based on engine_mode variable β β |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββΌββββββββββββββββββββββ |
| β |
| FIX 4.4 TCP :9001 |
| β |
| βΌ |
| βββββββββββββββββββββββββββββ |
| β eunex_me (C++ engine) β |
| β FIXAcceptorActor :9001 β |
| β OEG β MECore β MDG β CH β |
| βββββββββββββββββββββββββββββ |
| ``` |
|
|
| ### Switching Modes |
|
|
| **From the UI:** |
| The header contains a toggle switch `[Python βββββ C++]` with a status dot: |
| - **Green dot**: C++ engine is running and reachable on port 9001 |
| - **Red dot**: C++ engine not detected |
| - Health check runs every 15 seconds |
|
|
| **From the API:** |
| ```bash |
| # Check current mode and C++ availability |
| GET /engine/mode |
| β {"mode": "python", "cpp_available": true, "cpp_host": "localhost", "cpp_port": 9001} |
| |
| # Switch to C++ engine |
| POST /engine/mode {"mode": "cpp"} |
| β {"mode": "cpp"} |
| |
| # Switch back to Python |
| POST /engine/mode {"mode": "python"} |
| β {"mode": "python"} |
| ``` |
|
|
| ### Order Routing |
|
|
| When `engine_mode == "cpp"`: |
| 1. Dashboard builds a FIX `NewOrderSingle` (35=D) message |
| 2. Sends it over TCP to `eunex_me:9001` |
| 3. C++ engine matches and returns `ExecutionReport` (35=8) |
| 4. Bridge parses the response and updates dashboard state (orders, trades, snapshots) |
| 5. SSE events broadcast to browser as usual |
|
|
| When `engine_mode == "python"`: |
| - Orders processed by the built-in `MatchingEngine` class (default behavior) |
|
|
| **Simulation orders** (`source=sim`, `source=seed`) always use the Python engine regardless of mode. |
|
|
| ### FIX Message Flow (C++ Mode) |
|
|
| ``` |
| Browser Dashboard C++ Engine |
| βββββββ βββββββββ ββββββββββ |
| β POST /order/new β β |
| β ββββββββββββββββββΊ β β |
| β β 35=D NewOrderSingle β |
| β β βββββββββββββββββββββΊ β |
| β β Book::newOrder() |
| β β match β Trade |
| β β 35=8 ExecutionReport β |
| β β βββββββββββββββββββββ β |
| β β update state, SSE β |
| β SSE: order, trade β β |
| β ββββββββββββββββββ β β |
| ``` |
|
|
| ### Connection Lifecycle |
|
|
| ``` |
| Toggle ON (β C++) Toggle OFF (β Python) |
| βββββββββββββββββββ ββββββββββββββββββββββββ |
| 1. TCP connect(:9001) 1. Send FIX Logout (35=5) |
| 2. Send FIX Logon (35=A) 2. Close TCP socket |
| 3. Start recv thread 3. Set mode = "python" |
| 4. Set mode = "cpp" 4. Orders route to MatchingEngine |
| 5. Orders route to bridge |
| ``` |
|
|
| --- |
|
|
| ## 18. Project Structure |
|
|
| ``` |
| EuNEx/ |
| βββ CMakeLists.txt Build configuration |
| βββ README.md Project overview |
| β |
| βββ src/ |
| β βββ main.cpp Engine entry point, actor wiring |
| β β |
| β βββ common/ |
| β β βββ Types.hpp Core types (Price_t, Order, Trade, etc.) |
| β β βββ Book.hpp Order book interface |
| β β βββ Book.cpp Price-time priority matching engine |
| β β |
| β βββ engine/ |
| β β βββ SimplxShim.hpp Actor framework (Simplx API compatible) |
| β β |
| β βββ actors/ |
| β β βββ Events.hpp All inter-actor event definitions |
| β β βββ OEGActor.hpp/cpp Order Entry Gateway |
| β β βββ MECoreActor.hpp/cpp Matching Engine core (per symbol) |
| β β βββ MDGActor.hpp/cpp Market Data Gateway |
| β β βββ FIXAcceptorActor.hpp/cpp FIX 4.4 TCP protocol gateway |
| β β βββ ClearingHouseActor.hpp/cpp Trade clearing & member positions |
| β β βββ AITraderActor.hpp/cpp Automated trading members |
| β β |
| β βββ net/ |
| β β βββ SocketCompat.hpp Cross-platform socket abstraction |
| β β |
| β βββ recovery/ |
| β β βββ RecoveryProxy.hpp/cpp Master/Mirror recovery gating |
| β β βββ (FragmentStore) In-memory persistence (β Kafka future) |
| β β |
| β βββ iaca/ |
| β β βββ Fragment.hpp IACA fragment definitions |
| β β βββ IacaAggregator.hpp/cpp Fragment chain assembly & completion |
| β β βββ (handlers) IA message generators |
| β β |
| β βββ persistence/ |
| β βββ PersistenceStore.hpp Store interface |
| β βββ KafkaStore.hpp Kafka adapter (optional, librdkafka) |
| β |
| βββ dashboard/ |
| β βββ app.py Flask dashboard + dual engine bridge |
| β βββ database.py SQLite (orders, trades, OHLCV) |
| β βββ templates/index.html Trading UI with engine toggle |
| β |
| βββ clearing_house/ |
| β βββ app.py Flask CH portal + API |
| β βββ ch_database.py SQLite (members, holdings) |
| β βββ ch_ai_trader.py AI trading (momentum/mean_revert/random/llm) |
| β βββ templates/ CH web UI (leaderboard, portfolios) |
| β |
| βββ fix_gateway/ |
| β βββ fix_server.py Python FIX 4.4 TCP acceptor |
| β |
| βββ shared/ |
| β βββ config.py Centralized config (auto-loads .env) |
| β |
| βββ .env LLM and service configuration |
| β |
| βββ tests/ |
| β βββ test_orderbook.cpp Book unit tests (26 cases) |
| β βββ test_matching_engine.cpp ME integration tests |
| β βββ test_threaded_engine.cpp Multi-threaded actor tests |
| β βββ test_clearing_house.cpp Clearing house tests (7 cases) |
| β βββ test_fix_gateway.cpp FIX gateway tests (5 cases) |
| β βββ test_ai_trader.cpp AI trader tests (6 cases) |
| β |
| βββ examples/ |
| β βββ ping_pong.cpp Basic actor communication demo |
| β βββ simple_match.cpp Simple order matching demo |
| β |
| βββ docs/ |
| βββ developers-guide.md This document |
| βββ process-diagram.md Detailed Optiq architecture diagrams |
| ``` |
|
|
| --- |
|
|
| ## 19. Build & Test |
|
|
| ### Prerequisites |
|
|
| - **C++20 compiler**: MSVC 19.30+, GCC 12+, Clang 15+ |
| - **CMake**: 3.16+ |
| - **Optional**: librdkafka (for Kafka persistence) |
|
|
| ### Build Commands |
|
|
| ```bash |
| # Configure (default: tests ON, Kafka OFF) |
| cmake -B build -DEUNEX_BUILD_TESTS=ON |
| |
| # Build |
| cmake --build build --config Release |
| |
| # Run tests |
| cd build && ctest -C Release --output-on-failure |
| |
| # Run engine |
| ./build/Release/eunex_me |
| ``` |
|
|
| ### CMake Options |
|
|
| | Option | Default | Description | |
| |---------------------|---------|------------------------------------| |
| | `EUNEX_USE_SIMPLX` | OFF | Use real Simplx framework | |
| | `EUNEX_USE_KAFKA` | OFF | Enable Kafka persistence | |
| | `EUNEX_BUILD_TESTS` | ON | Build test binaries | |
| | `EUNEX_BUILD_EXAMPLES` | ON | Build example binaries | |
|
|
| ### Test Suites |
|
|
| ``` |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| β Test Suite β Cases β What it verifies β |
| ββββββββββββββββββββββββββββΌββββββββΌββββββββββββββββββββββββββββββ€ |
| β OrderBookTest β 26 β Book matching, all TIFs, β |
| β β β multi-level sweeps, cancels β |
| ββββββββββββββββββββββββββββΌββββββββΌββββββββββββββββββββββββββββββ€ |
| β MatchingEngineTest β - β MECoreActor event handling β |
| ββββββββββββββββββββββββββββΌββββββββΌββββββββββββββββββββββββββββββ€ |
| β ThreadedEngineTest β - β Multi-core Engine, mailbox β |
| ββββββββββββββββββββββββββββΌββββββββΌββββββββββββββββββββββββββββββ€ |
| β ClearingHouseTest β 7 β Capital, holdings, P&L, β |
| β β β session mapping, leaderboardβ |
| ββββββββββββββββββββββββββββΌββββββββΌββββββββββββββββββββββββββββββ€ |
| β FIXGatewayTest β 5 β Symbol mapping, actor β |
| β β β lifecycle, OEG routing β |
| ββββββββββββββββββββββββββββΌββββββββΌββββββββββββββββββββββββββββββ€ |
| β AITraderTest β 6 β Strategy execution, multi- β |
| β β β symbol, clearing integrationβ |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ |
| ``` |
|
|
| --- |
|
|
| ## 20. Configuration |
|
|
| ### Runtime Configuration (main.cpp) |
|
|
| | Parameter | Value | Description | |
| |---------------------|-----------|--------------------------------------| |
| | FIX port | 9001 | FIXAcceptorActor TCP listen port | |
| | Symbols | 7 | AAPL, MSFT, GOOGL, TSLA, NVDA, AMD, ENX | |
| | AI members | 10 | MBR01-MBR10 | |
| | Initial capital | 100,000.0 | Per clearing member | |
| | AI trade interval | ~3s | Per round in main loop | |
| | Price scale | 10^8 | Fixed-point decimal places | |
|
|
| ### Seed Orders |
|
|
| The engine pre-populates order books with spread-defining orders: |
|
|
| ``` |
| AAPL: Sell 155.00/100, 154.50/200 | Buy 153.50/150, 153.00/100 |
| MSFT: Sell 325.00/100, 324.50/150 | Buy 323.50/200, 323.00/100 |
| GOOGL: Sell 142.00/100, 141.50/200 | Buy 140.50/150, 140.00/100 |
| TSLA: Sell 376.00/80, 375.50/120 | Buy 374.50/100, 374.00/80 |
| NVDA: Sell 202.00/100, 201.50/150 | Buy 200.50/120, 200.00/100 |
| AMD: Sell 321.00/90, 320.50/130 | Buy 319.50/110, 319.00/80 |
| ENX: Sell 147.00/60, 146.50/100 | Buy 145.50/80, 145.00/60 |
| ``` |
|
|
| --- |
|
|
| ## 21. Extending EuNEx |
|
|
| ### Adding a New Symbol |
|
|
| 1. Define a new `SymbolIndex_t` constant in `main.cpp` |
| 2. Create a `MECoreActor` for it, passing OEG, MDG, and CH actor IDs |
| 3. Register it with `oeGateway->mapSymbol(newSym, bookActor->getActorId())` |
| 4. Add it to `AITraderActor`'s symbol list |
| 5. Add seed orders if desired |
|
|
| ### Adding a New Actor |
|
|
| 1. Create `src/actors/MyActor.hpp` inheriting `tredzone::Actor` |
| 2. Register event handlers in constructor: `registerEventHandler<EventT>(*this)` |
| 3. Create `Event::Pipe` members for each destination actor |
| 4. Add `.cpp` to `EUNEX_CORE_SOURCES` in `CMakeLists.txt` |
| 5. Wire into topology in `main.cpp` |
|
|
| ### Adding a New Event Type |
|
|
| 1. Define the struct in `src/actors/Events.hpp` inheriting `tredzone::Actor::Event` |
| 2. Add `onEvent(const MyEvent&)` to receiving actors |
| 3. Register handler: `registerEventHandler<MyEvent>(*this)` |
| 4. Push via pipe: `pipe.push<MyEvent>(args...)` |
|
|
| ### Adding a New Trading Strategy |
|
|
| 1. Add enum value to `Strategy` in `AITraderActor.hpp` |
| 2. Implement `strategyMyStrategy()` method |
| 3. Add case to `submitOrder()` switch |
| 4. Assign to desired members in `initMembers()` |
|
|
| ### Future Roadmap |
|
|
| ``` |
| Current (v0.9.5) Planned |
| ββββββββββββββββ βββββββββββββββββββββββββββββββββββ |
| |
| β Limit + Market + Stop orders β‘ Pegged, Mid-Point, Iceberg |
| β IOC, FOK, Day β‘ GTD, GTC, VFU, VFCU |
| β Trading phases (PreOpen/CTS) β‘ TAL, VDO, Reserved phases |
| β FIX 4.4 gateway β‘ FIX 5.0 SP2 + SBE binary |
| β Kafka Bus persistence β‘ Kafka consumer replay |
| β SimplxShim (emulation) β‘ Real Simplx multi-core |
| β 7 symbols (real prices) β‘ Multi-segment, partitions |
| β Single partition β‘ Cross-partition routing |
| β Clearing house + AI traders β‘ EuroCCP/LCH integration |
| β IACA fragments β‘ IACA FINISH + COPY + IDS |
| β Python bridge (JSON) β‘ SBE multicast MDG |
| β Market simulation (C++ + Py) β‘ Multi-day backtesting |
| β Ticker tape + OHLCV charts β‘ SATURN ARM (MiFID II RTS 22) |
| β Daily close persistence β‘ PTB (Post-Trade Box) |
| β AI Analyst (Ollama/Groq/HF) |
| β LLM trading strategy (CH) |
| β Message Flow Visualizer |
| β Engine mode switch (Py β C++) |
| β .env auto-loading config |
| ``` |
|
|
| --- |
|
|
| *Generated for EuNEx v0.9.5 β Euronext Optiq Architecture Simulator* |
|
|