EuNEx / README.md
RayMelius's picture
Add HF Spaces metadata to README
ed3758a
metadata
title: EuNEx
emoji: πŸ“ˆ
colorFrom: green
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

EuNEx β€” Euronext Optiq Architecture Learning Project

C++ actor-based matching engine that mirrors the Euronext Optiq architecture, ported from the StockEx Python prototype.

Architecture Mapping

StockEx (Python/Kafka)              EuNEx (C++/Simplx)                 Optiq (Production)
─────────────────────               ──────────────────                  ──────────────────
fix_oeg_server.py        β†’   OEGActor                 β†’   OEActor
  Kafka 'orders' topic   β†’     Event::Pipe            β†’     Simplx Event::Pipe
matcher.py               β†’   MECoreActor              β†’   LogicalCoreActor + Book
  match_order()          β†’     Book::newOrder()        β†’     RecoveryCause β†’ IACA Cause β†’ forwardToBook
  handle_cancel()        β†’     Book::cancelOrder()     β†’     CancelOrderData
  handle_amend()         β†’     Book::modifyOrder()     β†’     ModifyOrderData
  Kafka 'trades' topic   β†’     TradeEvent via Pipe     β†’     IACA fragment chain
dashboard.py (SSE)       β†’   MDGActor                  β†’   MDLimitLogicalCoreHandler
  /orderbook/<sym>       β†’     BookUpdateEvent         β†’     PublishLimitUpdateRequest
  /trades                β†’     TradeEvent              β†’     IACA β†’ IA SBE message
database.py (SQLite)     β†’   RecoveryProxy (memory)    β†’   RecoveryProxy β†’ Kafka
  save_trade()           β†’     FragmentStore::append() β†’   PersistenceAgent β†’ Kafka produce
fix_oeg_server.py        β†’   FIXAcceptorActor          β†’   FIX 4.4 OEG Acceptor
  NewOrderSingle         β†’     35=D handling           β†’     Optiq FIX gateway
  ExecutionReport        β†’     35=8 response           β†’     Execution reports
ch_ai_trader.py          β†’   ClearingHouseActor        β†’   Clearing House (PTB path)
  AI strategies          β†’   AITraderActor             β†’   Trading obligations

Actor Topology (v0.9)

 Core 0: OEGActor + FIXAcceptorActor     ← Order entry & FIX protocol
 Core 1: MECoreActor (per symbol)         ← Matching engine (Book)
 Core 2: MDGActor                         ← Market data snapshots
 Core 3: ClearingHouseActor + AITrader    ← Post-trade & AI members

Dual Engine Architecture

The dashboard can run in two modes, switchable at runtime via a toggle in the UI:

                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚           Browser (:8090)          β”‚
                          β”‚     [Python ○────● C++] toggle     β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                         β”‚
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚       Dashboard (Flask :8090)      β”‚
                          β”‚                                    β”‚
                          β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚
                          β”‚  β”‚  Python ME  β”‚  β”‚ C++ Bridge  β”‚  β”‚
                          β”‚  β”‚MatchingEng β”‚  β”‚ FIX 4.4 TCP β”‚  β”‚
                          β”‚  β”‚(built-in)  β”‚  β”‚  client     β”‚  β”‚
                          β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜  β”‚
                          β”‚   mode=python      mode=cpp       β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                                     β”‚ FIX 4.4 TCP
                                                     β–Ό
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚       C++ Matching Engine             β”‚
                          β”‚       eunex_me (:9001)               β”‚
                          β”‚                                      β”‚
                          β”‚  OEG β†’ MECore β†’ MDG β†’ ClearingHouse  β”‚
                          β”‚  FIXAcceptorActor (TCP :9001)        β”‚
                          β”‚  Multi-threaded actors               β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
Mode Engine How orders are matched Use case
Python Built-in MatchingEngine In-process, same Flask app Demo, learning, no compilation needed
C++ eunex_me via FIX 4.4 TCP to C++ actors on port 9001 Production-like, microsecond latency

Service Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    nginx (:7860)                        β”‚
β”‚              Reverse proxy (Docker)                      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                   β”‚                   β”‚
         β–Ό                   β–Ό                   β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Dashboard     β”‚ β”‚ Clearing House   β”‚ β”‚  FIX Gateway     β”‚
β”‚   (:8090)       β”‚ β”‚ (:8091)          β”‚ β”‚  (:9001 TCP)     β”‚
β”‚                 β”‚ β”‚                  β”‚ β”‚                  β”‚
β”‚ Order Book      β”‚ β”‚ 10 AI Members   β”‚ β”‚ FIX 4.4 Acceptorβ”‚
β”‚ Trade Charts    β”‚ β”‚ Leaderboard     β”‚ β”‚ NewOrder/Cancel  β”‚
β”‚ OHLCV History   β”‚ β”‚ Portfolios      β”‚ β”‚ Amend/ExecRpt   β”‚
β”‚ SQLite DB       β”‚ β”‚ Settlements     β”‚ β”‚                  β”‚
β”‚ SSE Streaming   β”‚ β”‚                 β”‚ β”‚                  β”‚
β”‚ Engine Switch   β”‚ β”‚ LLM Trading     β”‚ β”‚                  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜
         β”‚                   β”‚                    β”‚
         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                             β”‚
                             β–Ό
               β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
               β”‚    C++ Matching Engine      β”‚
               β”‚    (eunex_me)               β”‚
               β”‚    Multi-threaded actors    β”‚
               β”‚    Price-time priority      β”‚
               β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Quick Start (Linux)

# Install dependency
pip install flask

# Start all services (dashboard + FIX gateway + clearing house)
./run.sh

# Services:
#   Dashboard:       http://localhost:8090
#   Clearing House:  http://localhost:8091
#   FIX Gateway:     localhost:9001 (TCP)

# Stop all
./run.sh stop

# Check status
./run.sh status

Quick Start (Docker β€” Windows/Linux)

cd docker
docker compose up --build

# All services via nginx:
#   Dashboard:       http://localhost:7860
#   Clearing House:  http://localhost:7860/ch/
#   Kafka:           localhost:9092

Build C++ Engine

cmake -B build -DEUNEX_BUILD_TESTS=ON
cmake --build build --config Release

# Run matching engine (FIX server on port 9001)
./build/Release/eunex_me

# Run all tests (7 suites)
cd build && ctest -C Release

# Use with dashboard: start eunex_me, then toggle "C++" in the dashboard header

With Kafka Persistence

# Compile with Kafka support (requires librdkafka-dev)
cmake -B build -DEUNEX_USE_KAFKA=ON
cmake --build build --config Release

# Run with Kafka (set broker address)
EUNEX_KAFKA_BROKERS=localhost:9092 ./build/Release/eunex_me

# Topics: eunex.orders, eunex.trades, eunex.market-data, eunex.recovery.fragments

Without EUNEX_USE_KAFKA, the engine compiles with a no-op stub and runs standalone.

FIX Gateway

The C++ engine includes a built-in FIX 4.4 acceptor on TCP port 9001. A Python FIX gateway is also available:

python fix_gateway/fix_server.py
python fix_gateway/fix_server.py test

Supports: NewOrderSingle (35=D), OrderCancelRequest (35=F), OrderCancelReplaceRequest (35=G), ExecutionReport (35=8).

Configuration (.env)

All Python services auto-load settings from .env in the project root. No extra packages needed β€” shared/config.py parses it at startup.

# .env β€” LLM and service configuration
LLM_PROVIDER=auto              # ollama | groq | hf | auto (try all)
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=llama3.1
HF_MODEL=Qwen/Qwen2.5-7B-Instruct
# HF_TOKEN=hf_...              # or auto-discovered from ~/.cache/huggingface/token
# GROQ_API_KEY=gsk_...
Variable Default Description
LLM_PROVIDER auto LLM backend: ollama, groq, hf, or auto (tries all in order)
OLLAMA_HOST http://localhost:11434 Ollama server URL
OLLAMA_MODEL llama3.1 Ollama model name
HF_TOKEN (auto-discovered) HuggingFace API token; reads ~/.cache/huggingface/token if not set
HF_MODEL Qwen/Qwen2.5-7B-Instruct HuggingFace model (free tier)
GROQ_API_KEY (empty) Groq Cloud API key
GROQ_MODEL llama-3.1-8b-instant Groq model name
EUNEX_DASHBOARD_PORT 8090 Dashboard HTTP port
EUNEX_CH_PORT 8091 Clearing House HTTP port
EUNEX_FIX_PORT 9001 FIX Gateway TCP port
EUNEX_KAFKA_BROKERS localhost:9092 Kafka broker address

AI Market Analyst

The Dashboard includes an AI-powered market analyst that generates real-time commentary on trading activity. It uses a fallback chain of LLM providers:

  1. Ollama (local) β€” best for development, requires Ollama installed
  2. Groq (cloud) β€” fast inference, requires GROQ_API_KEY
  3. HuggingFace (cloud) β€” free tier with HF_TOKEN, uses Qwen 2.5 7B

The dashboard shows a green/red status dot and lets you select the active Ollama model from a dropdown. Set LLM_PROVIDER=auto to try all providers in order until one succeeds.

Clearing House

10 AI trading members (MBR01-MBR10) with 4 strategies:

  • Momentum: follow price trends
  • Mean Reversion: fade price moves
  • Random: noise trading
  • LLM: AI-driven decisions with natural language explanations (uses same provider chain)

Features: capital tracking, holdings per symbol, P&L, leaderboard, strategy selection per member, LLM trading explanations panel.

Project Structure

EuNEx/
β”œβ”€β”€ src/                                # C++ matching engine
β”‚   β”œβ”€β”€ main.cpp                        # Entry point, actor wiring
β”‚   β”œβ”€β”€ engine/SimplxShim.hpp           # Multi-threaded actor engine
β”‚   β”œβ”€β”€ common/
β”‚   β”‚   β”œβ”€β”€ Types.hpp                   # Price, Order, Trade, enums
β”‚   β”‚   └── Book.hpp/cpp               # Price-time priority matching
β”‚   β”œβ”€β”€ actors/
β”‚   β”‚   β”œβ”€β”€ Events.hpp                  # Inter-actor event types
β”‚   β”‚   β”œβ”€β”€ OEGActor.hpp/cpp           # Order Entry Gateway
β”‚   β”‚   β”œβ”€β”€ MECoreActor.hpp/cpp        # Matching Engine core (per symbol)
β”‚   β”‚   β”œβ”€β”€ MDGActor.hpp/cpp           # Market Data Gateway
β”‚   β”‚   β”œβ”€β”€ FIXAcceptorActor.hpp/cpp   # FIX 4.4 TCP acceptor
β”‚   β”‚   β”œβ”€β”€ ClearingHouseActor.hpp/cpp # Trade clearing & member positions
β”‚   β”‚   └── AITraderActor.hpp/cpp      # Automated trading members
β”‚   β”œβ”€β”€ net/SocketCompat.hpp           # Cross-platform socket abstraction
β”‚   β”œβ”€β”€ persistence/
β”‚   β”‚   β”œβ”€β”€ PersistenceStore.hpp        # Abstract store + InMemoryStore
β”‚   β”‚   β”œβ”€β”€ KafkaBus.hpp                # Multi-topic Kafka publisher (Optiq KFK)
β”‚   β”‚   └── KafkaStore.hpp              # Kafka persistence (optional)
β”‚   β”œβ”€β”€ recovery/RecoveryProxy.hpp/cpp  # Recovery Cause/Effect
β”‚   └── iaca/
β”‚       β”œβ”€β”€ Fragment.hpp                # IACA fragment definitions
β”‚       └── IacaAggregator.hpp/cpp      # Fragment chain aggregation
β”œβ”€β”€ dashboard/
β”‚   β”œβ”€β”€ app.py                          # Flask dashboard + matching engine
β”‚   β”œβ”€β”€ database.py                     # SQLite (orders, trades, OHLCV)
β”‚   └── templates/index.html            # Trading UI with Chart.js
β”œβ”€β”€ fix_gateway/
β”‚   └── fix_server.py                   # Python FIX 4.4 TCP acceptor
β”œβ”€β”€ clearing_house/
β”‚   β”œβ”€β”€ app.py                          # Flask CH portal + API
β”‚   β”œβ”€β”€ ch_database.py                  # SQLite (members, holdings)
β”‚   β”œβ”€β”€ ch_ai_trader.py                 # AI trading strategies
β”‚   └── templates/                      # CH web UI
β”œβ”€β”€ .env                                # LLM and service configuration
β”œβ”€β”€ shared/config.py                    # Centralized configuration (auto-loads .env)
β”œβ”€β”€ docker/
β”‚   β”œβ”€β”€ docker-compose.yml              # Kafka + EuNEx (all services)
β”‚   β”œβ”€β”€ Dockerfile                      # Multi-stage Linux build
β”‚   └── nginx.conf                      # Reverse proxy configuration
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ test_orderbook.cpp              # Book unit tests (26 cases)
β”‚   β”œβ”€β”€ test_matching_engine.cpp        # ME integration tests
β”‚   β”œβ”€β”€ test_threaded_engine.cpp        # Multi-threaded engine tests
β”‚   β”œβ”€β”€ test_clearing_house.cpp         # Clearing house tests (7 cases)
β”‚   β”œβ”€β”€ test_fix_gateway.cpp            # FIX gateway tests (5 cases)
β”‚   └── test_ai_trader.cpp             # AI trader tests (6 cases)
β”œβ”€β”€ examples/
β”‚   β”œβ”€β”€ ping_pong.cpp                   # Actor basics tutorial
β”‚   └── simple_match.cpp                # Matching with Recovery + IACA
└── docs/
    β”œβ”€β”€ developers-guide.md             # Detailed developers guide
    └── process-diagram.md              # Architecture diagrams

Documentation

Next Steps

  1. Multi-threaded actor engine βœ“ SimplxShim with mailbox queues
  2. Kafka persistence βœ“ KafkaStore + Docker Compose (KRaft mode)
  3. FIX gateway βœ“ C++ FIXAcceptorActor + Python fallback
  4. Clearing House βœ“ ClearingHouseActor + AITraderActor
  5. Market simulation βœ“ Realistic AI trading + Dashboard auto-simulation
  6. AI Analyst βœ“ Ollama/Groq/HuggingFace market commentary
  7. Message Flow Visualizer βœ“ Developer pipeline tracing tool
  8. Engine Mode Switch βœ“ Python ↔ C++ toggle with FIX 4.4 bridge
  9. SBE encoding β€” replace event structs with SBE-encoded messages
  10. Master/Mirror failover β€” implement full Recovery replay on Mirror node
  11. Trading phases β€” pre-open, uncrossing, continuous, close, TAL
  12. Additional order types β€” Stop, Pegged, Mid-Point, Iceberg