Orchestrator / CONTROL_PLANE_ARCHITECTURE.md
evgeniy778's picture
Add Control Plane Architecture v1.0
db72ecd verified
|
Raw
History Blame Contribute Delete
7.52 kB
# =====================================================
# Apckeyl Framework
# CONTROL PLANE ARCHITECTURE
# Version 1.0
# =====================================================
# 1. Purpose
The Apckeyl Orchestrator is the central Control Plane
of the Apckeyl Framework.
The Orchestrator is responsible for coordination,
routing, task management, state tracking and communication
between independent processing modules.
The Orchestrator is NOT required to perform heavy
AI processing itself.
The framework therefore separates:
- Control Plane
- Compute Plane
This separation is a fundamental architectural principle
of the Apckeyl Framework.
# 2. Control Plane
The Control Plane contains the components responsible
for managing the system.
Primary responsibilities:
- receive tasks;
- validate tasks;
- create task records;
- assign task IDs;
- determine the required processing module;
- route tasks;
- monitor task state;
- handle errors;
- collect results;
- maintain logs;
- manage external Spaces and services;
- provide a unified interface for Telegram and other clients.
# 3. Compute Plane
The Compute Plane contains independent processing modules.
Compute modules perform the actual work.
Examples:
- Image Generation;
- Video Generation;
- Image Upscaler;
- Video Upscaler;
- Story Character Studio;
- Stock Factory;
- other AI processing services.
A Compute module should be independently replaceable.
The Orchestrator should not depend on the internal
implementation of a Compute module.
# 4. Core Principle
The fundamental relationship is:
CONTROL PLANE
|
v
ROUTING
|
v
COMPUTE MODULE
|
v
RESULT
|
v
CONTROL PLANE
The Orchestrator controls the task.
The external module performs the task.
# 5. Orchestrator Responsibilities
The Orchestrator may contain:
- Orchestrator Core;
- Task Manager;
- Queue Manager;
- Space Manager;
- Download Manager;
- Logger;
- Resource Monitor;
- Module Registry;
- API Gateway;
- configuration management.
# 6. Compute Module Responsibilities
A Compute Module should:
1. receive a valid request;
2. process the request;
3. produce a result;
4. return a status;
5. return an error when processing fails.
The Compute Module should not need to know
how the entire Apckeyl Framework is organized.
# 7. First Compute Module
The first production Compute Module connected to
the Control Plane is:
Apckeyl_RealESRGAN
Its purpose is image upscaling.
The existing Apckeyl_RealESRGAN Space remains an
independent processing Space.
It must NOT be replaced, deleted or moved as part
of the Control Plane migration.
# 8. Communication
The preferred communication model is:
Client
|
v
Orchestrator
|
v
Compute Module
|
v
Orchestrator
|
v
Client
Possible clients include:
- Telegram Bot;
- future web interface;
- internal tools;
- other Apckeyl services.
# 9. Telegram Integration
The Telegram Bot is treated as a client of the
Control Plane.
The Telegram Bot should not contain the complete
processing architecture.
Instead:
Telegram
|
v
ApckeylBot
|
v
Orchestrator
|
v
Compute Module
This allows the Telegram interface to remain
independent from the processing modules.
# 10. Module Registry
The Orchestrator should maintain a registry of
available Compute Modules.
Conceptually:
module_id
module_name
module_type
endpoint
status
capabilities
version
Example:
module_id:
realesrgan
module_name:
Apckeyl_RealESRGAN
module_type:
image_upscaler
status:
available
# 11. Task Routing
A task should contain enough information for the
Orchestrator to determine which module is required.
Example:
task_type:
image_upscale
The Orchestrator resolves:
image_upscale
|
v
realesrgan
|
v
Apckeyl_RealESRGAN
# 12. Task Independence
Tasks must not depend on the physical location
of a Compute Module.
The Compute Module may later be:
- a Hugging Face Space;
- another cloud service;
- an API;
- a local service;
- another compatible processing backend.
The Control Plane should remain unchanged whenever
possible.
# 13. Replaceable Compute Modules
A Compute Module is replaceable.
For example:
Apckeyl_RealESRGAN
|
v
Image Upscaler Module A
can later become:
Image Upscaler Module B
without redesigning the entire Orchestrator.
# 14. Failure Isolation
A failure of one Compute Module should not
automatically destroy the Control Plane.
Example:
Apckeyl_RealESRGAN
|
ERROR
|
X
The Orchestrator should remain operational.
The task should receive a failed state and an
appropriate error description.
# 15. ZeroGPU Consideration
The current Hugging Face Orchestrator Space may operate
under ZeroGPU limitations.
Therefore the framework must NOT depend on the
Orchestrator Space having permanent CPU or GPU compute
capacity.
The Control Plane must remain lightweight.
Heavy processing should be delegated to Compute Modules.
# 16. Hardware Independence
The Apckeyl Framework must not depend on the user's
personal computer as a permanent server.
The user's computer is a development and administration
environment only.
The production architecture should remain operational
without the user's old laptop.
# 17. Existing Working Spaces
Existing working Spaces should be preserved.
In particular:
Apckeyl_RealESRGAN
must remain an independent production module.
The Control Plane migration must not destroy or
rewrite existing working modules.
# 18. Future Architecture
The intended architecture is:
APCKEYL FRAMEWORK
|
v
+------------------+
| CONTROL PLANE |
| ORCHESTRATOR |
+--------+---------+
|
+-----------------+-----------------+
| | |
v v v
Image Factory Video Factory Stock Factory
| | |
v v v
Image Modules Video Modules Stock Modules
|
v
Apckeyl_RealESRGAN
# 19. Design Goal
The main design goal is:
One Control Plane.
Many independent Compute Modules.
The framework should allow new capabilities to be
added without rebuilding the entire system.
# 20. Development Strategy
Development should proceed incrementally.
Recommended order:
1. Control Plane architecture;
2. Orchestrator Core;
3. Module Registry;
4. Task Manager;
5. Space Manager;
6. communication layer;
7. first Compute Module integration;
8. Telegram integration;
9. additional Compute Modules;
10. Stock Factory;
11. Image Generation;
12. Video Generation;
13. Story Character Studio.
# 21. Architectural Rule
The following rule is mandatory:
ORCHESTRATOR = CONTROL
COMPUTE MODULE = WORK
The Orchestrator coordinates.
The Compute Module processes.
Neither component should unnecessarily assume
the responsibilities of the other.