DEVELOPER.md

Developer Guide

Table of contents

• Community
• Configuration
  • Misc
  • Operations
    • STT
    • T2T
    • TTS
    • FILTER_AUDIO
    • FILTER_TEXT
    • EMBEDDING
• REST API Spec
• Websocket Events
  • Shared
  • Job-Specific
• Creating Custom Integrations
  • Some Definitions
  • Making Operations
  • Adding Managed Processes
  • Making Applications
  • Extending Configuration
  • Extending API
• Known Issues

Community

Join the Discord for discussions related to this project!

Configuration

Take me to the top!

An example configuration is provided. Example prompts can be found under prompts Depending on which integrations are configured for use, additional setup is required.

Misc

Take me to the top!

Prompting

Find prompt files for personality and scenario under prompts/characters and prompts/scenes respectively. There are also general instructions, but you shouldn't need to edit this. Place your prompt text files in these directories.

For the configuration file:

• instruction_prompt_filename: (str) Filename of prompt under prompts/instructions (excluding .txt)
• character_prompt_filename: (str) Filename of prompt under prompts/characters (excluding .txt)
• scene_prompt_filename: (str) Filename of prompt under prompts/scenes (excluding .txt)
• character_name: (str) Name of character
• history_length: (int) Number of lines in script to retain

There is also name_translations for translating a user to another name.

name_translations:
    old-name: new-name

Operations

Take me to the top!

• Setup

  • Azure
  • Fish
  • Kobold
  • OpenAI
  • RVC

• stt

  • azure
  • fish
  • kobold
  • openai

• t2t

  • kobold
  • openai

• tts/mcp

  • azure
  • fish
  • kobold
  • melo
  • openai
  • pytts

• filter_audio

  • pitch
  • rvc

• filter_text

  • chunker_sentence
  • filter_clean
  • emotion_roberta
  • mod_koala

• embedding

  • openai

Operations listed in config are loaded by default when starting jaison-core. Enabled operations are listed under property operations. For example:

operations: 
- role: stt
  id: fish
- role: t2t
  id: openai
- role: filter_text
  id: filter_clean
- role: filter_text
  id: chunker_sentence
- role: tts
  id: azure

This loads fish for STT, openai for T2T, all of filter_clean and chunker_sentence for filter_text, and azure for TTS. These are loaded in order. Filters are applied in the order they were loaded, with the earliest applying first before the rest. For non-filter operations, only one can be specified, otherwise older ones are overwritten.

Each operation may have its own configuration depending on the specific operation. For example:

operations: 
- type: tts
  id: azure
  azure_ttsg_voice: "en-US-AshleyNeural"

Below is a list of all available operations and their respective configuration for each type. There are also common setup guides for specific platforms.

Setup

Take me to the top!

Azure

To use Azure Speech Services, you will need an API key and know your region.

1. Go to Azure
2. Make an account or sign in
3. Search for "Resource groups"
4. Create a new one
   • Default subscription tier for new accounts is free. No need to change
   • Pick a region close to you
5. Open the resource group
6. Click Create and search for "SpeechServices"
7. Create a new Speech service
   • Select the resource group you just made
   • Pick a region close to you
   • Pick "Standard S0" for free tier pricing
8. Open newly created Speech service
9. Scroll to bottom of overview and copy any KEY and the Location/Region
10. In the main project, copy .env-template or one created prior
11. Paste what you just copied into AZURE_API_KEY and AZURE_REGION respectively after the "="

Fish

To use Fish Audio, you will need an API key. Premium tier IS NOT necessary, however you will still need to pay for what you use.

1. Go to Fish Audio
2. Make an account or sign in
3. Navigate to "API" tab
4. Get API credits if you don't have any
5. Go to API Keys
6. Click "Create Secret Key"
   • Set a long expiry date or have it never expires. When it expires, you will need to replace the old key with a new one
7. Copy the "Secret Key" you just generated from the "API List"
8. In the main project, copy .env-template or one created prior
9. Paste what you just copied into FISH_API_KEY after the "="

Kobold

To use Kobold CPP, You will need the application and set it up.

1. Download KoboldCPP
   • Get the right one for your system
   • Windows users will want one that ends in .exe
   • If you have an NVidia GPU, get one with CUDA (cu12, cuda1150, cuda1210)
2. Move the downloaded application to this project under models/kobold
3. Run this application to bring up its setup interface
4. Download the model(s) you want to use as detailed here
   • For STT, download any of the .bin here Generally, base and small are good enough
5. Move the downloaded model(s) into the same models/kobold folder
6. Configure your KoboldCPP through the interface
   • Select a fitting preset (CuBLAS for NVidia GPUs. Vulkan is good for most)
   • Setup T2T model in "Model Files" if applicable
   • Setup STT and TTS model in "Audio" if applicable
   • Everything doesn't need to be adjusted
7. Click "Save" and save to same models/kobold folder using any name
8. Update this project's configuration file
   • kobold_filepath with full filepath to Kobold application
   • kcpps_filepath with full filepath to saved Kobold config file
   • If on windows, make sure each \ is \\ in the path as shown in example.yaml

OpenAI

To use OpenAI (not some other OpenAI-like API), you will need an API key. If you want to use another OpenAI-like API, see the statement after the main set of steps here.

1. Go to OpenAI Dashboard
2. Make an account or sign in
3. Navigate to "Profile" then "Secrets"
4. Create and copy the key
5. In the main project, copy .env-template or one created prior
6. Paste what you just copied into OPENAI_API_KEY after the "="

For other OpenAI-like APIs, they generally have their own setup instructions. Usually this involves replacing the API key above with theirs. You will additionally need to update the base URL inside this project's configuration file. These are listed as openai_t2t_base_url for example. Each of STT, T2T, and TTS have separately configurable base URLs.

RVC

To use RVC, you will need the download all necessary assets and create (or find) a voice model.

1. Ensure you have Git and GitLFS on your system
2. Clone the RVC Project with git clone https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI.git
3. Inside the project you just cloned, run the command python tools/download_models.py
4. Confirm you have downloaded the models by looking in assets/hubert and see if you have a file hubert_base.pt (NOT hubert_inputs.pth)
5. If downloaded, copy all the contents from that project's assets folder into this projects assets/rvc
6. Train or find an RVC voice model. Training is recommended only if you have a GPU with at least 8GB of VRAM. NVidia GPUs are easier to setup for training. You may the project you took the assets file of to train a model using their web UI. Setup for that project can be found here
7. Copy the model into this project
   • .pth into assets/rvc/weights. If trained, this can be found under assets/weights of the other project
   • index folder containing a .index file into models/rvc. If trained, this can be found under logs of the other project. If you only have the .index file and not a folder, make a folder with the same name as your .pth file
8. In the main project, copy .env-template or one created prior, and ensure it has the section for RVC. DO NOT MODIFY THESE

stt

Take me to the top!

azure

• compatibility -> all
• paid -> no

Use Azure Speech Services for speech-input.

Configuration:

• language (str) Input speech language

fish

• compatibility -> all
• paid -> yes

Use Fish Audio for speech-input.

No additional configuration

kobold

• compatibility -> all
• paid -> no

Use KoboldCPP for speech-input.

Additional configuration

• suppress_non_speech (bool) for skipping non-speech sounds
• langcode (str) for code of input language

openai

• compatibility -> depends
• paid -> depends

Default to use OpenAI's service, which is compatible with all but paid. Can also be used with applications/services that have OpenAI-like API such as Ollama.

Configuration:

• base_url (str) for specifying endpoint (OpenAI or some other application/service)
• model (str) model to use
• language (str) language

t2t/mcp

Take me to the top!

kobold

• compatibility -> limited
• paid -> no

Direct support for models on KoboldCPP. More flexible samplers than OpenAI-like APIs.

Configuration:

• max_context_length (int) max context length of model
• max_length (int) max length allowable for model
• quiet (bool) quiet output
• rep_pen (float) sampler
• rep_pen_range (int) sampler
• rep_pen_slope (int) sampler
• temperature (float) sampler
• tfs (int) sampler
• top_a (int) sampler
• top_k (int) sampler
• top_p (float) sampler
• typical (int) sampler

openai

• compatibility -> depends
• paid -> depends

Default to use OpenAI's service, which is compatible with all but paid. Can also be used with applications/services that have OpenAI-like API such as Ollama.

Configuration:

• base_url (str) for specifying endpoint (OpenAI or some other application/service)
• model (str) for model ID
• temperature (float) for adjusting temperature
• top_p (float) for adjusting top P
• presence_penalty (float) for adjusting presence penalty
• frequency_penalty (float) for adjusting frequency penalty

tts

Take me to the top!

azure

• compatibility -> all
• paid -> no

Use Azure Speech Services for natural sounding synthesized speech.

Configuration:

• voice (str) ID of void from voice gallery (ID used in their sample code for speech_synthesis_voice_name)

fish

• compatibility -> all
• paid -> yes

Use Fish Audio for voice-cloned text-to-speech.

Configuration:

• model_id (str) for voice model ID
• backend (str) for model to use
• normalize (bool) for normalizing input message for clearer pronunciation
• latency (str) one of "normal" (stable) or "balanced" (faster but choppy)

kobold

• compatibility -> limited
• paid -> no

Use KoboldCPP for TTS. Mostly for completion sake, and not recommended for use.

Configuration:

• voice (str) voice to use

melo

• compatibility -> all
• paid -> no

Use MeloTTS for local AI TTS. This is recommended choice for fast, consistent latency and control over the voice if you know what your doing.

Configuration:

• config_filepath (str) Filepath to model config
• model_filepath (str) Filepath to model
• speaker_id (str) Name of speaker in model
• device (str) Pytorch device (cpu or cuda)
• language (str) Language code
• sdp_ratio (float) Ratio for expressiveness
• noise_scale (float) Noise seeding input
• noise_scale_w (float) Noise width seeding input
• speed (float) Speed of final speech

openai

• compatibility -> depends
• paid -> depends

Default to use OpenAI's service, which is compatible with all but paid. Can also be used with applications/services that have OpenAI-like API.

Configuration:

• base_url (str) for specifying endpoint (OpenAI or some other application/service)
• voice (str) for voice name
• model (str) for voice model

pytts

• compatibility -> all
• paid -> no

Use system's speech synthesizer (SAPI for Windows, ESpeak for Linux) to generate speech.

Configuration:

• voice (str) for voice ID (a list of these is printed on start when configured to be used)
• gender (str) for voice gender if applicable

filter_audio

Take me to the top!

pitch

• compatibility -> all
• paid -> no

Pitch generated audio up and down a number of semi-tones

Configuration:

• pitch_amount (int) pitch shift in semi-tones

rvc

• compatibility -> limited
• paid -> no

Use voice changers trained using RVC.

Configuration:

• voice (str) for model name
• f0_up_key (int) for changing voice pitch
• f0_method (str) for generation method
• f0_file (str) for frequency filepath
• index_file (str) for index filepath
• index_rate (float) for index rate
• filter_radius (int)
• resample_sr (int) for resampling audio to another sample rate
• rms_mix_rate (int)
• protect (float)

filter_text

Take me to the top!

chunker_sentence

• compatibility -> all
• paid -> no

Accumulate output from T2T model into sentences before passing them down the pipeline.

No additional configuration

emotion_roberta

• compatibility -> all
• paid -> no

Use SamLowe/roberta-base-go_emotions to castegorize response into an emotion.

No additional configuration

filter_clean

• compatibility -> all
• paid -> no

Accumulate output from T2T model into sentences before passing them down the pipeline.

No additional configuration

mod_koala

• compatibility -> limited
• paid -> no

Use Koala/Text-Moderation to categorize and filter offensive responses.

No additional configuration

embedding

Take me to the top!

openai

• compatibility -> all
• paid -> yes

Generate embeddings using OpenAI embedding models

Configuration:

• base_url (str) URL to openai-compatible API endpoint
• model (str) Name of model to use

REST API

Take me to the top!

API spec is made with OpenAPI 3.1.0 standard and can be found api.yaml.

Please read the description of the endpoint you are interested in. If it specifies use of websockets to communicate status or results, you will need to setup a websocket for updates on your request. Using such REST API endpoints are successful when they successfully queue a job and doesn't mean the job itself was successful.

Please see Websocket Events for websocket messages related to each job.

Websocket Events

Take me to the top!

Websockets are used for several reasons

• Ensure all applications are notified of changes even if they didn't request it
• Enable long-lived requests such as responses which may take a few seconds to finish
• Real-time feedback and streaming of responses to reduce latency
• Allow predictable, sequential behavior and locking state during response generation

While each job is unique, they follow similar patterns in terms of generated events. The following are generated in order

1. Job start
2. Job-specific events (as many as it needs) ...
3. One of 2 events
   • Job finish
   • Job cancelled

Each job is ran sequentially in the order they were queued. Events are also sent in order they were generated. You can expect to receive all events in this predictable order and process 1 job's events at a time.

These events are detailed in the following sections.

Shared

Take me to the top!

Some Common Enums and Definitions

Job Types

• response: POST /api/response
• context_clear: DELETE /api/context
• context_request_add: POST /api/context/request
• context_conversation_add_text: POST /api/context/conversation/text
• context_conversation_add_audio: POST /api/context/conversation/audio
• context_custom_register: POST /api/context/custom
• context_custom_remove: DELETE /api/context/custom
• context_custom_add: PUT /api/context/custom
• operation_load: POST /api/operations/load
• operation_reload_from_config: POST /api/operations/reload
• operation_unload: POST /api/operations/unload
• operation_use: POST /api/operations/use
• config_load: PUT /api/config/load
• config_update: PUT /api/config/update
• config_save: POST /api/config/save

Error Types

• operation_unknown_type: Specified an unknown operation type
• operation_unknown_id: Specified an unknown operation ID for that type
• operation_duplicate: Tried loading a filter that's already loaded
• operation_unloaded: Tried using a filter that's not loaded
• operation_active: Tried activating an operation that's already active (should never occur, lmk if it does)
• operation_inactive: Tried deactivating an operation that's already inactive, or using an inactive operation (should never occur, lmk if it does)
• config_unknown_field: Tried updating or loading a configuration with an invalid field
• config_unknown_file: Tried loading a configuration file that doesn't exist
• job_unknown: Tried starting an invalid job type(should never occur, lmk if it does)
• job_cancelled: Job was cancelled via the REST API

Job Start

Signify the start of a job's processing and the arguments provided. The arguments provided are what's included in the original REST API call's body if it was valid to create a job in the first place. For audio_bytes, due to size, it is simply returned as a boolean indicating if it was included.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "start": { "input argument keyword": "input argument value", ... }
    }
}

Job Finish

Signify the successful end of a job's processing.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": true,
        "success": true
    }
}

Job Cancelled

Signify the unsuccessful end of a job's processing. This may be due to some error during processing, or the result of an application cancelling the job through the REST API.

These will only be emitted once the job has started processing, even if the job was cancelled before then. Therefore, if an application cancels a job, it won't receive the cancelled event until all jobs prior have finished processing and this is put on next. The job is then cancelled immediately and all are notified.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": true,
        "success": false,
        "result": {
            "type": "error type",
            "reason": "error message"
        }
    }
}

Job-Specific

Take me to the top!

response

Events contain details about generation and are sent in order they appear below.

Immediately after LLM generation but before text filters

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "instruction_prompt": "Instructions for the LLM",
        }
    }
}

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "history": [
                {"type": "raw", "message": "Example of raw user input"},
                {"type": "request", "time": 1234, "message": "Example of request message"},
                {"type": "chat", "time": 1234, "user": "some user or AI name", "message": "Example of chat message"},
                {"type": "tool", "time": 1234, "tool": "some tool name", "message": "Example of tool result"},
                {"type": "custom", "time": 1234, "id": "some custom context id", "message": "Example of context"}
            ],
        }
    }
}

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "raw_content": "Response from LLM before application of text filters",
        }
    }
}

The following events are looped (once reaching end, loops back to this first event and continuing if more is generated).

This event's results depend on the filters applied. Some operations such as emotion_roberta augment the result by adding emotion alongside the content property for example.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "content": "Response from LLM after filters",
            "other augmented properties": "their value",
            ...
        }
    }
}

If audio is included, can produce multiple (each chunk for the next packet of audio):

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "audio_bytes": "base64 utf-8 encoded bytes",
            "sr": 123,
            "sw": 123,
            "ch": 123
        }
    }
}

context_clear

No job-specific events.

context_request_add

Events contain details context added. Only one is generated.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "timestamp": 12345,
            "content": "content as given in arguments",
            "line": "[request]: as it appears in the script"
        }
    }
}

context_conversation_add_text

Events contain details context added. Only one is generated.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "user": "name of user associated with line",
            "timestamp": 12345,
            "content": "content as given in arguments",
            "line": "[line]: as it appears in the script"
        }
    }
}

context_conversation_add_audio

Events contain details context added. Only one is generated.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "user": "name of user associated with line",
            "timestamp": 12345,
            "content": "content as given in arguments",
            "line": "[line]: as it appears in the script"
        }
    }
}

context_custom_register

No job-specific events.

context_custom_remove

No job-specific events.

context_custom_add

Events contain details context added. Only one is generated.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "timestamp": 12345,
            "content": "content as given in arguments",
            "line": "[line]: as it appears in the script"
        }
    }
}

operation_load

Events contain details of loaded operation. One is generated per operation listed.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "type": "operation type",
            "id": "operation id"
        }
    }
}

operation_reload_from_config

No job-specific events.

operation_unload

Events contain details of unloaded operation. One is generated per operation listed.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": {
            "type": "operation type",
            "id": "operation id"
        }
    }
}

operation_use

Events contain results from using operation. Multiple of these can be generated if output is streamed. Resulting chunk differs between operations, but usual behavior is generalized under Creating Custom Integrations - Operations.

{
    "status": 200,
    "message": "job type",
    "response": {
        "job_id": "job uuid generated when first created",
        "finished": false,
        "result": { "output chunk property": "output chunk value" }
    }
}

config_load

No job-specific events.

config_update

No job-specific events.

config_save

No job-specific events.

Creating Custom Integrations

Take me to the top!

In case you really want to use an unsupported service, directly implement a model into jaison-core, or just make and share an external application with jaison-core as it's backend, this guide should help you navigate and work on the code like Limit does.

• Some Definitions
• Making Operations
  • Implementing an Operation
  • Connecting an Operation for Use
• Adding Managed Processes
  • Implementing a Process
  • Connecting a Process for Use
  • Connecting with Operations for Management
• Adding MCP Servers
• Making Applications
• Extending Configuration
• Extending API
  • Non-Job-Based Endpoints
  • Job-Based Endpoints

Some Definitions

Take me to the top!

Operation - A unit of compute that assists in creating or modifying a response.

Active operation - Operation that has started and can be used.

Inactive operation - Operation that has never been started or has closed and can't be used.

Process - An program that has to run in a separate process from jaison-core. When referred to in this context, it generally means jaison-core is responsible for starting and stopping this process (it is a child process to jaison-core and not another server you manually booted up on the side).

Application - A program that uses the REST API or websocket server of jaison-core.

Application layer - Main implementation of functionality for all REST API endpoints. utils/jaison.py is the file Limit refers to as the "application layer" whereas operations are seen as the "hardware layer".

Event - Message sent through a websocket from jaison-core to an application

Job - Special request created through the REST API. These are tasks to be completed after all previously created tasks are complete. They run one at a time and wait in a queue to be processed next. They outlive the original API request that made them, and they communicate back their results and status through websockets. Each job is associated with a single function in the application layer. Simply, they are queued functions that will produce events.

Making Operations

Take me to the top!

Everything you need to make a basic operation is in utils/operations.

To make a new operation, make a new file in the directory corresponding to your operation type. In this file, you will be implementing the base operation of that type. You can find that in the base.py in that type's directory.

Implementing an Operation

There are 2 inherited attributes: op_type: (str) operation type specifier op_id: (str) the operation id you specified in __init__

There are 6 functions to note:

__init__(self): must be implemented with no additional arguments. In here, you must also call super().__init__(op_id) where op_id will be the id of this operation, unique to the one's of the same type (there are multiple kobold, but each kobold operation is in a different type). You can initialize attributes in here, but this is only ran once and is synchronous.

__call__: DO NOT IMPLEMENT

async start(self): This is where you'll actually setup your operation. Make any connections, asynchronous calls, etc. This will be called every time the operation is loaded. Don't worry about closing before starting as it's handled automatically. Remember to call await super().start() at the beginning.

async close(self): This is where you'll stop your operation. Close any connections and clean it up. This is what's called before every start if the operation has already started. Remember to call await super().close() at the beginning.

async _parse_chunk(self, chunk_in): Extract information from input dictionary chunk_in, validate, and use as input to _generate. There is a default one already implemented, but if you need to parse additional fields not parsed by default (such as emotion for an emotion-based tts), then reimplement it with the same spec as the base.

async _generate(self, **kwargs): Must be implemented. Instead of returning, use yield even if you only use it once. Results from _parse_chunk are used as kwargs here. Perform the calculation and yield the dictionary that contains at least the fields specified in base.py.

Connecting an Operation for Use

All operations are accessed from the OperationManager located in utils/operations/manager.py. Everything here is dynamic except for function loose_load_operation. This is what you'll be modifying.

1. Find function loose_load_operation
2. Find the case that matches your operation's type
3. Extend the if-else block
   • the op_id you match should be the one you initialized before, and is also the id you use in configuration
   • add your import statement here, not globally
   • return an instance like the rest of them

You can now use your custom operation.

Adding Managed Processes

Take me to the top!

Implementing a Process

If you have an operation that depends on another running application, you can have jaison-core automatically start and stop that application whenever that operation is in use or not. This is done for KoboldCPP, and can be done for your application as well as long as you can start and get an instance of that process in Python (see utils/processes/processes/koboldcpp.py for example).

Code for managing processes can be found in utils/processes. Process specific code is in utils/processes/processes. You will need to implement BaseProcess found in utils/processes/base.py.

You only need to implement 2 functions. All else should not be modified. Check the base implementation to know which these are.

__init__: Be sure to call super().__init__(process_id) where process_id is the a unique name chose purely for logging purposes.

async reload(self): Starting logic. You will need to start the process and save it to the process attribute. You can also save the port is applicable for use in your operations.

Connecting a Process for Use

All processes are accessed through the ProcessManager found in utils/processes/manager.py. We need to add it here so it's exposed for use.

1. Open utils/processes/manager.py
2. Add an entry to the ProcessType enum for your process.
3. Create a new case in function load
   • Import your process in there
   • Add a new instance with the enum as the key
   • asynchronously call reload on that instance

Connecting with Operations for Management

The process does not start until an operation demands it. Likewise, it does not stop until there are no more operations that use it. To setup this relationship, we need to know 2 functions from the ProcessManager:

link(link_id, process_type): Link an operation to that process. This lets the process know it's being used by that operation. link_id is an ID unique across all operations for that specific operation. process_type is the enum you created for your process.

unlink(link_id, process_type): Unlink an operation to that process. This lets the process know the operation no longer needs it (because its closing or just doesn't need it). link_id is an ID unique across all operations for that specific operation. process_type is the enum you created for your process.

When all links are gone, a process will unload itself. Once an operation links up again, the process will start up again. For examples of how this is used, see any kobold operation.

There are additional helper functions you may find useful:

get_process(process_type): Get the instance of that process. Useful if you need direct access to its attributes such as port.

signal_reload(process_type): Have the process restart on the next clock cycle. Typically not needed for an operation and moreso for restarting a process with modified configuration.

signal_unload(process_type): Have the process foribly unload on the next clock cycle. Ignores existing links and just shuts down the process. Typically not needed for an operation and moreso for jaison-core shutdown.

Adding MCP Servers

Take me to the top!

This project has an MCP client built in. Tool calls are generated by a separately configured tool-calling LLM (the one with role mcp) given the current user and system prompt as context. This tool-calling occurs in the response pipeline just before the prompts for the personality LLM is generated. Tools are automatically described and their results appended to the script for any MCP server, and any well documented MCP server will be compatible with this project.

To add an MCP server, add the MCP server directory to models/mcp. For example, I have an MCP server in the file internet.py, so I can put it in models/mcp/internet/internet.py. To configure the project to deploy and use that server, in the yaml config, add a new entry under mcp. For example:

mcp:
- id: example_server
  command: python
  args: ["example_mcp_server.py"]
  cwd: "path/to/server/directory"

The id can be any arbitrary, unique id of your choice. The rest are self explanatory. You may use any MCP server (it doesn't have to be Python, and if it is Python, it should work with the current Python version and dependencies.).

Making Applications

Take me to the top!

Applications can vary in form and function. I [Limit] am not going to tell you how to make your application, but here are some pointers.

All interactions are started through the REST API. I've extensively documented it in using the OpenAPI standard in api.yaml and under the REST API section.

Majority of interactions are job-based. It will most likely be necessary to create a websocket session. It's recommended to create a long-lived websocket connection and iterate through all incoming events indefinitely. Events can be associated with a specific job via job_id and the type of job via the message. For more information on these events from order to structure, see the Websocket Events section.

Extending Configuration

Take me to the top!

All configuration lives in utils/config.py. They are accessible all throughout the code by importing this module and fetching the singleton via Config(). Extending this configuration is as simple as adding a new attribute. This attribute must have a type hint and a default value. Now you can configure this value from your config files using the same name as the attribute.

Extending API

Take me to the top!

The API is implemented using Quart in utils/server/app_server.py. Every endpoint follows a very similar style, and has an entry for functionality and another entry for handling CORS. Regardless of if your making a job-based or non-job-based API endpoint, you need to create both of these entries.

Example functionality entry:

@app.route('/api/config', methods=['GET'])
async def get_current_config():
    pass

Example CORSE entry:

@app.route('/api/config', methods=['OPTIONS']) 
async def preflight_config():
    return create_preflight('GET')

The CORS entry will always return a call to create_preflight(method) and that suffices.

As for functional entries, their implementation differs if they are job-based or not.

Non-Job-Based Endpoints

Example

@app.route('/api/config', methods=['GET'])
async def get_current_config():
    return create_response(200, f"Current config gotten", JAIson().get_current_config(), cors_header)

This is the typical structure of a non-job-based endpoint. This kind of endpoint does not queue a job. It is your traditional REST API endpoint.

create_response normalizes the response returned from the actual function. You can find the implementation in utils/server/common.py. In the snippet, besides the obious of changing defined function name, route, and possibly method, we also need to change the message and function call used in create_response.

Messages here hold no importance beside potential logging in applications.

All functions are defined in the application layer. This is by convention and up to you if you want to do that. You may return any JSON-serializable data-type, and this will appear in the response field of the body.

Job-Based Endpoints

Example

@app.route('/api/response', methods=['POST'])
async def response():
    return await _request_job(JobType.RESPONSE)

These need to be defined after the definition of _request_job. Job-based endpoints are a lot more complicated to setup, so bear with me.

Besides the obvious of changing the API endpoint and method, you need to change the JobType to the correct enum. If you're making a new endpoint, chances are you don't have an enum for your job yet. To create a an enum, go to utils/jaison.py and add it to JobType. The string chosen here is what's used in message in events (used to identify which job type event results from).

To associate this enum with a job's function, you need to add a case for it under the function create_job. Copy the format of all other lines, only replacing the enum and the function called (DO NOT AWAIT THIS FUNCTION).

You will need to correctly define your job's function as well. Define a new async function to JAIson as follows

    async def my_job_function(
        self,
        job_id: str,
        job_type: JobType,
        ...
    ):
        ...

There are several requirements here:

• The only args should be self, job_id, and job_type
• All arguments you expect to receive from the request body are listed as kwargs. You should not put **kwargs unless you intend to validate requests bodies in this function.
• THIS MUST BE AN ASYNC FUNCTION

Websocket events follow a predictable order, so its best you follow the order of emitted events to avoid breaking applications.

1. Start with await self._handle_broadcast_start(job_id, job_type, {kwargs})
   • if one of your kwargs is expected to be large, replace it with a shortform or boolean indicator so listeners can confirm paramenters of job
2. End with await self._handle_broadcast_success(job_id, job_type)

You don't need to handle error events as these are done automatically when the coroutine throws an exception.

Implement the rest of your function inbetween. To communicate status and results, use await _handle_broadcast_event(job_id, job_type, {whatever you want}). Whatever you put in the dictionary is what's put in results in the event.

Now your new job-based endpoint is all setup.

Known Issues

Take me to the top!

jaison-core will not capture kill signals until all websocket connections are closed. Since jaison-core itself does not let go of these connections, the applications themselves must terminate the connection before jaison-core can shutdown.

No data validation alongside insecure connections make this application vulnerable to all sorts of security attacks. Not recommended to host outside of private network.