docs/source/en/guide.md

# OpenStoryline User Guide

---

## 0. Environment Setup

See the [README](https://github.com/FireRedTeam/FireRed-OpenStoryline/blob/main/README.md) section.

## 1. Basic Usage

### 1.1 Getting Started

You can start creating in two ways:

1. **You have your own media**

   * Click the file upload button on the left side of the chat box and select your images/videos.
   * Then type your editing goal in the input field, for example: *Use my footage to edit a family vlog with an upbeat rhythm.*

2. **You don’t have media**

   * Just describe the theme/mood.
   * For example: *Help me create a summer beach travel vlog—sunny, fresh, and cheerful.*

Automatic asset retrieval is powered by [Pexels](https://www.pexels.com/). Please enter your Pexels API key in the website sidebar.

**Disclaimer:** We only provide the tool. All assets downloaded or used via this tool (e.g., Pexels images) are fetched by the user through the API. We assume no responsibility for the content of videos generated by users, the legality of the assets, or any copyright/portrait-right disputes arising from the use of this tool. Please comply with Pexels’ license when using it: [https://www.pexels.com/zh-cn/license](https://www.pexels.com/zh-cn/license)
[https://www.pexels.com/terms-of-service](https://www.pexels.com/terms-of-service)

If you just want to explore it first, you can also use it like a normal chat model, for example:

* “Introduce yourself”

<img src="https://github.com/user-attachments/assets/a7c102a0-299d-4fcb-a890-0bcb165867d2" alt="demo" width="500">

### 1.2 Editing

OpenStoryline supports **intent intervention and partial redo at any stage**. After a step completes, you can simply describe what you want to change in one sentence. The agent will locate the step that needs to be rerun, without restarting from the beginning. For example:

* Remove the clip where the camera is filming the sky.
* Switch to a more upbeat background music.
* Change the subtitle color to better match the sunset theme.

<img src="https://github.com/user-attachments/assets/18c1ac82-873d-4ced-beb3-443d0fc9192c" alt="demo" width="500">

### 1.3 Style Imitation

With the style imitation Skill, you can reproduce almost any writing style to generate copy. For example:

* Generate copy in a Shakespearean style for me.
* Mimic the tone of my social media posts.

<img src="https://github.com/user-attachments/assets/67edcb95-a71d-447c-ac13-ae28d0bbd698" alt="demo" width="500">

### 1.4 Interrupting

At any moment while the agent is running, if its behavior is not as expected, you can:

* Click the **Stop** button on the right side of the input box to stop the model reply and tool calls, **or**
* Press **Enter** to send a new prompt—the system will automatically interrupt and follow your new instruction.

Interrupting does **not** clear the current progress. Existing replies and executed tool results will be kept, and you can continue from the current state.

### 1.5 Switching Languages

Click the language button in the top-right corner of the page to switch between Chinese and English:

* The sidebar and tool-call cards will switch display language accordingly.
* The prompt language used inside tools will also switch.
* Past chat history will **not** be automatically translated.

### 1.6 Saving

After you polish a satisfying video, you can ask the agent to **summarize the editing logic** (rhythm, color tone, transition habits, etc.) and save it as your personal **“Editing Skill.”**

Next time you edit similar content, simply ask the agent to use this Skill to reproduce the style.

<img src="https://github.com/user-attachments/assets/d99faca2-233c-49d0-829e-336b2b76a46d" alt="demo" width="500">

### 1.7 Mobile Usage

**Warning: The commands below will expose your service to your local network. Use only on trusted networks. Do NOT run these commands on public networks.**

If your media is on your phone and it’s inconvenient to transfer, you can use the following steps to use the editing agent on mobile.

1. Fill in the LLM/VLM/Pexels/TTS configuration in config.toml.
2. Change your web startup command to:

   ```bash
   # Reminder: --host 0.0.0.0 exposes the service to your LAN/public network. Use only on trusted networks.
   uvicorn agent_fastapi:app --host 0.0.0.0 --port 7860
   ```

3. Find your computer’s IP address:

   * **Windows:** run `ipconfig` in Command Prompt (cmd) and locate the IPv4 address
   * **Mac:** hold **Option** and click the Wi-Fi icon
   * **Linux:** run `ifconfig` in the terminal

4. Then open the following address in your phone browser:

   ```
   {your_computer_ip}:7860
   ```

---

## 2. Advanced Usage

Due to copyright and distribution constraints, open-source resources may not be sufficient for many users’ editing needs. Therefore, we provide methods to add and build private asset libraries.

---

### 2.1 Custom Music Library

Put your private music files into:

`./resource/bgms`

Then tag your music by writing metadata into:

`./resouce/bgms/meta.json`

Restart the MCP service to apply changes.

**Tag Dimensions**

* **scene:** Vlog, Travel, Relaxing, Emotion, Transition, Outdoor, Cafe, Evening, Scenery, Food, Date, Club
* **genre:** Pop, BGM, Electronic, R&B/Soul, Hip Hop/Rap, Rock, Jazz, Folk, Classical, Chinese Style
* **mood:** Dynamic, Chill, Happy, Sorrow, Romantic, Calm, Excited, Healing, Inspirational
* **lang:** bgm, en, zh, ko, ja

**How to Tag**

* **Manual tagging:** Copy the format of other items in `meta.json` and add tags accordingly. **Note:** the `description` field is required.
* **Auto tagging:** Use `qwen3-omni-flash` for automatic tagging (requires a Qwen model API key).

Qwen3-omni labeling script:

```bash
export QWEN_API_KEY="you_api_key"
python -m scripts.omni_bgm_label
```

Auto tags may not be fully accurate. If you need strong recommendations for specific scenarios, it’s recommended to manually review the results.

---

### 2.2 Custom Font Library

Put your private font files into:

`./resource/fonts`

Then tag the fonts by editing:

`./resource/fonts/font_info.json`

Restart the MCP service to apply changes.

**Tag Dimensions**

* **class:** Creative, Handwriting, Calligraphy, Basic
* **lang:** zh, en

**How to Tag**
Currently only manual tagging is supported—edit `./resource/fonts/font_info.json` directly.

---

### 2.3 Custom Copywriting Template Library

Put your private copywriting templates into:

`./resource/script_templates`

Then tag them by writing metadata into:

`./resource/fonts/meta.json`

Restart the MCP service to apply changes.

**Tag Dimensions**

* **tags:** Life, Food, Beauty, Entertainment, Travel, Tech, Business, Vehicle, Health, Family, Pets, Knowledge

**How to Tag**

* **Manual tagging:** Copy the format of other items in `meta.json` and add tags accordingly. **Note:** the `description` field is required.
* **Auto tagging:** Use DeepSeek for automatic tagging (requires the corresponding API key).

DeepSeek labeling script:

```bash
export DEEPSEEK_API_KEY="you_api_key"
python -m scripts.llm_script_template_label
```

Auto tags may not be fully accurate. If you need strong recommendations for specific scenarios, it’s recommended to manually review the results.

---

### 2.4 Custom Skill Library

The repository includes two built-in Skills: one for writing-style imitation and another for saving editing workflows. If you want more custom skills, you can add them as follows:

1. Create a new folder under `.storyline/skills`.
2. Inside that folder, create a file named `SKILL.md`.
3. The `SKILL.md` must start with:

   ```markdown
   ---
   name: yous_skill_folder_name
   description: your_skill_function_description
   ---
   ```

   The `name` must match the folder name.
4. Then write the detailed skill content (its role setting, which tools it should call, output format, etc.).
5. Restart the MCP service to apply changes.