"Click on the image to get started in 4 minutes with Kestra."
## Live Demo Try Kestra using our [live demo](https://demo.kestra.io). ## What is Kestra Kestra is an open-source, **event-driven** orchestrator that simplifies data operations and improves collaboration between engineers and business users. By bringing **Infrastructure as Code** best practices to data pipelines, Kestra allows you to build reliable workflows and manage them with confidence. Thanks to the **declarative YAML interface** for defining orchestration logic, everyone who benefits from analytics can participate in the data pipeline creation process. The UI automatically adjusts the YAML definition any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is defined declaratively in code, even if some workflow components are modified in other ways.  ## Key concepts 1. `Flow` is the main component in Kestra. It's a container for your tasks and orchestration logic. 2. `Namespace` is used to provide logical isolation, e.g., to separate development and production environments. Namespaces are like folders on your file system ā they organize flows into logical categories and can be nested to provide a hierarchical structure. 3. `Tasks` are atomic actions in a flow. By default, all tasks in the list will be executed sequentially, with additional customization options, a.o. to run tasks in parallel or allow a failure of specific tasks when needed. 4. `Triggers` define when a flow should run. In Kestra, flows are triggered based on events. Examples of such events include: - a regular time-basedĀ **schedule** - anĀ **API**Ā call (*webhook trigger*) - ad-hoc execution from theĀ **UI** - a **flow trigger** - flows can be triggered from other flows using a [flow trigger](https://kestra.io/docs/developer-guide/triggers/flow) or a [subflow](https://kestra.io/docs/flow-examples/subflow), enabling highly modular workflows. - **customĀ events**, including a new file arrival (*file detection event*), a new message in a message bus, query completion, and more. 5. `Inputs` allow you to pass runtime-specific variables to a flow. They are strongly typed, and allow additional [validation rules](https://kestra.io/docs/developer-guide/inputs#input-validation). ## Extensible platform via plugins Most tasks in Kestra are available as [plugins](https://kestra.io/plugins), but many type of tasks are available in the core library, including a.o. script tasks supporting various programming languages (e.g., Python, Node, Bash) and the ability to orchestrate your business logic packaged into Docker container images. To create your own plugins, check the [plugin developer guide](https://kestra.io/docs/plugin-developer-guide). ## Rich orchestration capabilities Kestra provides a variety of tasks to handle both simple and complex business logic, including: - retries - timeout - error handling - conditional branching - dynamic tasks - sequential and parallel tasks - skipping tasks or triggers when needed by setting the flag `disabled` to `true`. - configuring dependencies between tasks, flows and triggers - advanced scheduling and trigger conditions - backfills - documenting your flows, tasks and triggers by adding a markdown description to any component - adding labels to add additional metadata to your flows such as the flow owner or team: ```yaml id: hello namespace: prod description: Hi from `Kestra` and a **markdown** description. labels: owner: john-doe team: data-engineering tasks: - id: hello type: io.kestra.core.tasks.log.Log message: Hello world! description: a *very* important task disabled: false timeout: 10M retry: type: constant # type: string interval: PT15M # type: Duration maxDuration: PT1H # type: Duration maxAttempt: 5 # type: int warningOnRetry: true # type: boolean, default is false - id: parallel type: io.kestra.core.tasks.flows.Parallel concurrent: 3 tasks: - id: task1 type: io.kestra.plugin.scripts.shell.Commands commands: - 'echo "running {{task.id}}"' - 'sleep 10' - id: task2 type: io.kestra.plugin.scripts.shell.Commands commands: - 'echo "running {{task.id}}"' - 'sleep 10' - id: task3 type: io.kestra.plugin.scripts.shell.Commands commands: - 'echo "running {{task.id}}"' - 'sleep 10' triggers: - id: schedule type: io.kestra.core.models.triggers.types.Schedule cron: "*/15 * * * *" backfill: start: 2023-06-25T14:00:00Z ``` ## Built-in code editor You can write workflows directly from the UI. When writing your workflows, the UI provides: - autocompletion - syntax validation - embedded plugin documentation - topology view (view of your dependencies in a Directed Acyclic Graph) that get updated live as you modify and add new tasks. ## Getting Started To get a local copy up and running, follow the steps below. ### Prerequisites Make sure that Docker is installed and running on your system. The default installation requires the following: - [Docker](https://docs.docker.com/engine/install/) - [Docker Compose](https://docs.docker.com/compose/install/) ### Launch Kestra Download the Docker Compose file: ```sh curl -o docker-compose.yml https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml ``` Alternatively, you can use `wget https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml`. Start Kestra: ```sh docker-compose up ``` Open `http://localhost:8080` in your browser and create your first flow. ### Hello-World flow Here is a simple example logging hello world message to the terminal: ```yaml id: hello namespace: prod tasks: - id: hello-world type: io.kestra.core.tasks.log.Log message: Hello world! ``` For more information: - Follow the [getting started tutorial](https://kestra.io/docs/getting-started/). - Read the [documentation](https://kestra.io/docs/) to understand how to: - [Develop your flows](https://kestra.io/docs/developer-guide/) - [Deploy Kestra](https://kestra.io/docs/administrator-guide/) - Use our [Terraform provider](https://kestra.io/docs/terraform/) to deploy your flows - Develop your [own plugins](https://kestra.io/docs/plugin-developer-guide/). ## Plugins Kestra is built on a [plugin system](https://kestra.io/plugins/). You can find your plugin to interact with your provider; alternatively, you can follow [these steps](https://kestra.io/docs/plugin-developer-guide/) to develop your own plugin. For a full list of plugins, check the [plugins page](https://kestra.io/plugins/). Here are some examples of the available plugins: