diff --git "a/data/evaluation/finetuned/testset_data.json" "b/data/evaluation/finetuned/testset_data.json" new file mode 100644--- /dev/null +++ "b/data/evaluation/finetuned/testset_data.json" @@ -0,0 +1,116 @@ +[ + { + "user_input":"Howw doo I use the Shopify CLI to createe a neww extensionn for Shopify Floww actions?", + "reference_contexts":[ + "To create an action that merchants can use in their workflows, you need to add the action to your app. The action needs to contain the following information: The fields that the merchant needs to complete when they add the action to their workflows The URL that Shopify Flow uses to send (POST) the contents (JSON payload) of the action to your app You also need to configure your app to process the data from the POST request when it arrives and to send status codes back to Shopify Flow. To enhance the merchant experience and more closely integrate external systems, you can also [build a custom configuration page](\/docs\/apps\/build\/flow\/actions\/build-config-ui). To improve the reliability of your action, you can add [custom validation](\/docs\/apps\/build\/flow\/actions\/endpoints#custom-validation) for action properties. ## Requirements You have the following: A test web server that has access to the Internet, so that it can receive POST requests from Shopify Flow A test app that works with the test web server A development store that has [Shopify Flow](https:\/\/apps.shopify.com\/flow) and the test app installed ## Step 1: Create a Flow Action To give your Flow action a meaningful name, use the following guidelines: Use a present tense verb + object acted on format. For example, `Place auction bid`. Use sentence case. Don't use punctuation. Separate words using spaces. ### Using Shopify CLI Use the Shopify CLI to generate a new extension: 1. Navigate to your app directory. 2. Run the following command: ```bash #!\/bin\/bash shopify app generate extension ``` 3. Select the `Flow Action` as the type of extension. 4. Provide a meaningful name for your extension. After you've followed the prompts, Shopify CLI generates the extension\u2019s file representation in your app's `\/extensions` directory and gives you a success message. You can then go into your app's `\/extensions` directory and start editing your new extension. The file structure of your extension should look like the following: ``` \/place auction bid shopify.extension.toml ``` To learn more about the extensions file structure, refer to [App structure](\/docs\/apps\/build\/cli-for-apps\/app-structure) and the documentation for your extension type. ### Using the Partner Dashboard 1. In your Partner Dashboard, click [Apps](https:\/\/partners.shopify.com\/current\/apps). 2. Select the app that you want to add your Shopify Flow action to. 3. Click **Extensions**, then click **Create** or **Create extension**. 4. Under the **Flow** tab, click **Flow\/Actions**. 5. In the **Extension name** field, name your action, such as `Place auction bid`. This name is used only for internal purposes. 6. Enter a title and description for the action. In the **Action Preview** area, you can see how the title and action display to merchants when they're choosing actions in Shopify Flow. 7. Enter the URL for the action execution endpoint that you created. Shopify Flow sends the action's JSON payload to this endpoint when it's about to execute your action. ## Step 2: Customize a Flow action configuration file In this section you'll use the default action template and update it to be a functional extension example. Once you have generated a Flow extension using Shopify CLI, follow the instructions below: 1. Change the description to `Place a bid on an auction`. 2. Update the `extensions.runtime_url` to an endpoint where you can receive the runtime request. 3. On the second `settings.fields` field, update the following values: - `type` to `number_decimal` - `key` to `amount` - `name` to `Bid Amount` - Add a `description` property and set it to `The amount of the bid` ```bash [[extensions]] name = \"Place Auction Bid\" type = \"flow_action\" handle = \"place-bid\" description = \"Place a bid on an auction\" runtime_url = \"https:\/\/your-server-domain\/path\/to\/action\/handler\" [settings] [[settings.fields]] type = \"customer_reference\" required = true [[settings.fields]] type = \"number_decimal\" key = \"amount\" name = \"Bid Amount\" description = \"The amount of the bid\" required = true ``` ##" + ], + "reference":"To use the Shopify CLI to generate a new extension for Shopify Flow actions, follow these steps: 1. Navigate to your app directory. 2. Run the command `shopify app generate extension`. 3. Select `Flow Action` as the type of extension. 4. Provide a meaningful name for your extension. After completing these steps, Shopify CLI generates the extension\u2019s file representation in your app's `\/extensions` directory and provides a success message. You can then start editing your new extension in the `\/extensions` directory.", + "synthesizer_name":"single_hop_specifc_query_synthesizer" + }, + { + "user_input":"How do you configure a web server to handle Shopify Flow actions?", + "reference_contexts":[ + "Step 3: Configure your web server To build a Shopify Flow action, you need to add a service to your web server to listen for the JSON payload that Shopify Flow sends when the action runs. Optionally, you can also add the following: An endpoint to validate actions A [custom configuration page](\/docs\/apps\/build\/flow\/actions\/build config ui), and an endpoint that lets merchants preview your custom configuration page Add the following API endpoints to your server: | Endpoint | Purpose | | --- | --- | | [Flow action execution](\/docs\/apps\/flow\/actions\/endpoints#flow-action-execution) | The endpoint where the automation tool sends your action's payload. The payload contains data that you can use to execute the action in your app.| | [Custom configuration page preview](\/docs\/apps\/flow\/actions\/endpoints#custom-configuration-page-preview) | An endpoint that provides data about your [custom configuration page](\/docs\/apps\/build\/flow\/actions\/build-config-ui) to display in the automation tool. This endpoint is required if you want to use a custom configuration page.| | [Custom validation](\/docs\/apps\/flow\/actions\/endpoints#custom-validation) | An endpoint that validates the contents of merchant-configurable properties in an action payload when an action is saved. This endpoint is required if you want to use a custom configuration page.| To learn more about the endpoint requirements for your server, refer to [Action endpoints](\/docs\/apps\/build\/flow\/actions\/endpoints). To learn how to create a custom configuration page, refer to [Build a custom configuration page](\/docs\/apps\/build\/flow\/actions\/build-config-ui). ##" + ], + "reference":"To configure a web server to handle Shopify Flow actions, you need to add a service to your web server to listen for the JSON payload that Shopify Flow sends when the action runs. Additionally, you can add an endpoint to validate actions, a custom configuration page, and an endpoint that lets merchants preview your custom configuration page. The necessary API endpoints include: Flow action execution, Custom configuration page preview, and Custom validation.", + "synthesizer_name":"single_hop_specifc_query_synthesizer" + }, + { + "user_input":"How can I test an action created in the Partner Dashboard using Shopify Flow?", + "reference_contexts":[ + "Step 4: Enable the draft version of your action Running [`app dev`](\/docs\/api\/shopify-cli\/app\/app-dev) allows changes made to local files to update the draft version of your Flow task extensions. The draft version is only available in your development store. > Note: > When [`app dev`](\/docs\/api\/shopify-cli\/app\/app-dev) is running and \"Development store preview\" is enabled, the draft version of a task will appear in your development store _in place_ of the deployed version. Other shops will continue to see the deployed version of your task (if one exists). Draft versions can be identified by the \"draft\" badge. To see the deployed version of the task in your development store, turn off \"Development store preview\" in the \"Extensions\" section of your app in [Shopify Partners](https:\/\/partners.shopify.com\/). 1. Navigate to your app directory. 2. Run the following command to start using draft versions of your extension(s): ```bash #!\/bin\/bash shopify app dev ``` 3. Follow the prompts. ## Step 5: Test the action After you've created an action in the Partner Dashboard and added support for it in your web server, you can test the action in Shopify Flow on your development store. 1. In your development store, create a [workflow](https:\/\/www.shopify.com\/admin\/apps\/flow) that uses the action. For example, add the trigger that you created in the [Triggers guide](\/docs\/apps\/build\/flow\/triggers\/create) and this action to a workflow. 2. If you created a custom configuration page, then ensure that the preview displays and that the custom configuration page is accessible. 3. If you added any custom validation, then ensure that it works as expected. 4. Trigger the workflow. For example, in your web server, run the event that sends the trigger information to Shopify Flow. When the workflow completes, your web server has sent data to Shopify Flow because of the trigger. Shopify Flow has sent this data to a web server that logged the information to its console because of the action. ## Step 6: Deploy your extension > Note: > Deploying extensions using the `app deploy` command also publishes the extensions. We recommend testing changes by using [`app dev`](\/docs\/api\/shopify-cli\/app\/app-dev) or deploying to a test app before deploying them to a production app. Use Shopify CLI to deploy your extensions: 1. Navigate to your app directory. 2. Run the following command to start deploying your extension(s): ```bash #!\/bin\/bash shopify app deploy ``` 3. Follow the prompts. When you receive confirmation that the deploy was successful, your extensions have been released. ## Verifying requests For security reasons, make sure that you verify the following elements in each request: The POST request's HMAC header (either `x shopify hmac sha256` or `http x shopify hmac sha256`). The HMAC header should be verified before you process the payload. For more information, refer to [Verifying requests](\/docs\/apps\/build\/flow\/actions\/endpoints#verifying requests). The payload `handle`. This ID should match the `handle` of the action that you created, and can be retrieved from the payload preview. ## Next steps Connect your app to Shopify Flow so that events that occur in your app can [trigger workflows](\/docs\/apps\/build\/flow\/triggers). Learn how to receive [lifecycle events from Shopify Flow](\/docs\/apps\/build\/flow\/track lifecycle events) about the stores that are using your triggers in enabled workflows. Learn more about how to [return complex data](\/docs\/apps\/build\/flow\/configure complex data types) in a Flow action." + ], + "reference":"After you've created an action in the Partner Dashboard and added support for it in your web server, you can test the action in Shopify Flow on your development store. First, create a workflow in your development store that uses the action. Ensure that any custom configuration page displays correctly and that custom validation works as expected. Then, trigger the workflow by running the event in your web server that sends the trigger information to Shopify Flow. When the workflow completes, your web server will have sent data to Shopify Flow because of the trigger, and Shopify Flow will have sent this data to a web server that logged the information to its console because of the action.", + "synthesizer_name":"single_hop_specifc_query_synthesizer" + }, + { + "user_input":"how do you use toml when making new trigger extensions with shopify cli and what are the important parts to configure?", + "reference_contexts":[ + "When you create a new trigger extension using Shopify CLI, a basic version of the TOML configuration file structure is generated. In this guide, you'll learn about configuring the different sections and properties of the configuration file, including extension properties, extension fields, reference field types, custom field types, and more. This guide will also inform you how to make HTTP requests to Flow to start the workflows in which your extension is the trigger. ## TOML > Note: > Creating Flow extensions using Shopify CLI is an exciting new feature that is currently in development. As with any developing feature, it's important to note that the Flow's CLI capabilities will continue to evolve and improve over time. Developers can expect additional functionality, enhancements, and improvements to be added as development progresses. > >To create Flow extensions using [Shopify CLI](https:\/\/www.npmjs.com\/package\/@shopify\/cli), ensure you have the latest version installed. When you create a new trigger extension using Shopify CLI, you'll get a basic version of the TOML configuration file structure which should look like the following example: ```bash [[extensions]] name = \"Auction Bid\" type = \"flow_trigger\" handle = \"auction bid\" description = \"Your description\" [settings] [[settings.fields]] type = \"customer_reference\" [[settings.fields]] type = \"single_line_text_field\" key = \"your field key\" ``` ### Trigger extension properties Extension properties are listed in the `[[extensions]]` section and enable you to define the interface between Flow and your event. | Property name | Description | Rules | | --------------------------- | ------------------------------------------------------------- | -------------------------------------- | | `name`
Required<\/span> | Name of your extension. Will be the merchant-facing name of your task in the editor. This should be something that is human readable. | | | `type`
Required<\/span> | The type of your extension. This should always be set to \u201cflow_trigger\u201d for Flow triggers. | - Value must be `flow_trigger`. | `handle`
Required<\/span> | A unique identifier for your extension. This property cannot be changed once you\u2019ve run the `dev` or `deploy` command. | - Cannot exceed 30 characters.
- Must be unique across your app's extensions.
- Must only contain alphanumeric characters and hyphens. | | `description`
Optional<\/span> | A description of your extension. This description will be shown in the Flow editor navigation panel. | | ### Trigger extension fields Trigger extension fields are listed in the `[settings]` section, with each field using a `[[settings.field]]` header. These fields define the payload your event will send to Flow. You can add more than one field to your Flow trigger. The order of the fields in the TOML file is preserved when they're being rendered in the editor configuration panel. When sending a trigger payload, all fields defined in a trigger are required. | Property name | Description | Rules | | ------------------------------ | --------------------------------------------------------------------------------------------------- | --------------------------------| | `type`
Required<\/span> | The field type. | - [Accepted custom field types](#custom-field-types).
- [Accepted reference field types](#reference-field-types). | | `key`
Optional<\/span> | A unique key that identifies your field. This should be human readable since it will appear in the Flow editor in the environment picker menu. | - Required for custom field types.
Should only contain alphabetic values or spaces.
- This property is not valid for reference field types. | | `description`
Required<\/span> | A description of the field. This will appear in the Flow editor configuration panel. | ###" + ], + "reference":"When you create a new trigger extension using Shopify CLI, a basic version of the TOML configuration file structure is generated. Important parts to configure include extension properties such as `name`, `type`, `handle`, and `description` in the `[[extensions]]` section, and trigger extension fields in the `[settings]` section, where each field uses a `[[settings.field]]` header. These fields define the payload your event will send to Flow, and all fields defined in a trigger are required when sending a trigger payload.", + "synthesizer_name":"single_hop_specifc_query_synthesizer" + }, + { + "user_input":"How do you use Shopify App Bridge to render a custom configuration page in Shopify Flow, and what are the requirements for setting up the necessary endpoints?", + "reference_contexts":[ + "<1-hop>\n\nTo give merchants a more seamless action configuration experience, and to allow them to manage resources that are external to Shopify Flow, you can embed a page from your app in the Shopify Flow editor. In your Shopify Flow action configuration, merchants see a preview with an image and text that's fetched from your [custom configuration page preview URL](\/docs\/apps\/build\/flow\/actions\/endpoints#custom-configuration-page-preview). Merchants can click the button to access the custom configuration page.
\"A<\/figure> Your custom configuration page is then displayed in a frame in the Shopify admin.
\"The<\/figure> In this tutorial, you'll learn how to render a custom configuration page in Shopify Flow, customize the page frame, and access data relevant to your action in the custom configuration page context. ## Requirements You've created a [Partner account](https:\/\/www.shopify.com\/partners). You've [created an app](\/docs\/apps\/build\/scaffold app). ## Resources To implement this feature, you'll use the following: [Shopify App Bridge](\/docs\/api\/app bridge) App Bridge components App Bridge actions specific to the custom configuration page ## Implementing a custom configuration page To build a custom configuration page, you'll [use Shopify App Bridge to render a page from your app page in Shopify Flow](#use-shopify-app-bridge-to-render-your-app-page). From the context of the custom configuration page, you can then [access step and property information](#access-action-information) that you can use to display the appropriate information. You can also [add additional buttons](#add-buttons-to-the-app-bridge-title-bar) to the App Bridge title bar, or [trigger a redirect to the previous page](#return-to-the-previous-page). ##", + "<2-hop>\n\nUse Shopify App Bridge to render your app page > Note: > The specifics of the Custom Configuration Page integration varies between Shopify App Bridge versions. Make sure you implement the integration specific to your Shopify App Bridge version. To render your custom configuration page, you need to integrate Shopify App Bridge on the route that you want to render. To learn about setting up Shopify App Bridge, refer to one of the following pages: [Getting started with Shopify App Bridge](\/docs\/api\/app bridge\/previous versions\/app bridge from npm\/app setup) [Getting started with App Bridge React](\/docs\/api\/app bridge library#react) ### Access action information In the context of the custom configuration page, Shopify Flow makes the following action information available: **A `step_reference` search parameter **: `step_reference` is a unique ID for the step within a workflow, and can be used to identify the resource that the merchant is requesting. **Property data **: Properties contains the extension fields data that make up your [action payload schema](\/docs\/apps\/build\/flow\/actions\/endpoints#request). The properties are passed as an object containing the properties as key value pairs: ```json { : } ``` ### Shopify App Bridge integration for versions 4.X.X and up #### Register to the Custom Configuration Page's intent To access property data with Shopify App Bridge version 4.X.X and up, you will need to use the `shopify.intents` API. The following example code allows you to register to the Custom Configuration Page's intent: ```jsx import { useAppBridge } from '@shopify\/app-bridge-react' const Application = () => { const shopify = useAppBridge() const [intent, setIntent] = useState({}) useEffect(() => { const cleanup = shopify.intents.register((intent) => { setIntent(intent) }) return () => cleanup() }, []); return <>...<\/> } ``` The `intent` object will contain the following data: | Field | Data Type | Description | | ------- | --------- | ----------------------------------------------------------------------------------------------------- | | action | `string` | The action that has been registered for. In the case of the Custom Configuration Page, it will always be set to `configure`. | | type | `string` | A GID with the following structure: `gid:\/\/flow\/stepReference\/`. | | data | `object` | An object that contains the `properties` data. | | finish | `method` | A function that allows you to navigate to the previous page. | The register method also returns a cleanup function, which you can use to unregister from the intent when your component is unmounting. #### Return to the previous page By default, the title bar of the custom configuration page includes an **Exit** button that the user can use to return to the previous page. You can choose to trigger a redirect to the previous page using the `intent.finish()` method: ```jsx