Update scenario and documentation

* Scenario will now work with the latest version of scenario runner
* Remove prebuilt scenario runner
* Update README instructions on running the scenario

README.md

@@ -36,19 +36,19 @@ Neural Super Sampling (NSS) is a parameter prediction model for real-time tempor
 
 NSS is under active development with regular updates planned. As we increase the size and diversity of the training dataset we expect to see significant quality improvements. Follow Arm to stay up to date on the latest releases.
 
-The model is released under Arm's [AI Model Community License](https://huggingface.co/Arm/neural-super-sampling/blob/main/Arm_AI_Model_Community_License_v1_0_PRE-1154.pdf) which allows NSS to be retrained on datasets captured from your own content. Future releases of the [Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym) will provide the tools to capture and convert content for use in (re)retraining..
+The model is released under Arm's [AI Model Community License](https://huggingface.co/Arm/neural-super-sampling/blob/main/Arm_AI_Model_Community_License_v1_0_PRE-1154.pdf) which allows NSS to be retrained on datasets captured from your own content. Future releases of the [Neural Graphics Model Gym](https://github.com/arm/neural-graphics-model-gym) will provide the tools to capture and convert content for use in (re)retraining.
 
 ## Uses
 
 NSS can be directly integrated into graphics pipelines using ML extensions for Vulkan®. See included ML SDK for Vulkan [scenario](https://huggingface.co/Arm/neural-super-sampling/tree/main/scenario) for the simplest way to evaluate the model. The scenario includes the necessary pre- and post-processing compute shaders along with a single frame worth of input data.
 
-The recommended way of integrating the model into a graphics pipeline is by using the [VGF Library](https://github.com/arm/ai-ml-sdk-vgf-library/tree/main) from the ML SDK for Vulkan. 
+The recommended way of integrating the model into a graphics pipeline is by using the [VGF Library](https://github.com/arm/ai-ml-sdk-vgf-library/tree/main) from the ML SDK for Vulkan.
 
 NSS is released under a [permissive license](https://huggingface.co/Arm/neural-super-sampling/blob/main/Arm_AI_Model_Community_License_v1_0_PRE-1154.pdf) designed to foster innovation in the graphics industry and provide differentiation to content creators.
 
 ### Direct Use
 
-NSS has been integrated into Unreal Engine via two plugins, the [NSS Plugin for Unreal Engine](https://github.com/arm/neural-graphics-for-unreal/). See our [quick start guide](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/nss-unreal/) for step-by-step instructions on how to use NSS in Unreal® Engine.
+NSS has been integrated into Unreal Engine via the [NSS Plugin for Unreal Engine](https://github.com/arm/neural-graphics-for-unreal/). See our [quick start guide](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/nss-unreal/) for step-by-step instructions on how to use NSS in Unreal® Engine.
 
 ### Out-of-Scope Use
 
@@ -67,52 +67,60 @@ application running at a higher frame rate.
 
 ## How to Get Started with the Model
 
-This repository contains pre-trained weights and compiled NSS model in VGF format ready for integration with Vulkan applications.
+This repository contains pre-trained weights and a compiled NSS model in VGF format, ready for integration with Vulkan applications.
 
 The included Scenario demonstrates full execution of the model on a Vulkan compute-capable system. An Emulation Layer is provided to implement ML Extensions for Vulkan where it is not supported by the native Vulkan driver.
 
 ### Download and Prepare the Scenario
 
-Clone the NSS model repository from Hugging Face
-```powershell
+These instructions will work on both Windows and Linux. On Windows, we strongly recommend the use of PowerShell over `cmd.exe` ("Command Prompt").
+
+1. Ensure that Git LFS is installed. See [the project's website](https://git-lfs.com/) for more details.
+
+2. Clone the NSS model repository from Hugging Face:
+```bash
 git clone https://huggingface.co/Arm/neural-super-sampling
 cd neural-super-sampling
-git lfs pull # Make sure Git LFS is installed - https://git-lfs.com/
+git lfs pull
 ```
 
-### Run the Scenario
+### Install the Scenario Runner and Emulation Layers
 
-The NSS Hugging Face repository includes pre-built Windows® binaries for ML Emulation Layer for Vulkan and Scenario Runner. For other platforms,
-- build from source following the instructions for [Building the Emulation Layer from source](https://github.com/arm/ai-ml-emulation-layer-for-vulkan/blob/main/README.md#building-the-emulation-layer-from-source) and [Building the Scenario Runner from source](https://github.com/arm/ai-ml-sdk-scenario-runner/blob/main/README.md#building-scenario-runner-from-source)
-- adapt the instructions below accordingly 
+1. Follow the steps in the Python documentation's ["Create a new virtual environment"](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#create-a-new-virtual-environment) guide. We will assume below that you installed this environment into a `.venv` subdirectory.
 
-1. Set the required environment variables:
+2. Follow the steps in the Python documentation's ["Activate a virtual environment"](https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/#activate-a-virtual-environment) guide.
 
-   On Windows:
-   ```powershell
-   $env:VK_LAYER_PATH="$PWD\bin\windows-x86_64"
-   $env:VK_INSTANCE_LAYERS="VK_LAYER_ML_Graph_Emulation;VK_LAYER_ML_Tensor_Emulation"
-   ```
+   *Note for Windows users:* It is not necessary to determine the location of the Python interpreter as described in this guide. For reference, however, the correct PowerShell command to do this is `gcm python` rather than the given `where python`.
 
-   On Linux (assuming the Emulation Layer binaries and JSON files and Scenario Runner executable are copied to `bin/linux-x86_64`):
-   ```powershell
-   export LD_LIBRARY_PATH=$PWD/bin/linux-x86_64:$LD_LIBRARY_PATH
-   export VK_LAYER_PATH=$PWD/bin/linux-x86_64
-   export VK_INSTANCE_LAYERS=VK_LAYER_ML_Graph_Emulation:VK_LAYER_ML_Tensor_Emulation
+3. Install Arm's Scenario Runner and Emulation Layer into your new virtual environment:
+   ```bash
+   pip install ai-ml-sdk-scenario-runner ai-ml-emulation-layer-for-vulkan
    ```
-   
-2. Execute the scenario:
 
-   On Windows:
-   ```powershell
-   bin\windows-x86_64\scenario-runner.exe --scenario scenario\scenario.json --output out
-   ```
+4. The Emulation Layer's documentation explains how its environment variables should be configured on [Linux](https://github.com/arm/ai-ml-emulation-layer-for-vulkan/blob/main/README.md#usage-on-linux) and on [Windows](https://github.com/arm/ai-ml-emulation-layer-for-vulkan/blob/main/README.md#usage-on-windows). The following is a simplified summary, assuming that you are at the root of the NSS model repository.
+
+   * On Linux, enter these commands into Bash:
+     ```bash
+     EMULATION_LAYER_DIR=$(find $PWD/.venv -name emulation_layer -type d)
+     export LD_LIBRARY_PATH="$EMULATION_LAYER_DIR/deploy/lib:$LD_LIBRARY_PATH"
+     export VK_LAYER_PATH="$EMULATION_LAYER_DIR/deploy/share/vulkan/explicit_layer.d:$VK_LAYER_PATH"
+     export VK_INSTANCE_LAYERS="VK_LAYER_ML_Graph_Emulation:VK_LAYER_ML_Tensor_Emulation"
+     ```
 
-   On Linux:
-   ```powershell
-   bin/linux-x86_64/scenario-runner --scenario scenario/scenario.json --output out
+    * On Windows, enter these commands into PowerShell (`cmd.exe` requires a different syntax):
+    ```powershell
+    $env:VK_LAYER_PATH="$pwd\.venv\Lib\site-packages\emulation_layer\deploy\bin;$env:VK_LAYER_PATH"
+    $env:VK_INSTANCE_LAYERS="VK_LAYER_ML_Graph_Emulation;VK_LAYER_ML_Tensor_Emulation"
+    ```
+
+### Run the Scenario
+
+- Execute the Scenario Runner:
+
+   ```bash
+   scenario-runner --scenario scenario/scenario.json --output out
    ```
-   
+
 - Output images are encoded as `B10G11R11_UFLOAT`. This format is common for framebuffers but not widely supported by image viewers. Use [RenderDoc](https://renderdoc.org/) to view these images.
 
 ## Training and Evaluation
@@ -121,13 +129,13 @@ For background on NSS architecture and training read our blog: [How Neural Super
 
 Training and evaluation details, including model architecture code, training pipeline, and test configurations, are available at:
 
-- Model training code: https://github.com/arm/neural-graphics-model-gym
-- Examples and tutorials: https://github.com/arm/neural-graphics-model-gym-examples
-- Sample dataset: https://huggingface.co/datasets/Arm/neural-graphics-dataset
+- Model training code: <https://github.com/arm/neural-graphics-model-gym>
+- Examples and tutorials: <https://github.com/arm/neural-graphics-model-gym-examples>
+- Sample dataset: <https://huggingface.co/datasets/Arm/neural-graphics-dataset>
 
 ### 🔎 Model Explorer VGF extension
 
-The [VGF extension to Model Explorer](https://github.com/arm/vgf-adapter-model-explorer) provides a simple interface to visualize model and analyse VGF composition. 
+The [VGF extension to Model Explorer](https://github.com/arm/vgf-adapter-model-explorer) provides a simple interface to visualize model and analyse VGF composition.
 
 ![Model Explorer screenshot](resources/model-explorer-screenshot.png)
 
@@ -145,8 +153,8 @@ The [VGF extension to Model Explorer](https://github.com/arm/vgf-adapter-model-e
 🙋🏻‍♀️ For questions or feedback please [start a discussion](https://huggingface.co/Arm/neural-super-sampling/discussions)
 
 ## Trademark notice
-Arm® is a registered trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere.
-
-Windows® is a trademark of the Microsoft group of companies.
 
-Vulkan® is a registered trademark of the [Khronos® Group](https://www.khronos.org/legal/trademarks).
+* Arm® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.
+* Python® is a registered trademark of the Python Software Foundation.
+* Vulkan® is a registered trademark of the [Khronos® Group](https://www.khronos.org/legal/trademarks).
+* Windows® is a trademark of the Microsoft group of companies.

bin/windows-x86_64/VkLayer_Graph.dll

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:0a2fd54f62bef850685bcc4331714e0590a0a3aef28ea2dd6aa8c9e6f68f4da0
-size 7437312

bin/windows-x86_64/VkLayer_Graph.json

@@ -1,29 +0,0 @@
-{
-    "file_format_version": "1.0.0",
-    "layer": {
-        "name": "VK_LAYER_ML_Graph_Emulation",
-        "type": "INSTANCE",
-        "library_path": ".\\VkLayer_Graph.dll",
-        "api_version": "1.3.0",
-        "implementation_version": "1",
-        "description": "ML Graph Emulation Layer",
-        "functions": {
-            "vkGetInstanceProcAddr": "graphGetInstanceProcAddr",
-            "vkGetDeviceProcAddr": "graphGetDeviceProcAddr"
-        },
-        "device_extensions": [
-            {
-                "name": "VK_ARM_data_graph",
-                "spec_version": "1",
-                "entrypoints": [
-                    "vkGetPhysicalDeviceGraphInstructionSetsARM",
-                    "vkCreateGraphPipelinesARM",
-                    "vkCreateGraphPipelineSessionARM",
-                    "vkGetGraphPipelineSessionMemoryRequirementsARM",
-                    "vkBindGraphPipelineSessionMemoryARM",
-                    "vkDestroyGraphPipelineSessionARM",
-                    "vkCmdDispatchGraphARM"
-                ]
-            }]
-    }
-}

bin/windows-x86_64/VkLayer_Tensor.dll

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:15a7e7f9b4ff74f530a550669740f1c8b5295bdbe9a0474da3e3b9e906c4ce76
-size 5689344

bin/windows-x86_64/VkLayer_Tensor.json

@@ -1,31 +0,0 @@
-{
-    "file_format_version": "1.0.0",
-    "layer": {
-        "name": "VK_LAYER_ML_Tensor_Emulation",
-        "type": "INSTANCE",
-        "library_path": ".\\VkLayer_Tensor.dll",
-        "api_version": "1.3.0",
-        "implementation_version": "1",
-        "description": "ML Tensor Emulation Layer",
-        "functions": {
-            "vkGetInstanceProcAddr": "tensorGetInstanceProcAddr",
-            "vkGetDeviceProcAddr": "tensorGetDeviceProcAddr"
-        },
-        "device_extensions": [
-            {
-                "name": "VK_ARM_tensors",
-                "spec_version": "1",
-                "entrypoints": [
-                    "vkCreateTensorARM",
-                    "vkDestroyTensorARM",
-                    "vkCreateTensorViewARM",
-                    "vkDestroyTensorViewARM",
-                    "vkGetTensorMemoryRequirementsARM",
-                    "vkBindTensorMemoryARM",
-                    "vkGetDeviceTensorMemoryRequirementsARM",
-                    "vkCmdCopyTensorARM"
-                ]
-            }
-        ]
-    }
-}

bin/windows-x86_64/scenario-runner.exe

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:5a99f1076dec1f504d64387969a3d18dc687c7f95c505dca812e4fcbca60f3d2
-size 5285376

scenario/scenario.json

@@ -1,11 +1,5 @@
 {
     "commands": [
-        {
-            "mark_boundary": {
-                "frame_id": 0,
-                "resources": []
-            }
-        },
         {
             "dispatch_compute": {
                 "shader_ref": "0_pre_process",
@@ -212,7 +206,6 @@
         },
         {
             "mark_boundary": {
-                "frame_id": 1,
                 "resources": [
                     "out_colour"
                 ]
@@ -504,6 +497,9 @@
                 "dst": "./out_feedback.dds",
                 "format": "VK_FORMAT_R8G8B8A8_SNORM",
                 "shader_access": "writeonly",
+                "memory_group": {
+                    "id": "out_feedback"
+                },
                 "mips": 1,
                 "min_filter": "LINEAR",
                 "mag_filter": "LINEAR",
@@ -524,8 +520,8 @@
                 ],
                 "format": "VK_FORMAT_R8_SINT",
                 "shader_access": "readwrite",
-                "alias_target": {
-                    "resource_ref": "out_feedback"
+                "memory_group": {
+                    "id": "out_feedback"
                 },
                 "tiling": "LINEAR"
             }
@@ -541,6 +537,9 @@
                 ],
                 "format": "VK_FORMAT_R8G8B8A8_SNORM",
                 "shader_access": "readonly",
+                "memory_group": {
+                    "id": "out_tp"
+                },
                 "mips": 1,
                 "min_filter": "LINEAR",
                 "mag_filter": "LINEAR",
@@ -561,8 +560,8 @@
                 ],
                 "format": "VK_FORMAT_R8_SINT",
                 "shader_access": "readwrite",
-                "alias_target": {
-                    "resource_ref": "out_tp"
+                "memory_group": {
+                    "id": "out_tp"
                 },
                 "tiling": "LINEAR"
             }
@@ -578,6 +577,9 @@
                 ],
                 "format": "VK_FORMAT_R8G8B8A8_SNORM",
                 "shader_access": "readonly",
+                "memory_group": {
+                    "id": "out_k3"
+                },
                 "mips": 1,
                 "min_filter": "LINEAR",
                 "mag_filter": "LINEAR",
@@ -597,8 +599,8 @@
                 ],
                 "format": "VK_FORMAT_R8_SINT",
                 "shader_access": "readwrite",
-                "alias_target": {
-                    "resource_ref": "out_k3"
+                "memory_group": {
+                    "id": "out_k3"
                 },
                 "tiling": "LINEAR"
             }
@@ -614,6 +616,9 @@
                 ],
                 "format": "VK_FORMAT_R8G8B8A8_SNORM",
                 "shader_access": "readonly",
+                "memory_group": {
+                    "id": "out_k2"
+                },
                 "mips": 1,
                 "min_filter": "LINEAR",
                 "mag_filter": "LINEAR",
@@ -633,8 +638,8 @@
                 ],
                 "format": "VK_FORMAT_R8_SINT",
                 "shader_access": "readwrite",
-                "alias_target": {
-                    "resource_ref": "out_k2"
+                "memory_group": {
+                    "id": "out_k2"
                 },
                 "tiling": "LINEAR"
             }
@@ -650,6 +655,9 @@
                 ],
                 "format": "VK_FORMAT_R8G8B8A8_SNORM",
                 "shader_access": "readonly",
+                "memory_group": {
+                    "id": "out_k1"
+                },
                 "mips": 1,
                 "min_filter": "LINEAR",
                 "mag_filter": "LINEAR",
@@ -669,8 +677,8 @@
                 ],
                 "format": "VK_FORMAT_R8_SINT",
                 "shader_access": "readwrite",
-                "alias_target": {
-                    "resource_ref": "out_k1"
+                "memory_group": {
+                    "id": "out_k1"
                 },
                 "tiling": "LINEAR"
             }
@@ -686,6 +694,9 @@
                 ],
                 "format": "VK_FORMAT_R8G8B8A8_SNORM",
                 "shader_access": "readonly",
+                "memory_group": {
+                    "id": "out_k0"
+                },
                 "mips": 1,
                 "min_filter": "LINEAR",
                 "mag_filter": "LINEAR",
@@ -705,8 +716,8 @@
                 ],
                 "format": "VK_FORMAT_R8_SINT",
                 "shader_access": "readwrite",
-                "alias_target": {
-                    "resource_ref": "out_k0"
+                "memory_group": {
+                    "id": "out_k0"
                 },
                 "tiling": "LINEAR"
             }
@@ -840,4 +851,4 @@
             }
         }
     ]
-}
+}