Add files using upload-large-folder tool

pythonProject/.venv/Lib/site-packages/onnx-1.19.0.dist-info/METADATA

@@ -0,0 +1,136 @@
+Metadata-Version: 2.4
+Name: onnx
+Version: 1.19.0
+Summary: Open Neural Network Exchange
+Author-email: ONNX Contributors <onnx-technical-discuss@lists.lfaidata.foundation>
+License: Apache License v2.0
+Project-URL: Homepage, https://onnx.ai/
+Project-URL: Repository, https://github.com/onnx/onnx
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.22
+Requires-Dist: protobuf>=4.25.1
+Requires-Dist: typing_extensions>=4.7.1
+Requires-Dist: ml_dtypes
+Provides-Extra: reference
+Requires-Dist: Pillow; extra == "reference"
+Dynamic: license-file
+
+<!--
+Copyright (c) ONNX Project Contributors
+
+SPDX-License-Identifier: Apache-2.0
+-->
+
+<p align="center"><img width="40%" src="https://github.com/onnx/onnx/raw/main/docs/onnx-horizontal-color.png" /></p>
+
+[![PyPI - Version](https://img.shields.io/pypi/v/onnx.svg)](https://pypi.org/project/onnx)
+[![CI](https://github.com/onnx/onnx/actions/workflows/main.yml/badge.svg)](https://github.com/onnx/onnx/actions/workflows/main.yml)
+[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3313/badge)](https://bestpractices.coreinfrastructure.org/projects/3313)
+[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/onnx/onnx/badge)](https://api.securityscorecards.dev/projects/github.com/onnx/onnx)
+[![REUSE compliant](https://api.reuse.software/badge/github.com/onnx/onnx)](https://api.reuse.software/info/github.com/onnx/onnx)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+
+[Open Neural Network Exchange (ONNX)](https://onnx.ai) is an open ecosystem that empowers AI developers
+to choose the right tools as their project evolves. ONNX provides an open source format for AI models, both deep learning and traditional ML. It defines an extensible computation graph model, as well as definitions of built-in operators and standard
+data types. Currently we focus on the capabilities needed for inferencing (scoring).
+
+ONNX is [widely supported](http://onnx.ai/supported-tools) and can be found in many frameworks, tools, and hardware. Enabling interoperability between different frameworks and streamlining the path from research to production helps increase the speed of innovation in the AI community. We invite the community to join us and further evolve ONNX.
+
+# Use ONNX
+
+* [Documentation of ONNX Python Package](https://onnx.ai/onnx/)
+* [Tutorials for creating ONNX models](https://github.com/onnx/tutorials)
+* [Pre-trained ONNX models](https://github.com/onnx/models)
+
+# Learn about the ONNX spec
+
+* [Overview](https://github.com/onnx/onnx/blob/main/docs/Overview.md)
+* [ONNX intermediate representation spec](https://github.com/onnx/onnx/blob/main/docs/IR.md)
+* [Versioning principles of the spec](https://github.com/onnx/onnx/blob/main/docs/Versioning.md)
+* [Operators documentation](https://github.com/onnx/onnx/blob/main/docs/Operators.md)
+* [Operators documentation](https://onnx.ai/onnx/operators/index.html) (latest release)
+* [Python API Overview](https://github.com/onnx/onnx/blob/main/docs/PythonAPIOverview.md)
+
+# Programming utilities for working with ONNX Graphs
+
+* [Shape and Type Inference](https://github.com/onnx/onnx/blob/main/docs/ShapeInference.md)
+* [Graph Optimization](https://github.com/onnx/optimizer)
+* [Opset Version Conversion](https://github.com/onnx/onnx/blob/main/docs/docsgen/source/api/version_converter.md)
+
+# Contribute
+
+ONNX is a community project and the open governance model is described [here](https://github.com/onnx/onnx/blob/main/community/readme.md). We encourage you to join the effort and contribute feedback, ideas, and code. You can participate in the [Special Interest Groups](https://github.com/onnx/onnx/blob/main/community/sigs.md) and [Working Groups](https://github.com/onnx/onnx/blob/main/community/working-groups.md) to shape the future of ONNX.
+
+Check out our [contribution guide](https://github.com/onnx/onnx/blob/main/CONTRIBUTING.md) to get started.
+
+If you think some operator should be added to ONNX specification, please read
+[this document](https://github.com/onnx/onnx/blob/main/docs/AddNewOp.md).
+
+# Community meetings
+
+The schedules of the regular meetings of the Steering Committee, the working groups and the SIGs can be found [here](https://onnx.ai/calendar)
+
+Community Meetups are held at least once a year. Content from previous community meetups are at:
+
+* 2020.04.09 <https://lf-aidata.atlassian.net/wiki/spaces/DL/pages/14091402/LF+AI+Day+-ONNX+Community+Virtual+Meetup+-+Silicon+Valley+-+2020+April+9>
+* 2020.10.14 <https://lf-aidata.atlassian.net/wiki/spaces/DL/pages/14092138/LF+AI+Day+-+ONNX+Community+Workshop+-+2020+October+14>
+* 2021.03.24 <https://lf-aidata.atlassian.net/wiki/spaces/DL/pages/14092424/Instructions+for+Event+Hosts+-+LF+AI+Data+Day+-+ONNX+Virtual+Community+Meetup+-+March+2021>
+* 2021.10.21 <https://lf-aidata.atlassian.net/wiki/spaces/DL/pages/14093194/LF+AI+Data+Day+ONNX+Community+Virtual+Meetup+-+October+2021>
+* 2022.06.24 <https://lf-aidata.atlassian.net/wiki/spaces/DL/pages/14093969/ONNX+Community+Day+-+2022+June+24>
+* 2023.06.28 <https://lf-aidata.atlassian.net/wiki/spaces/DL/pages/14094507/ONNX+Community+Day+2023+-+June+28>
+
+
+
+# Discuss
+
+We encourage you to open [Issues](https://github.com/onnx/onnx/issues), or use [Slack](https://lfaifoundation.slack.com/) (If you have not joined yet, please use this [link](https://join.slack.com/t/lfaifoundation/shared_invite/zt-o65errpw-gMTbwNr7FnNbVXNVFkmyNA) to join the group) for more real-time discussion.
+
+# Follow Us
+
+Stay up to date with the latest ONNX news. [[Facebook](https://www.facebook.com/onnxai/)] [[Twitter](https://twitter.com/onnxai)]
+
+# Roadmap
+
+A roadmap process takes place every year. More details can be found [here](https://github.com/onnx/steering-committee/tree/main/roadmap)
+
+# Installation
+
+ONNX released packages are published in PyPi.
+
+```sh
+pip install onnx # or pip install onnx[reference] for optional reference implementation dependencies
+```
+
+[ONNX weekly packages](https://pypi.org/project/onnx-weekly/) are published in PyPI to enable experimentation and early testing.
+
+Detailed install instructions, including Common Build Options and Common Errors can be found [here](https://github.com/onnx/onnx/blob/main/INSTALL.md)
+
+# Testing
+
+ONNX uses [pytest](https://docs.pytest.org) as test driver. In order to run tests, you will first need to install `pytest`:
+
+```sh
+pip install pytest
+```
+
+After installing pytest, use the following command to run tests.
+
+```sh
+pytest
+```
+
+# Development
+
+Check out the [contributor guide](https://github.com/onnx/onnx/blob/main/CONTRIBUTING.md) for instructions.
+
+# License
+
+[Apache License v2.0](LICENSE)
+
+# Code of Conduct
+
+[ONNX Open Source Code of Conduct](https://onnx.ai/codeofconduct.html)

pythonProject/.venv/Lib/site-packages/onnx-1.19.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (1.19.0)
+Root-Is-Purelib: false
+Tag: cp310-cp310-win_amd64
+

pythonProject/.venv/Lib/site-packages/onnx-1.19.0.dist-info/licenses/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

pythonProject/.venv/Lib/site-packages/onnx-1.19.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+onnx

pythonProject/.venv/Lib/site-packages/onnx/test/cpp/utf8_conversion_test.cc

@@ -0,0 +1,27 @@
+// Copyright (c) ONNX Project Contributors
+
+/*
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+#ifdef _WIN32
+#include <string>
+
+#include "gtest/gtest.h"
+#include "onnx/common/path.h"
+namespace ONNX_NAMESPACE::Test {
+
+TEST(UTF8Test, WideStringConvertion) {
+  std::string utf8_str(u8"世界，你好！");
+  EXPECT_EQ(ONNX_NAMESPACE::wstring_to_utf8str(ONNX_NAMESPACE::utf8str_to_wstring(utf8_str)), utf8_str);
+}
+
+TEST(UTF8Test, TryConvertUTF8) {
+  std::string utf8_str(u8"世界，你好！");
+  auto wstr = ONNX_NAMESPACE::utf8str_to_wstring(utf8_str);
+  auto wstr2 = ONNX_NAMESPACE::utf8str_to_wstring(
+      std::string(reinterpret_cast<const char*>(wstr.c_str()), sizeof(std::wstring::value_type) * wstr.size()), true);
+  EXPECT_EQ(wstr, wstr2);
+}
+} // namespace ONNX_NAMESPACE::Test
+#endif

pythonProject/.venv/Lib/site-packages/onnx/test/version_converter/automatic_conversion_test_base.py

@@ -0,0 +1,149 @@
+# Copyright (c) ONNX Project Contributors
+
+# SPDX-License-Identifier: Apache-2.0
+from __future__ import annotations
+
+import string
+import unittest
+from typing import TYPE_CHECKING, Any, cast
+
+import onnx
+from onnx import TensorProto, ValueInfoProto, helper, shape_inference, version_converter
+
+if TYPE_CHECKING:
+    from collections.abc import Sequence
+
+LATEST_OPSET = onnx.defs.onnx_opset_version()
+
+
+class TestAutomaticConversion(unittest.TestCase):
+    def _test_model_conversion(
+        self, to_opset: int, model: str | onnx.ModelProto
+    ) -> None:
+        if isinstance(model, str):
+            model = onnx.parser.parse_model(model)
+        onnx.checker.check_model(model)
+        shape_inference.infer_shapes(model, strict_mode=True)
+
+        converted = version_converter.convert_version(model, to_opset)
+        onnx.checker.check_model(converted)
+        shape_inference.infer_shapes(converted, strict_mode=True)
+
+    def _test_model_conversion_fails(
+        self, to_opset: int, model: str | onnx.ModelProto
+    ) -> None:
+        if isinstance(model, str):
+            model = onnx.parser.parse_model(model)
+        onnx.checker.check_model(model)
+        shape_inference.infer_shapes(model, strict_mode=True)
+
+        with self.assertRaises(RuntimeError):
+            version_converter.convert_version(model, to_opset)
+
+    def _test_op_conversion(
+        self,
+        op: str,
+        from_opset: int,
+        input_shapes: Sequence[Sequence[int | None] | str] = ((3, 4, 5),),
+        output_shapes: Sequence[Sequence[int | None]] = ((3, 4, 5),),
+        input_types: Sequence[Any] | None = None,
+        output_types: Sequence[Any] | None = None,
+        initializer: Sequence[Any] = (),
+        attrs: dict[str, Any] | None = None,
+        seq_inputs: Sequence[int] = (),
+        seq_outputs: Sequence[int] = (),
+        optional_inputs: Sequence[int] = (),
+        optional_outputs: Sequence[int] = (),
+        is_upgrade: bool = True,
+    ) -> None:
+        """Test conversion.
+
+        Args:
+            op: A string representing the name of the operator to test.
+            from_opset: An integer representing the lowest opset version to convert.
+            input_shapes: A sequence of tuples or strings representing the shapes of the input tensors.
+                The default value is ((3, 4, 5),).
+            output_shapes: A sequence of tuples representing the shapes of the output tensors.
+                The default value is ((3, 4, 5),).
+            input_types: An optional sequence of types representing the data types of the input tensors.
+            output_types: An optional sequence of types representing the data types of the output tensors.
+            initializer: A sequence of values representing the initial values of the input tensors.
+            attrs: An optional dictionary of attributes for the operator.
+            seq_inputs: A sequence of integers representing the indices of the input tensors that are sequences.
+            seq_outputs: A sequence of integers representing the indices of the output tensors that are sequences.
+            optional_inputs: A sequence of integers representing the indices of the input tensors that are optional.
+            optional_outputs: A sequence of integers representing the indices of the output tensors that are optional.
+            is_upgrade: A boolean value indicating whether to run the version converter from from_opset to
+                the most recent opset version (True) or from the most recent opset version to from_opset (False).
+                The default value is True. In both cases, runs checker and shape inference on the final model.
+        """
+        if attrs is None:
+            attrs = {}
+
+        n_inputs = len(input_shapes)
+        letters = list(string.ascii_lowercase)[:n_inputs]
+        input_names = [
+            letter if shape != "" else ""
+            for (letter, shape) in zip(letters, input_shapes)
+        ]
+        if input_types is None:
+            input_types = [TensorProto.FLOAT] * n_inputs
+        is_sequence = [0 if id not in seq_inputs else 1 for id in range(n_inputs)]
+        is_optional = [0 if id not in optional_inputs else 1 for id in range(n_inputs)]
+        # turn empty strings into [0] to ease type analysis, even though those entries
+        # will be ignored
+        input_shapes_cast = cast(
+            "list[list[int]]",
+            [[0] if isinstance(shape, str) else shape for shape in input_shapes],
+        )
+        inputs: list[ValueInfoProto] = []
+        for name, ttype, shape, is_seq, is_opt in zip(
+            input_names, input_types, input_shapes_cast, is_sequence, is_optional
+        ):
+            if name != "":
+                if is_seq:
+                    inputs += [
+                        helper.make_tensor_sequence_value_info(name, ttype, shape)
+                    ]
+                elif is_opt:
+                    type_proto = helper.make_tensor_type_proto(ttype, shape)
+                    optional_type_proto = helper.make_optional_type_proto(type_proto)
+                    inputs += [helper.make_value_info(name, optional_type_proto)]
+                else:
+                    inputs += [helper.make_tensor_value_info(name, ttype, shape)]
+
+        n_outputs = len(output_shapes)
+        output_names = list(string.ascii_lowercase)[n_inputs : n_inputs + n_outputs]
+        if output_types is None:
+            output_types = [TensorProto.FLOAT] * n_outputs
+        is_sequence = [0 if id not in seq_outputs else 1 for id in range(n_outputs)]
+        is_optional = [
+            0 if id not in optional_outputs else 1 for id in range(n_outputs)
+        ]
+        output_shapes_cast = cast(
+            "list[list[int]]",
+            [[0] if isinstance(shape, str) else shape for shape in output_shapes],
+        )
+        outputs: list[ValueInfoProto] = []
+        for name, ttype, shape, is_seq, is_opt in zip(
+            output_names, output_types, output_shapes_cast, is_sequence, is_optional
+        ):
+            if is_seq:
+                outputs += [helper.make_tensor_sequence_value_info(name, ttype, shape)]
+            elif is_opt:
+                type_proto = helper.make_tensor_type_proto(ttype, shape)
+                optional_type_proto = helper.make_optional_type_proto(type_proto)
+                outputs += [helper.make_value_info(name, optional_type_proto)]
+            else:
+                outputs += [helper.make_tensor_value_info(name, ttype, shape)]
+
+        node = helper.make_node(op, input_names, output_names, **attrs)
+        graph = helper.make_graph([node], op, inputs, outputs, initializer)
+        start_opset = from_opset if is_upgrade else LATEST_OPSET
+        end_opset = LATEST_OPSET if is_upgrade else from_opset
+        original = helper.make_model(
+            graph,
+            producer_name="test",
+            opset_imports=[helper.make_opsetid("", start_opset)],
+        )
+        self._test_model_conversion(end_opset, original)

pythonProject/.venv/Lib/site-packages/onnx/test/version_converter/automatic_downgrade_test.py

@@ -0,0 +1,106 @@
+# Copyright (c) ONNX Project Contributors
+
+# SPDX-License-Identifier: Apache-2.0
+from __future__ import annotations
+
+import unittest
+
+import automatic_conversion_test_base
+import numpy as np
+import parameterized
+
+import onnx
+from onnx import helper
+
+#####################################################################################
+# Every test calls _test_op_conversion to downgrade a model from the most recent opset version
+# to a early version and runs checker + shape inference on the downgraded model.
+####################################################################################
+
+
+class TestAutomaticDowngrade(automatic_conversion_test_base.TestAutomaticConversion):
+    def _test_op_downgrade(self, op: str, *args, **kwargs):
+        self._test_op_conversion(op, *args, **kwargs, is_upgrade=False)
+
+    @parameterized.parameterized.expand(
+        [
+            "ReduceL1",
+            "ReduceL2",
+            "ReduceLogSum",
+            "ReduceLogSumExp",
+            "ReduceMean",
+            "ReduceMax",
+            "ReduceMin",
+            "ReduceProd",
+            "ReduceSum",
+            "ReduceSumSquare",
+        ]
+    )
+    def test_reduce_ops(self, op) -> None:
+        # TODO: need to add test cases for missing axes input which depends on this pr:
+        # https://github.com/onnx/onnx/pull/5613
+        axes = helper.make_tensor(
+            "b", onnx.TensorProto.INT64, dims=[3], vals=np.array([0, 1, 2])
+        )
+        self._test_op_downgrade(
+            op,
+            from_opset=13,
+            input_shapes=[[3, 4, 5], [3]],
+            output_shapes=[[1, 1, 1]],
+            input_types=[onnx.TensorProto.FLOAT, onnx.TensorProto.INT64],
+            initializer=[axes],
+        )
+
+    def test_dft20_no_axis(self) -> None:
+        self._test_model_conversion(
+            to_opset=19,
+            model="""
+            <ir_version: 9, opset_import: [ "" : 20]>
+            dft_no_axis (float[N, M, 1] x) => (float[N, M, 2] y)
+            {
+                y = DFT (x)
+            }
+        """,
+        )
+
+    def test_dft20_initializer_axis(self) -> None:
+        self._test_model_conversion(
+            to_opset=19,
+            model="""
+            <ir_version: 9, opset_import: [ "" : 20]>
+            dft_no_axis (float[N, M, 1] x, int64 dft_length) => (float[N, K, 2] y)
+            <int64 axis = {1}>
+            {
+                y = DFT (x, dft_length, axis)
+            }
+        """,
+        )
+
+    def test_dft20_constant_axis(self) -> None:
+        self._test_model_conversion(
+            to_opset=19,
+            model="""
+            <ir_version: 9, opset_import: [ "" : 20]>
+            dft_no_axis (float[N, M, 1] x, int64 dft_length) => (float[N, K, 2] y)
+            {
+                axis = Constant <value = int64{1}>()
+                y = DFT (x, dft_length, axis)
+            }
+        """,
+        )
+
+    def test_dft20_unknown_axis(self) -> None:
+        self._test_model_conversion_fails(
+            to_opset=19,
+            model="""
+            <ir_version: 9, opset_import: [ "" : 20]>
+            dft_no_axis (float[N, M, 1] x, int64 dft_length, int64 axis) => (float[P, K, 2] y)
+            {
+                y = DFT (x, dft_length, axis)
+            }
+        """,
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

pythonProject/.venv/Lib/site-packages/onnx/test/version_converter/automatic_upgrade_test.py

@@ -0,0 +1,1964 @@
+# Copyright (c) ONNX Project Contributors
+
+# SPDX-License-Identifier: Apache-2.0
+from __future__ import annotations
+
+import unittest
+
+import automatic_conversion_test_base
+import numpy as np
+
+import onnx
+from onnx import TensorProto, helper
+
+#####################################################################################
+# Every test calls _test_op_conversion to upgrade a model from an initial opset version
+# to the most recent version and runs checker and shape inference on the final upgraded model.
+####################################################################################
+
+
+class TestAutomaticUpgrade(automatic_conversion_test_base.TestAutomaticConversion):
+    @classmethod
+    def setUpClass(cls):
+        cls.tested_ops = []
+
+    def _test_op_upgrade(self, op, *args, **kwargs):
+        self.tested_ops.append(op)
+        self._test_op_conversion(op, *args, **kwargs, is_upgrade=True)
+
+    def test_Abs(self) -> None:
+        self._test_op_upgrade("Abs", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Acosh(self) -> None:
+        self._test_op_upgrade("Acosh", 9)
+
+    def test_Acos(self) -> None:
+        self._test_op_upgrade("Acos", 7)
+
+    def test_And(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "And",
+            7,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.BOOL, TensorProto.BOOL],
+            [TensorProto.BOOL],
+        )
+
+    def test_Asinh(self) -> None:
+        self._test_op_upgrade("Asinh", 9)
+
+    def test_Atanh(self) -> None:
+        self._test_op_upgrade("Atanh", 9)
+
+    def test_Add_1(self) -> None:
+        self._test_op_upgrade(
+            "Add", 1, [[3, 4, 5], [3, 4, 5]], attrs={"consumed_inputs": [0]}
+        )
+
+    def test_Add_2(self) -> None:
+        self._test_op_upgrade(
+            "Add", 1, [[3, 4, 5], [5]], attrs={"consumed_inputs": [0], "broadcast": 1}
+        )
+
+    def test_Add_3(self) -> None:
+        self._test_op_upgrade(
+            "Add",
+            1,
+            [[3, 4, 5], [3]],
+            attrs={"consumed_inputs": [0], "broadcast": 1, "axis": 0},
+        )
+
+    def test_AffineGrid_2D(self) -> None:
+        N, _, H, W = 2, 3, 5, 6
+        self._test_op_upgrade("AffineGrid", 20, [[N, 2, 3], [4]], [[N, H, W, 2]])
+
+    def test_AffineGrid_3D(self) -> None:
+        N, _, D, H, W = 2, 3, 4, 5, 6
+        self._test_op_upgrade("AffineGrid", 20, [[N, 3, 4], [5]], [[N, D, H, W, 3]])
+
+    def test_ArgMax_1(self) -> None:
+        self._test_op_upgrade(
+            "ArgMax", 7, [[2, 3, 4]], [[1, 3, 4]], output_types=[TensorProto.INT64]
+        )
+
+    def test_ArgMax_2(self) -> None:
+        self._test_op_upgrade(
+            "ArgMax",
+            7,
+            [[2, 3, 4]],
+            [[2, 1, 4]],
+            output_types=[TensorProto.INT64],
+            attrs={"axis": 1},
+        )
+
+    def test_ArgMin_1(self) -> None:
+        self._test_op_upgrade(
+            "ArgMin", 7, [[2, 3, 4]], [[1, 3, 4]], output_types=[TensorProto.INT64]
+        )
+
+    def test_ArgMin_2(self) -> None:
+        self._test_op_upgrade(
+            "ArgMin",
+            7,
+            [[2, 3, 4]],
+            [[2, 1, 4]],
+            output_types=[TensorProto.INT64],
+            attrs={"axis": 1},
+        )
+
+    def test_Asin(self) -> None:
+        self._test_op_upgrade("Asin", 7)
+
+    def test_Atan(self) -> None:
+        self._test_op_upgrade("Atan", 7)
+
+    def test_Attention_1(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8]],
+            [[2, 3, 4, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_Attention_2(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 9, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8]],
+            [[2, 9, 4, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_Attention_3(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 10]],
+            [[2, 3, 4, 10]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_Attention_4(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8]],
+            [[2, 3, 4, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+            attrs={"scale": 2.0},
+        )
+
+    def test_Attention_5(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8]],
+            [[2, 3, 4, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+            attrs={"is_causal": 1},
+        )
+
+    def test_Attention_6(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8], [4, 6]],
+            [[2, 3, 4, 8]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+            ],
+            [TensorProto.FLOAT],
+        )
+
+    def test_Attention_7(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8], [4, 6]],
+            [[2, 3, 4, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.BOOL],
+            [TensorProto.FLOAT],
+        )
+
+    def test_Attention_8(self) -> None:
+        self._test_op_upgrade(
+            "Attention",
+            23,
+            [[2, 3, 4, 8], [2, 3, 6, 8], [2, 3, 6, 8]],
+            [[2, 3, 4, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+            attrs={"softcap": 2.0},
+        )
+
+    def test_AveragePool(self) -> None:
+        self._test_op_upgrade(
+            "AveragePool",
+            1,
+            [[1, 1, 5, 5]],
+            [[1, 1, 4, 4]],
+            attrs={"kernel_shape": [2, 2]},
+        )
+
+    def test_Bernoulli(self) -> None:
+        self._test_op_upgrade("Bernoulli", 15)
+
+    def test_BitShift(self) -> None:
+        self._test_op_upgrade(
+            "BitShift",
+            11,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.UINT8, TensorProto.UINT8],
+            [TensorProto.UINT8],
+            attrs={"direction": "RIGHT"},
+        )
+
+    def test_BatchNormalization_1(self) -> None:
+        self._test_op_upgrade(
+            "BatchNormalization",
+            1,
+            [[1, 3], [3], [3], [3], [3]],
+            [[1, 3]],
+            attrs={"consumed_inputs": [1, 1], "is_test": 1, "spatial": 1},
+        )
+
+    def test_BatchNormalization_2(self) -> None:
+        self._test_op_upgrade(
+            "BatchNormalization",
+            14,
+            [[1, 3], [3], [3], [3], [3]],
+            [[1, 3], [3], [3]],
+            attrs={"training_mode": 1},
+        )
+
+    def test_Cast(self) -> None:
+        # 5->6 adapter is missing
+        self._test_op_upgrade(
+            "Cast", 6, [[2, 3]], [[2, 3]], [TensorProto.INT64], attrs={"to": 1}
+        )
+
+    def test_Ceil(self) -> None:
+        self._test_op_upgrade("Ceil", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Celu(self) -> None:
+        self._test_op_upgrade("Celu", 12)
+
+    def test_Clip_1(self) -> None:
+        self._test_op_upgrade("Clip", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Clip_2(self) -> None:
+        self._test_op_upgrade("Clip", 1, attrs={"consumed_inputs": [0], "min": -1.4})
+
+    def test_Clip_3(self) -> None:
+        self._test_op_upgrade("Clip", 1, attrs={"consumed_inputs": [0], "max": 2.6})
+
+    def test_Clip_4(self) -> None:
+        self._test_op_upgrade(
+            "Clip", 1, attrs={"consumed_inputs": [0], "min": -1.4, "max": 2.6}
+        )
+
+    def test_Col2Im_4D(self) -> None:
+        self._test_op_upgrade("Col2Im", 18, [[1, 5, 5], [2], [2]], [[1, 1, 5, 5]])
+
+    def test_Col2Im_5D(self) -> None:
+        self._test_op_upgrade("Col2Im", 18, [[1, 10, 12], [3], [3]], [[1, 2, 3, 4, 5]])
+
+    def test_Compress(self) -> None:
+        self._test_op_upgrade(
+            "Compress",
+            9,
+            [[6, 7], [3]],
+            [[3]],
+            [TensorProto.FLOAT, TensorProto.BOOL],
+            [TensorProto.FLOAT],
+        )
+
+    def test_Concat(self) -> None:
+        self._test_op_upgrade("Concat", 1, [[2, 3], [2, 4]], [[2, 7]])
+
+    def test_constant(self) -> None:
+        value = helper.make_tensor(
+            "Value",
+            TensorProto.FLOAT,
+            dims=[3, 4, 5],
+            vals=np.random.rand(3, 4, 5).astype(np.float32).tobytes(),
+            raw=True,
+        )
+        self._test_op_upgrade("Constant", 1, [], attrs={"value": value})
+
+    def test_ConstantOfShape(self) -> None:
+        self._test_op_upgrade("ConstantOfShape", 9, [[3]])
+
+    def test_Conv_1(self) -> None:
+        self._test_op_upgrade(
+            "Conv", 1, [[1, 3, 5, 5], [4, 3, 2, 2], [4]], [[1, 4, 4, 4]]
+        )
+
+    def test_Conv_2(self) -> None:
+        self._test_op_upgrade(
+            "Conv", 1, [[1, 3, 5, 5], [4, 3, 2, 2], [4]], [[1, 4, 4, 4]]
+        )
+
+    def test_Conv_3(self) -> None:
+        self._test_op_upgrade(
+            "Conv",
+            1,
+            [[1, 3, 5, 5], [4, 1, 2, 2], [4]],
+            [[1, 4, 3, 7]],
+            attrs={
+                "dilations": [1, 2],
+                "group": 3,
+                "pads": [0, 1, 2, 3],
+                "strides": [2, 1],
+            },
+        )
+
+    def test_Convinteger(self) -> None:
+        self._test_op_upgrade(
+            "ConvInteger",
+            10,
+            [[1, 3, 5, 5], [4, 3, 2, 2], [4]],
+            [[1, 4, 4, 4]],
+            [TensorProto.UINT8, TensorProto.UINT8, TensorProto.UINT8],
+            [TensorProto.INT32],
+        )
+
+    def test_ConvTranspose(self) -> None:
+        self._test_op_upgrade(
+            "ConvTranspose", 1, [[1, 1, 5, 5], [1, 1, 3, 3]], [[1, 1, 7, 7]]
+        )
+
+    def test_DeformConv(self) -> None:
+        self._test_op_upgrade(
+            "DeformConv",
+            19,
+            [[1, 1, 3, 3], [1, 1, 2, 2], [1, 8, 2, 2]],
+            [[1, 1, 2, 2]],
+        )
+
+    def test_Cosh(self) -> None:
+        self._test_op_upgrade("Cosh", 9)
+
+    def test_Cos(self) -> None:
+        self._test_op_upgrade("Cos", 7)
+
+    def test_Cumsum(self) -> None:
+        self._test_op_upgrade(
+            "CumSum",
+            11,
+            [[3, 4, 5], []],
+            [[3, 4, 5]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_DepthToSpace(self) -> None:
+        self._test_op_upgrade(
+            "DepthToSpace", 1, [[1, 8, 3, 3]], [[1, 2, 6, 6]], attrs={"blocksize": 2}
+        )
+
+    def test_DequantizeLinear(self) -> None:
+        self._test_op_upgrade(
+            "DequantizeLinear",
+            10,
+            [[2, 3], [], []],
+            [[2, 3]],
+            [TensorProto.INT8, TensorProto.FLOAT, TensorProto.INT8],
+        )
+
+    def test_Det_1(self) -> None:
+        self._test_op_upgrade("Det", 11, [[3, 5, 5]], [[3]])
+
+    def test_Det_2(self) -> None:
+        self._test_op_upgrade("Det", 11, [[5, 5]], [[]])
+
+    def test_DynamicQuantizeLinear(self) -> None:
+        self._test_op_upgrade(
+            "DynamicQuantizeLinear",
+            11,
+            [[3, 4, 5]],
+            [[3, 4, 5], [], []],
+            output_types=[TensorProto.UINT8, TensorProto.FLOAT, TensorProto.UINT8],
+        )
+
+    def test_Div(self) -> None:
+        self._test_op_upgrade(
+            "Div", 1, [[3, 4, 5], [3, 1, 5]], attrs={"consumed_inputs": [0]}
+        )
+
+    def test_Dropout(self) -> None:
+        self._test_op_upgrade(
+            "Dropout", 1, attrs={"consumed_inputs": [0], "is_test": 1}
+        )
+
+    def test_Einsum_1(self) -> None:
+        self._test_op_upgrade(
+            "Einsum",
+            12,
+            [[3, 4, 5], [3, 5, 6]],
+            [[3, 4, 6]],
+            attrs={"equation": "bij, bjk -> bik"},
+        )
+
+    def test_Einsum_2(self) -> None:
+        self._test_op_upgrade(
+            "Einsum", 12, [[4, 5]], [[5, 4]], attrs={"equation": "ij->ji"}
+        )
+
+    def test_Elu(self) -> None:
+        self._test_op_upgrade("Elu", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Equal(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "Equal", 7, [[2, 3], [2, 3]], [[2, 3]], output_types=[TensorProto.BOOL]
+        )
+
+    def test_Erf(self) -> None:
+        self._test_op_upgrade("Erf", 9)
+
+    def test_Exp(self) -> None:
+        self._test_op_upgrade("Exp", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Expand(self) -> None:
+        shape = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[4], vals=np.array([5, 2, 6, 4])
+        )
+        self._test_op_upgrade(
+            "Expand",
+            8,
+            [[2, 1, 4], [4]],
+            [[5, 2, 6, 4]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+            initializer=[shape],
+        )
+
+    def test_EyeLike(self) -> None:
+        self._test_op_upgrade("EyeLike", 9, [[4, 5]], [[4, 5]])
+
+    def test_Flatten(self) -> None:
+        self._test_op_upgrade("Flatten", 1, [[3, 4, 5]], [[3, 20]], attrs={"axis": 1})
+
+    def test_Floor(self) -> None:
+        self._test_op_upgrade("Floor", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Gather(self) -> None:
+        self._test_op_upgrade(
+            "Gather",
+            1,
+            [[3, 4, 5], [6, 7]],
+            [[6, 7, 4, 5]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_GatherElements(self) -> None:
+        self._test_op_upgrade(
+            "GatherElements",
+            11,
+            [[3, 4, 5], [6, 7]],
+            [[6, 7]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_GatherND(self) -> None:
+        self._test_op_upgrade("GatherND", 11, [[1, 2, 3], [1, 2, 3]], [[1, 2]])
+
+    def test_Gelu_approximate_tanh(self) -> None:
+        self._test_op_upgrade("Gelu", 20, attrs={"approximate": "tanh"})
+
+    def test_Gelu(self) -> None:
+        self._test_op_upgrade("Gelu", 20)
+
+    def test_Gemm(self) -> None:
+        self._test_op_upgrade("Gemm", 1, [[5, 4], [4, 3], [3]], [[5, 3]])
+
+    def test_GlobalAveragePool(self) -> None:
+        self._test_op_upgrade("GlobalAveragePool", 1, [[1, 3, 10, 10]], [[1, 3, 1, 1]])
+
+    def test_GlobalMaxPool(self) -> None:
+        self._test_op_upgrade("GlobalMaxPool", 1, [[1, 3, 10, 10]], [[1, 3, 1, 1]])
+
+    def test_GlobalLpPool(self) -> None:
+        # 1->2 adapter is missing
+        self._test_op_upgrade("GlobalLpPool", 2, [[1, 3, 10, 10]], [[1, 3, 1, 1]])
+
+    def test_Greater(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "Greater", 7, [[2, 3], [2, 3]], [[2, 3]], output_types=[TensorProto.BOOL]
+        )
+
+    def test_GreaterOrEqual(self) -> None:
+        self._test_op_upgrade(
+            "GreaterOrEqual",
+            12,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            output_types=[TensorProto.BOOL],
+        )
+
+    def test_GridSample(self) -> None:
+        self._test_op_upgrade(
+            "GridSample",
+            16,
+            [[1, 1, 3, 3], [1, 3, 3, 2]],
+            [[1, 1, 3, 3]],
+            input_types=[TensorProto.FLOAT, TensorProto.FLOAT],
+            output_types=[TensorProto.FLOAT],
+            attrs={"mode": "nearest", "padding_mode": "border", "align_corners": 1},
+        )
+
+    def test_GRU_1(self) -> None:
+        # 2->3, 6->7 adapters are missing
+        self._test_op_upgrade(
+            "GRU",
+            7,
+            [[5, 3, 4], [1, 18, 4], [1, 18, 4]],
+            [[5, 1, 3, 6], [1, 3, 6]],
+            attrs={"hidden_size": 6},
+        )
+
+    def test_GRU_2(self) -> None:
+        # 2->3, 6->7 adapters are missing
+        self._test_op_upgrade(
+            "GRU",
+            7,
+            [[5, 3, 4], [2, 18, 4], [2, 18, 4]],
+            [[5, 2, 3, 6], [2, 3, 6]],
+            attrs={"hidden_size": 6, "direction": "bidirectional"},
+        )
+
+    def test_GRU_3(self) -> None:
+        # 2->3, 6->7 adapters are missing
+        self._test_op_upgrade(
+            "GRU",
+            7,
+            [[5, 3, 4], [1, 18, 4], [1, 18, 4], [1, 24], [5], [1, 5, 6]],
+            [[5, 1, 3, 6], [1, 3, 6]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+            ],
+            attrs={"hidden_size": 6},
+        )
+
+    def test_HardSigmoid(self) -> None:
+        self._test_op_upgrade("HardSigmoid", 1, attrs={"consumed_inputs": [0]})
+
+    def test_HardSwish(self) -> None:
+        self._test_op_upgrade("HardSwish", 14)
+
+    def test_Hardmax(self) -> None:
+        self._test_op_upgrade("Hardmax", 1)
+
+    def test_Identity(self) -> None:
+        self._test_op_upgrade("Identity", 1)
+
+    def test_If(self) -> None:
+        sub_output = [
+            helper.make_tensor_value_info("out", TensorProto.FLOAT, [3, 4, 5])
+        ]
+        then_tensor = helper.make_tensor(
+            "Value",
+            TensorProto.FLOAT,
+            dims=[3, 4, 5],
+            vals=np.random.rand(3, 4, 5).astype(np.float32).tobytes(),
+            raw=True,
+        )
+        then_node = helper.make_node("Constant", [], ["out"], value=then_tensor)
+        then_graph = helper.make_graph([then_node], "then_graph", [], sub_output, [])
+        else_tensor = helper.make_tensor(
+            "Value",
+            TensorProto.FLOAT,
+            dims=[3, 4, 5],
+            vals=np.random.rand(3, 4, 5).astype(np.float32).tobytes(),
+            raw=True,
+        )
+        else_node = helper.make_node("Constant", [], ["out"], value=else_tensor)
+        else_graph = helper.make_graph([else_node], "else_graph", [], sub_output, [])
+        self._test_op_upgrade(
+            "If",
+            1,
+            [[0]],
+            [[3, 4, 5]],
+            [TensorProto.BOOL],
+            attrs={"then_branch": then_graph, "else_branch": else_graph},
+        )
+
+    def test_ImageDecoder(self) -> None:
+        self._test_op_upgrade(
+            "ImageDecoder",
+            20,
+            [[None]],
+            [[None, None, 3]],
+            input_types=[TensorProto.UINT8],
+            output_types=[TensorProto.UINT8],
+        )
+
+    def test_InstanceNormalization(self) -> None:
+        self._test_op_upgrade(
+            "InstanceNormalization",
+            1,
+            [[1, 3], [3], [3]],
+            [[1, 3]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_IsInf(self) -> None:
+        self._test_op_upgrade(
+            "IsInf", 10, [[2, 3]], [[2, 3]], output_types=[TensorProto.BOOL]
+        )
+
+    def test_IsNaN(self) -> None:
+        self._test_op_upgrade(
+            "IsNaN", 9, [[2, 3]], [[2, 3]], output_types=[TensorProto.BOOL]
+        )
+
+    def test_LeakyRelu(self) -> None:
+        self._test_op_upgrade("LeakyRelu", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Less(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "Less", 7, [[2, 3], [2, 3]], [[2, 3]], output_types=[TensorProto.BOOL]
+        )
+
+    def test_LessOrEqual(self) -> None:
+        self._test_op_upgrade(
+            "LessOrEqual",
+            12,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            output_types=[TensorProto.BOOL],
+        )
+
+    def test_Log(self) -> None:
+        self._test_op_upgrade("Log", 1, attrs={"consumed_inputs": [0]})
+
+    def test_LogSoftmax(self) -> None:
+        self._test_op_upgrade("LogSoftmax", 1)
+
+    def test_Loop_1(self) -> None:
+        iter_count = onnx.helper.make_tensor_value_info(
+            "iter_count", onnx.TensorProto.INT64, []
+        )
+        cond_in = onnx.helper.make_tensor_value_info(
+            "cond_in", onnx.TensorProto.BOOL, []
+        )
+        x_in = onnx.helper.make_tensor_value_info("x_in", onnx.TensorProto.FLOAT, [1])
+        cond_out = onnx.helper.make_tensor_value_info(
+            "cond_out", onnx.TensorProto.BOOL, []
+        )
+        x_out = onnx.helper.make_tensor_value_info("x_out", onnx.TensorProto.FLOAT, [1])
+        x_scan = onnx.helper.make_tensor_value_info(
+            "x_scan", onnx.TensorProto.FLOAT, [1]
+        )
+        const = onnx.helper.make_node(
+            "Constant",
+            inputs=[],
+            outputs=["one"],
+            value=onnx.helper.make_tensor(
+                name="value",
+                data_type=onnx.TensorProto.FLOAT,
+                dims=[1],
+                vals=np.array([1]).astype(np.float32).astype(float),
+            ),
+        )
+        add = onnx.helper.make_node("Add", inputs=["x_in", "one"], outputs=["x_out"])
+        id_1 = onnx.helper.make_node("Identity", inputs=["x_out"], outputs=["x_scan"])
+        id_2 = onnx.helper.make_node(
+            "Identity", inputs=["cond_in"], outputs=["cond_out"]
+        )
+        loop_body = onnx.helper.make_graph(
+            [const, add, id_1, id_2],
+            "loop_body",
+            [iter_count, cond_in, x_in],
+            [cond_out, x_out, x_scan],
+        )
+        self._test_op_upgrade(
+            "Loop",
+            1,
+            [[], "", [1]],
+            [[1], [5, 1]],
+            [TensorProto.INT64, TensorProto.BOOL, TensorProto.FLOAT],
+            attrs={"body": loop_body},
+        )
+
+    def test_Loop_2(self) -> None:
+        iter_count = onnx.helper.make_tensor_value_info(
+            "iter_count", onnx.TensorProto.INT64, []
+        )
+        cond_in = onnx.helper.make_tensor_value_info(
+            "cond_in", onnx.TensorProto.BOOL, []
+        )
+        x_in = onnx.helper.make_tensor_value_info(
+            "x_in", onnx.TensorProto.FLOAT, [2, 1]
+        )
+        cond_out = onnx.helper.make_tensor_value_info(
+            "cond_out", onnx.TensorProto.BOOL, []
+        )
+        x_out = onnx.helper.make_tensor_value_info(
+            "x_out", onnx.TensorProto.FLOAT, [2, 1]
+        )
+        squeeze = onnx.helper.make_node(
+            "Squeeze", inputs=["x_in"], outputs=["squeeze_out"], axes=[1]
+        )
+        unsqueeze = onnx.helper.make_node(
+            "Unsqueeze", inputs=["squeeze_out"], outputs=["x_out"], axes=[1]
+        )
+        identity = onnx.helper.make_node(
+            "Identity", inputs=["cond_in"], outputs=["cond_out"]
+        )
+        loop_body = onnx.helper.make_graph(
+            [squeeze, unsqueeze, identity],
+            "loop_body",
+            [iter_count, cond_in, x_in],
+            [cond_out, x_out],
+        )
+        self._test_op_upgrade(
+            "Loop",
+            12,
+            [[], "", [2, 1]],
+            [[2, 1]],
+            [TensorProto.INT64, TensorProto.BOOL, TensorProto.FLOAT],
+            attrs={"body": loop_body},
+        )
+
+    def test_LpNormalization(self) -> None:
+        self._test_op_upgrade("LpNormalization", 1)
+
+    def test_LpPool(self) -> None:
+        # 1->2 adapter is missing
+        self._test_op_upgrade(
+            "LpPool", 2, [[1, 1, 5, 5]], [[1, 1, 4, 4]], attrs={"kernel_shape": [2, 2]}
+        )
+
+    def test_LRN_1(self) -> None:
+        self._test_op_upgrade("LRN", 1, attrs={"size": 3})
+
+    def test_LRN_2(self) -> None:
+        self._test_op_upgrade(
+            "LRN", 1, [[2, 3, 4, 5]], [[2, 3, 4, 5]], attrs={"size": 3}
+        )
+
+    def test_LSTM_1(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "LSTM",
+            7,
+            [[5, 3, 4], [1, 24, 4], [1, 24, 4]],
+            [[5, 1, 3, 6], [1, 3, 6], [1, 3, 6]],
+            attrs={"hidden_size": 6},
+        )
+
+    def test_LSTM_2(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "LSTM",
+            7,
+            [[5, 3, 4], [2, 24, 4], [2, 24, 4]],
+            [[5, 2, 3, 6], [2, 3, 6], [2, 3, 6]],
+            attrs={"hidden_size": 6, "direction": "bidirectional"},
+        )
+
+    def test_LSTM_3(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "LSTM",
+            7,
+            [
+                [5, 3, 4],
+                [1, 24, 4],
+                [1, 24, 4],
+                [1, 48],
+                [5],
+                [1, 5, 6],
+                [1, 5, 6],
+                [1, 18],
+            ],
+            [[5, 1, 3, 6], [1, 3, 6], [1, 3, 6]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+            ],
+            attrs={"hidden_size": 6},
+        )
+
+    def test_MatMul_1(self) -> None:
+        self._test_op_upgrade("MatMul", 1, [[2, 3], [3, 4]], [[2, 4]])
+
+    def test_MatMul_2(self) -> None:
+        self._test_op_upgrade("MatMul", 1, [[5, 2, 3], [5, 3, 4]], [[5, 2, 4]])
+
+    def test_MatMulInteger_1(self) -> None:
+        self._test_op_upgrade(
+            "MatMulInteger",
+            10,
+            [[2, 3], [3, 4]],
+            [[2, 4]],
+            [TensorProto.INT8, TensorProto.INT8],
+            [TensorProto.INT32],
+        )
+
+    def test_MatMulInteger_2(self) -> None:
+        self._test_op_upgrade(
+            "MatMulInteger",
+            10,
+            [[2, 3], [3, 4], [], []],
+            [[2, 4]],
+            [TensorProto.INT8, TensorProto.INT8, TensorProto.INT8, TensorProto.INT8],
+            [TensorProto.INT32],
+        )
+
+    def test_MatMulInteger_3(self) -> None:
+        self._test_op_upgrade(
+            "MatMulInteger",
+            10,
+            [[2, 3], [3, 4], [2], [4]],
+            [[2, 4]],
+            [TensorProto.INT8, TensorProto.INT8, TensorProto.INT8, TensorProto.INT8],
+            [TensorProto.INT32],
+        )
+
+    def test_Max(self) -> None:
+        self._test_op_upgrade(
+            "Max",
+            1,
+            [[2, 3, 4], [2, 3, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_MaxPool_1(self) -> None:
+        self._test_op_upgrade(
+            "MaxPool", 1, [[1, 1, 5, 5]], [[1, 1, 4, 4]], attrs={"kernel_shape": [2, 2]}
+        )
+
+    def test_MaxPool_2(self) -> None:
+        self._test_op_upgrade(
+            "MaxPool",
+            8,
+            [[1, 1, 5, 5]],
+            [[1, 1, 4, 4], [1, 1, 4, 4]],
+            output_types=[TensorProto.FLOAT, TensorProto.INT64],
+            attrs={"kernel_shape": [2, 2]},
+        )
+
+    def test_MaxRoiPool(self) -> None:
+        self._test_op_upgrade(
+            "MaxRoiPool",
+            1,
+            [[2, 3, 20, 20], [4, 5]],
+            [[4, 3, 3, 3]],
+            attrs={"pooled_shape": [3, 3]},
+        )
+
+    def test_MaxUnpool(self) -> None:
+        self._test_op_upgrade(
+            "MaxUnpool",
+            9,
+            [[1, 1, 5, 5], [1, 1, 5, 5]],
+            [[1, 1, 6, 6]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+            attrs={"kernel_shape": [2, 2]},
+        )
+
+    def test_Mean(self) -> None:
+        self._test_op_upgrade(
+            "Mean",
+            1,
+            [[2, 3, 4], [2, 3, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_MeanVarianceNormalization(self) -> None:
+        self._test_op_upgrade("MeanVarianceNormalization", 9, attrs={"axes": [1, 2]})
+
+    def test_Min(self) -> None:
+        self._test_op_upgrade(
+            "Min",
+            1,
+            [[2, 3, 4], [2, 3, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_Mish(self) -> None:
+        self._test_op_upgrade("Mish", 18)
+
+    def test_Mod_1(self) -> None:
+        self._test_op_upgrade("Mod", 10, [[2, 3], [2, 3]], [[2, 3]])
+
+    def test_Mod_2(self) -> None:
+        self._test_op_upgrade("Mod", 10, [[2, 3], [2, 3]], [[2, 3]], attrs={"fmod": 1})
+
+    def test_Mul(self) -> None:
+        self._test_op_upgrade(
+            "Mul",
+            1,
+            [[2, 3, 4], [2, 1, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_Multinomial(self) -> None:
+        self._test_op_upgrade(
+            "Multinomial",
+            7,
+            [[3, 5]],
+            [[3, 7]],
+            output_types=[TensorProto.INT32],
+            attrs={"sample_size": 7},
+        )
+
+    def test_Neg(self) -> None:
+        self._test_op_upgrade("Neg", 1, attrs={"consumed_inputs": [0]})
+
+    def test_NegativeLogLikelihoodLoss_1(self) -> None:
+        self._test_op_upgrade(
+            "NegativeLogLikelihoodLoss",
+            12,
+            [[3, 4, 5], [3, 5]],
+            [[]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_NegativeLogLikelihoodLoss_2(self) -> None:
+        self._test_op_upgrade(
+            "NegativeLogLikelihoodLoss",
+            12,
+            [[3, 4, 5], [3, 5], [4]],
+            [[]],
+            [TensorProto.FLOAT, TensorProto.INT64, TensorProto.FLOAT],
+        )
+
+    def test_NonMaxSuppression(self) -> None:
+        self._test_op_upgrade(
+            "NonMaxSuppression",
+            10,
+            [[2, 3, 4], [3, 5, 6]],
+            [[2, 3]],
+            output_types=[TensorProto.INT64],
+        )
+
+    def test_NonZero(self) -> None:
+        self._test_op_upgrade(
+            "NonZero", 9, [[3, 3]], [[2, 4]], output_types=[TensorProto.INT64]
+        )
+
+    def test_Not(self) -> None:
+        self._test_op_upgrade(
+            "Not", 1, [[2, 3]], [[2, 3]], [TensorProto.BOOL], [TensorProto.BOOL]
+        )
+
+    def test_OneHot(self) -> None:
+        self._test_op_upgrade("OneHot", 9, [[3, 4, 5], [], [2]], [[3, 4, 5, 6]])
+
+    def test_Or(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "Or",
+            7,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.BOOL, TensorProto.BOOL],
+            [TensorProto.BOOL],
+        )
+
+    def test_Pad(self) -> None:
+        # 1->2 adapter is missing
+        self._test_op_upgrade(
+            "Pad", 2, [[3, 4]], [[5, 8]], attrs={"pads": [1, 2, 1, 2], "value": 1.5}
+        )
+
+    def test_Pow(self) -> None:
+        self._test_op_upgrade("Pow", 1, [[2, 3, 4], [2, 3, 4]], [[2, 3, 4]])
+
+    def test_PRelu(self) -> None:
+        self._test_op_upgrade(
+            "PRelu",
+            1,
+            [[2, 3, 4], [2, 3, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_QLinearConv(self) -> None:
+        self._test_op_upgrade(
+            "QLinearConv",
+            10,
+            [[1, 3, 5, 5], [], [], [4, 3, 2, 2], [], [], [], []],
+            [[1, 4, 4, 4]],
+        )
+
+    def test_QLinearMatMul(self) -> None:
+        self._test_op_upgrade(
+            "QLinearMatMul", 10, [[2, 3], [], [], [3, 4], [], [], [], []], [[2, 4]]
+        )
+
+    def test_QuantizeLinear(self) -> None:
+        self._test_op_upgrade(
+            "QuantizeLinear",
+            10,
+            [[3, 4, 5], [], []],
+            [[3, 4, 5]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.UINT8],
+            [TensorProto.UINT8],
+        )
+
+    def test_RandomNormal(self) -> None:
+        self._test_op_upgrade(
+            "RandomNormal", 1, [], [[3, 4, 5]], attrs={"shape": [3, 4, 5]}
+        )
+
+    def test_RandomNormalLike(self) -> None:
+        like = helper.make_tensor(
+            "a",
+            TensorProto.FLOAT,
+            dims=[3, 4, 5],
+            vals=np.random.rand(3, 4, 5).astype(np.float32).tobytes(),
+            raw=True,
+        )
+        self._test_op_upgrade(
+            "RandomNormalLike", 1, [[3, 4, 5]], [[3, 4, 5]], initializer=[like]
+        )
+
+    def test_RandomUniform(self) -> None:
+        self._test_op_upgrade(
+            "RandomUniform", 1, [], [[3, 4, 5]], attrs={"shape": [3, 4, 5]}
+        )
+
+    def test_RandomUniformLike(self) -> None:
+        like = helper.make_tensor(
+            "a",
+            TensorProto.FLOAT,
+            dims=[3, 4, 5],
+            vals=np.random.rand(3, 4, 5).astype(np.float32).tobytes(),
+            raw=True,
+        )
+        self._test_op_upgrade(
+            "RandomUniformLike", 1, [[3, 4, 5]], [[3, 4, 5]], initializer=[like]
+        )
+
+    def test_Range(self) -> None:
+        start = helper.make_tensor("a", TensorProto.FLOAT, dims=[], vals=np.array([0]))
+        end = helper.make_tensor("b", TensorProto.FLOAT, dims=[], vals=np.array([12]))
+        step = helper.make_tensor("c", TensorProto.FLOAT, dims=[], vals=np.array([2]))
+        self._test_op_upgrade(
+            "Range", 11, [[], [], []], [[6]], initializer=[start, end, step]
+        )
+
+    def test_Reciprocal(self) -> None:
+        self._test_op_upgrade("Reciprocal", 1, attrs={"consumed_inputs": [0]})
+
+    def test_ReduceL1(self) -> None:
+        self._test_op_upgrade("ReduceL1", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceL2(self) -> None:
+        self._test_op_upgrade("ReduceL2", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceLogSum(self) -> None:
+        self._test_op_upgrade("ReduceLogSum", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceLogSumExp(self) -> None:
+        self._test_op_upgrade("ReduceLogSumExp", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceMean(self) -> None:
+        self._test_op_upgrade("ReduceMean", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceMax(self) -> None:
+        self._test_op_upgrade("ReduceMax", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceMin(self) -> None:
+        self._test_op_upgrade("ReduceMin", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceProd(self) -> None:
+        self._test_op_upgrade("ReduceProd", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceSum(self) -> None:
+        self._test_op_upgrade("ReduceSum", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_ReduceSumSquare(self) -> None:
+        self._test_op_upgrade("ReduceSumSquare", 1, [[3, 4, 5]], [[1, 1, 1]])
+
+    def test_Relu(self) -> None:
+        self._test_op_upgrade("Relu", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Reshape(self) -> None:
+        self._test_op_upgrade(
+            "Reshape",
+            1,
+            [[3, 4, 5]],
+            [[3, 10, 2]],
+            attrs={"consumed_inputs": [0], "shape": [3, 10, 2]},
+        )
+
+    def test_Resize(self) -> None:
+        self._test_op_upgrade("Resize", 10, [[3, 4, 5], [3]], [[3, 8, 15]])
+
+    def test_ReverseSequence(self) -> None:
+        self._test_op_upgrade(
+            "ReverseSequence",
+            10,
+            [[3, 4, 5], [4]],
+            [[3, 4, 5]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_RNN_1(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "RNN",
+            7,
+            [[5, 3, 4], [1, 6, 4], [1, 6, 4]],
+            [[5, 1, 3, 6], [1, 3, 6]],
+            attrs={"hidden_size": 6},
+        )
+
+    def test_RNN_2(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "RNN",
+            7,
+            [[5, 3, 4], [2, 6, 4], [2, 6, 4]],
+            [[5, 2, 3, 6], [2, 3, 6]],
+            attrs={"hidden_size": 6, "direction": "bidirectional"},
+        )
+
+    def test_RNN_3(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "RNN",
+            7,
+            [[5, 3, 4], [1, 6, 4], [1, 6, 4], [1, 12], [5], [1, 5, 6]],
+            [[5, 1, 3, 6], [1, 3, 6]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+            ],
+            attrs={"hidden_size": 6},
+        )
+
+    def test_RoiAlign_1(self) -> None:
+        self._test_op_upgrade(
+            "RoiAlign",
+            10,
+            [[2, 3, 20, 20], [10, 4], [10]],
+            [[10, 3, 1, 1]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_RoiAlign_2(self) -> None:
+        self._test_op_upgrade(
+            "RoiAlign",
+            16,
+            [[2, 3, 20, 20], [10, 4], [10]],
+            [[10, 3, 1, 1]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.INT64],
+            attrs={"coordinate_transformation_mode": "half_pixel"},
+        )
+
+    def test_RotaryEmbedding_1(self) -> None:
+        self._test_op_upgrade(
+            "RotaryEmbedding",
+            23,
+            [[2, 4, 3, 8], [2, 3, 4], [2, 3, 4]],
+            [[2, 4, 3, 8]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_RotaryEmbedding_2(self) -> None:
+        self._test_op_upgrade(
+            "RotaryEmbedding",
+            23,
+            [[2, 4, 3, 8], [50, 4], [50, 4], [2, 3]],
+            [[2, 4, 3, 8]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            [TensorProto.FLOAT],
+        )
+
+    def test_RotaryEmbedding_3(self) -> None:
+        self._test_op_upgrade(
+            "RotaryEmbedding",
+            23,
+            [[2, 3, 32], [50, 4], [50, 4], [2, 3]],
+            [[2, 3, 32]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            [TensorProto.FLOAT],
+            attrs={"num_heads": 4},
+        )
+
+    def test_RotaryEmbedding_4(self) -> None:
+        self._test_op_upgrade(
+            "RotaryEmbedding",
+            23,
+            [[2, 4, 3, 8], [50, 4], [50, 4], [2, 3]],
+            [[2, 4, 3, 8]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            [TensorProto.FLOAT],
+            attrs={"interleaved": 1},
+        )
+
+    def test_RotaryEmbedding_5(self) -> None:
+        self._test_op_upgrade(
+            "RotaryEmbedding",
+            23,
+            [[2, 4, 3, 8], [50, 4], [50, 4], [2, 3]],
+            [[2, 4, 3, 8]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            [TensorProto.FLOAT],
+            attrs={"rotary_embedding_dim": 4},
+        )
+
+    def test_Round(self) -> None:
+        self._test_op_upgrade("Round", 11)
+
+    def test_RMSNormalization(self) -> None:
+        self._test_op_upgrade(
+            "RMSNormalization",
+            23,
+            [[2, 3, 4, 5], [4, 5]],
+            [[2, 3, 4, 5]],
+            input_types=[TensorProto.FLOAT, TensorProto.FLOAT],
+            output_types=[TensorProto.FLOAT],
+            attrs={"axis": 2},
+        )
+
+    def test_Scatter(self) -> None:
+        self._test_op_upgrade(
+            "Scatter",
+            9,
+            [[2, 3], [1, 2], [1, 2]],
+            [[2, 3]],
+            [TensorProto.FLOAT, TensorProto.INT64, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_ScatterElements_1(self) -> None:
+        self._test_op_upgrade(
+            "ScatterElements",
+            11,
+            [[2, 3], [1, 2], [1, 2]],
+            [[2, 3]],
+            [TensorProto.FLOAT, TensorProto.INT64, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_ScatterElements_2(self) -> None:
+        self._test_op_upgrade(
+            "ScatterElements",
+            16,
+            [[2, 3], [1, 2], [1, 2]],
+            [[2, 3]],
+            [TensorProto.FLOAT, TensorProto.INT64, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+            attrs={"reduction": "add"},
+        )
+
+    def test_ScatterND_1(self) -> None:
+        self._test_op_upgrade(
+            "ScatterND",
+            11,
+            [[2, 3], [1, 2], [1, 2]],
+            [[2, 3]],
+            [TensorProto.FLOAT, TensorProto.INT64, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+        )
+
+    def test_ScatterND_2(self) -> None:
+        self._test_op_upgrade(
+            "ScatterND",
+            16,
+            [[2, 3], [1, 2], [1, 2]],
+            [[2, 3]],
+            [TensorProto.FLOAT, TensorProto.INT64, TensorProto.FLOAT],
+            [TensorProto.FLOAT],
+            attrs={"reduction": "mul"},
+        )
+
+    def test_Scan(self) -> None:
+        sum_in = onnx.helper.make_tensor_value_info(
+            "sum_in", onnx.TensorProto.FLOAT, [2]
+        )
+        next_in = onnx.helper.make_tensor_value_info(
+            "next_in", onnx.TensorProto.FLOAT, [2]
+        )
+        sum_out = onnx.helper.make_tensor_value_info(
+            "sum_out", onnx.TensorProto.FLOAT, [2]
+        )
+        scan_out = onnx.helper.make_tensor_value_info(
+            "scan_out", onnx.TensorProto.FLOAT, [2]
+        )
+        add_node = onnx.helper.make_node(
+            "Add", inputs=["sum_in", "next_in"], outputs=["sum_out"]
+        )
+        id_node = onnx.helper.make_node(
+            "Identity", inputs=["sum_out"], outputs=["scan_out"]
+        )
+        body = onnx.helper.make_graph(
+            [add_node, id_node], "scan_body", [sum_in, next_in], [sum_out, scan_out]
+        )
+        self._test_op_upgrade(
+            "Scan",
+            8,
+            ["", [1, 2], [1, 3, 2]],
+            [[1, 2], [1, 3, 2]],
+            attrs={"body": body, "num_scan_inputs": 1},
+        )
+
+    def test_Selu(self) -> None:
+        self._test_op_upgrade("Selu", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Shape(self) -> None:
+        self._test_op_upgrade(
+            "Shape", 1, [[3, 4, 5]], [[3]], output_types=[TensorProto.INT64]
+        )
+
+    def test_Shrink(self) -> None:
+        self._test_op_upgrade("Shrink", 9)
+
+    def test_Sigmoid(self) -> None:
+        self._test_op_upgrade("Sigmoid", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Sign(self) -> None:
+        self._test_op_upgrade("Sign", 9)
+
+    def test_Sinh(self) -> None:
+        self._test_op_upgrade("Sinh", 9)
+
+    def test_Sin(self) -> None:
+        self._test_op_upgrade("Sin", 7)
+
+    def test_Size(self) -> None:
+        self._test_op_upgrade(
+            "Size", 1, [[3, 4, 5]], [[]], output_types=[TensorProto.INT64]
+        )
+
+    def test_Slice(self) -> None:
+        self._test_op_upgrade(
+            "Slice",
+            1,
+            [[3, 4, 5]],
+            [[3, 2, 2]],
+            attrs={"axes": [1, 2], "starts": [0, 1], "ends": [2, 3]},
+        )
+
+    def test_Softmax_0(self) -> None:
+        self._test_op_upgrade("Softmax", 1, attrs={"axis": 0})
+
+    def test_Softmax_1(self) -> None:
+        self._test_op_upgrade("Softmax", 1, attrs={"axis": 1})
+
+    def test_Softmax_2(self) -> None:
+        self._test_op_upgrade("Softmax", 1, attrs={"axis": 2})
+
+    def test_Softmax_3(self) -> None:
+        self._test_op_upgrade("Softmax", 1, attrs={"axis": -1})
+
+    def test_Softmax_4(self) -> None:
+        self._test_op_upgrade("Softmax", 1, attrs={"axis": -2})
+
+    def test_Softmax_5(self) -> None:
+        self._test_op_upgrade("Softmax", 1, attrs={"axis": -3})
+
+    def test_Softplus(self) -> None:
+        self._test_op_upgrade("Softplus", 1)
+
+    def test_Softsign(self) -> None:
+        self._test_op_upgrade("Softsign", 1)
+
+    def test_SoftmaxCrossEntropyLoss(self) -> None:
+        self._test_op_upgrade(
+            "SoftmaxCrossEntropyLoss",
+            12,
+            [[3, 4, 5, 6], [3, 6]],
+            [[]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+        )
+
+    def test_SpaceToDepth(self) -> None:
+        self._test_op_upgrade(
+            "SpaceToDepth", 1, [[1, 3, 8, 8]], [[1, 12, 4, 4]], attrs={"blocksize": 2}
+        )
+
+    def test_Split(self) -> None:
+        # 1->2 adapter is missing
+        self._test_op_upgrade(
+            "Split",
+            2,
+            [[3, 4, 7]],
+            [[3, 4, 2], [3, 4, 1], [3, 4, 4]],
+            attrs={"axis": 2, "split": [2, 1, 4]},
+        )
+
+    def test_Sqrt(self) -> None:
+        self._test_op_upgrade("Sqrt", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Squeeze(self) -> None:
+        self._test_op_upgrade("Squeeze", 1, [[2, 1, 3, 4, 1]], [[2, 3, 4]])
+
+    def test_StringNormalizer(self) -> None:
+        self._test_op_upgrade(
+            "StringNormalizer",
+            10,
+            [[1, 3]],
+            [[1, 3]],
+            [TensorProto.STRING],
+            [TensorProto.STRING],
+            attrs={"case_change_action": "LOWER"},
+        )
+
+    def test_Sub(self) -> None:
+        self._test_op_upgrade(
+            "Sub",
+            1,
+            [[2, 3, 4], [2, 3, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_Sum(self) -> None:
+        self._test_op_upgrade(
+            "Sum",
+            1,
+            [[2, 3, 4], [2, 3, 4]],
+            [[2, 3, 4]],
+            attrs={"consumed_inputs": [0]},
+        )
+
+    def test_Swish(self) -> None:
+        self._test_op_upgrade("Swish", 24, attrs={"alpha": 0.2})
+
+    def test_Tanh(self) -> None:
+        self._test_op_upgrade("Tanh", 1, attrs={"consumed_inputs": [0]})
+
+    def test_Tan(self) -> None:
+        self._test_op_upgrade("Tan", 7)
+
+    def test_TfIdfVectorizer(self) -> None:
+        self._test_op_upgrade(
+            "TfIdfVectorizer",
+            9,
+            [[3]],
+            [[5]],
+            attrs={
+                "max_gram_length": 3,
+                "max_skip_count": 1,
+                "min_gram_length": 2,
+                "mode": "TFIDF",
+                "ngram_counts": [0, 20],
+                "ngram_indexes": [3, 4],
+            },
+        )
+
+    def test_ThresholdedRelu(self) -> None:
+        self._test_op_upgrade("ThresholdedRelu", 10)
+
+    def test_Tile(self) -> None:
+        # 5->6 adapter is missing
+        repeats = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[3], vals=np.array([1, 2, 3])
+        )
+        self._test_op_upgrade(
+            "Tile",
+            6,
+            [[3, 4, 5], [3]],
+            [[3, 8, 15]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+            initializer=[repeats],
+        )
+
+    def test_TopK(self) -> None:
+        self._test_op_upgrade(
+            "TopK",
+            1,
+            [[3, 4, 5]],
+            [[3, 4, 2], [3, 4, 2]],
+            output_types=[TensorProto.FLOAT, TensorProto.INT64],
+            attrs={"k": 2},
+        )
+
+    def test_Transpose(self) -> None:
+        self._test_op_upgrade(
+            "Transpose",
+            1,
+            [[1, 2, 5, 3, 7]],
+            [[1, 7, 5, 2, 3]],
+            attrs={"perm": [0, 4, 2, 1, 3]},
+        )
+
+    def test_Trilu(self) -> None:
+        self._test_op_upgrade("Trilu", 14)
+
+    def test_Unique_1(self) -> None:
+        self._test_op_upgrade("Unique", 11, [[3, 4, 5]], [[None]])
+
+    def test_Unique_2(self) -> None:
+        self._test_op_upgrade(
+            "Unique", 11, [[3, 4, 5]], [[3, None, 5]], attrs={"axis": 1}
+        )
+
+    def test_Unsqueeze(self) -> None:
+        self._test_op_upgrade(
+            "Unsqueeze", 1, [[3, 4, 5]], [[3, 4, 1, 5]], attrs={"axes": [2]}
+        )
+
+    def test_Upsample(self) -> None:
+        self._test_op_upgrade(
+            "Upsample",
+            1,
+            [[1, 3, 4, 5]],
+            [[1, 3, 6, 10]],
+            attrs={"width_scale": 2.0, "height_scale": 1.5},
+        )
+
+    def test_Where(self) -> None:
+        self._test_op_upgrade(
+            "Where",
+            9,
+            [[2, 3], [2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.BOOL, TensorProto.FLOAT, TensorProto.FLOAT],
+        )
+
+    def test_Xor(self) -> None:
+        # 6->7 adapter is missing
+        self._test_op_upgrade(
+            "Xor",
+            7,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.BOOL, TensorProto.BOOL],
+            [TensorProto.BOOL],
+        )
+
+    def test_CastLike(self) -> None:
+        self._test_op_upgrade(
+            "CastLike",
+            15,
+            [[2, 3, 4], [2, 1, 4]],
+            [[2, 3, 4]],
+            input_types=[TensorProto.FLOAT, TensorProto.FLOAT16],
+            output_types=[TensorProto.FLOAT16],
+        )
+
+    def test_LayerNormalization(self) -> None:
+        self._test_op_upgrade(
+            "LayerNormalization",
+            17,
+            [[2, 3, 4, 5], [4, 5], [4, 5]],
+            [[2, 3, 4, 5]],
+            input_types=[TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.FLOAT],
+            output_types=[TensorProto.FLOAT],
+            attrs={"axis": 2},
+        )
+
+    def _test_window_function(self, window_function_name: str) -> None:
+        size = helper.make_tensor("a", TensorProto.INT64, dims=[], vals=np.array([10]))
+        self._test_op_upgrade(
+            window_function_name,
+            17,
+            [[]],
+            [[10]],
+            [TensorProto.INT64],
+            initializer=[size],
+        )
+
+    def test_BlackmanWindow(self) -> None:
+        self._test_window_function("BlackmanWindow")
+
+    def test_HannWindow(self) -> None:
+        self._test_window_function("HannWindow")
+
+    def test_HammingWindow(self) -> None:
+        self._test_window_function("HammingWindow")
+
+    def test_DFT(self) -> None:
+        self._test_op_upgrade("DFT", 17, [[2, 16, 1], []], [[2, 16, 2]])
+        self._test_op_upgrade("DFT", 17, [[2, 16, 2], []], [[2, 16, 2]])
+        self._test_op_upgrade(
+            "DFT", 17, [[2, 16, 1], []], [[2, 9, 2]], attrs={"onesided": 1}
+        )
+        self._test_op_upgrade(
+            "DFT", 17, [[2, 16, 2], []], [[2, 9, 2]], attrs={"onesided": 1}
+        )
+        self._test_op_upgrade(
+            "DFT", 17, [[2, 16, 1], []], [[2, 16, 2]], attrs={"inverse": 1}
+        )
+        self._test_op_upgrade(
+            "DFT", 17, [[2, 16, 2], []], [[2, 16, 2]], attrs={"inverse": 1}
+        )
+        self._test_op_upgrade(
+            "DFT", 17, [[2, 16, 2], []], [[2, 16, 2]], attrs={"inverse": 1, "axis": 0}
+        )
+
+    def _test_short_time_fourier_transform(self, operator_name: str) -> None:
+        # Real
+        signal = helper.make_tensor(
+            "a",
+            TensorProto.FLOAT,
+            dims=[2, 64],
+            vals=np.random.rand(2, 64).astype(np.float32),
+        )
+        frame_step = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[1], vals=np.array([8])
+        )
+        window = helper.make_tensor(
+            "c", TensorProto.FLOAT, dims=[16], vals=np.ones(16).astype(np.float32)
+        )
+        self._test_op_upgrade(
+            operator_name,
+            17,
+            [[2, 64], [1], [16]],
+            [[2, 7, 16, 2]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            initializer=[signal, frame_step, window],
+        )
+
+        # Real Onesided
+        signal = helper.make_tensor(
+            "a",
+            TensorProto.FLOAT,
+            dims=[2, 64],
+            vals=np.random.rand(2, 64).astype(np.float32),
+        )
+        frame_step = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[1], vals=np.array([8])
+        )
+        window = helper.make_tensor(
+            "c", TensorProto.FLOAT, dims=[16], vals=np.ones(16).astype(np.float32)
+        )
+        self._test_op_upgrade(
+            operator_name,
+            17,
+            [[2, 64], [1], [16]],
+            [[2, 7, 9, 2]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            attrs={"onesided": 1},
+            initializer=[signal, frame_step, window],
+        )
+
+        # Complex
+        signal = helper.make_tensor(
+            "a",
+            TensorProto.FLOAT,
+            dims=[2, 64, 2],
+            vals=np.random.rand(2, 64, 2).astype(np.float32),
+        )
+        frame_step = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[1], vals=np.array([8])
+        )
+        window = helper.make_tensor(
+            "c", TensorProto.FLOAT, dims=[16], vals=np.ones(16).astype(np.float32)
+        )
+        self._test_op_upgrade(
+            operator_name,
+            17,
+            [[2, 64, 2], [1], [16]],
+            [[2, 7, 16, 2]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            initializer=[signal, frame_step, window],
+        )
+
+        # Complex Onesided
+        signal = helper.make_tensor(
+            "a",
+            TensorProto.FLOAT,
+            dims=[2, 64, 2],
+            vals=np.random.rand(2, 64, 2).astype(np.float32),
+        )
+        frame_step = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[1], vals=np.array([8])
+        )
+        window = helper.make_tensor(
+            "c", TensorProto.FLOAT, dims=[16], vals=np.ones(16).astype(np.float32)
+        )
+        frame_length = helper.make_tensor(
+            "e", TensorProto.INT64, dims=[1], vals=np.array([16])
+        )
+        self._test_op_upgrade(
+            operator_name,
+            17,
+            [[2, 64, 2], [1], [16]],
+            [[2, 7, 9, 2]],
+            [
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.INT64,
+            ],
+            attrs={"onesided": 1},
+            initializer=[signal, frame_step, window, frame_length],
+        )
+
+    def test_STFT(self) -> None:
+        self._test_short_time_fourier_transform("STFT")
+
+    def test_MelWeightMatrix(self) -> None:
+        num_mel_bins = helper.make_tensor(
+            "a", TensorProto.INT64, dims=[], vals=np.array([10])
+        )
+        dft_length = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[], vals=np.array([64])
+        )
+        sample_rate = helper.make_tensor(
+            "c", TensorProto.INT64, dims=[], vals=np.array([0])
+        )
+        lower_edge_hertz = helper.make_tensor(
+            "d", TensorProto.FLOAT, dims=[], vals=np.array([0])
+        )
+        upper_edge_hertz = helper.make_tensor(
+            "e", TensorProto.FLOAT, dims=[], vals=np.array([1])
+        )
+
+        self._test_op_upgrade(
+            "MelWeightMatrix",
+            17,
+            [[], [], [], [], []],
+            [[33, 10]],
+            [
+                TensorProto.INT64,
+                TensorProto.INT64,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+            ],
+            initializer=[
+                num_mel_bins,
+                dft_length,
+                sample_rate,
+                lower_edge_hertz,
+                upper_edge_hertz,
+            ],
+        )
+
+        num_mel_bins = helper.make_tensor(
+            "a", TensorProto.INT64, dims=[], vals=np.array([20])
+        )
+        dft_length = helper.make_tensor(
+            "b", TensorProto.INT64, dims=[], vals=np.array([31])
+        )
+        sample_rate = helper.make_tensor(
+            "c", TensorProto.INT64, dims=[], vals=np.array([0])
+        )
+        lower_edge_hertz = helper.make_tensor(
+            "d", TensorProto.FLOAT, dims=[], vals=np.array([0])
+        )
+        upper_edge_hertz = helper.make_tensor(
+            "e", TensorProto.FLOAT, dims=[], vals=np.array([1])
+        )
+
+        self._test_op_upgrade(
+            "MelWeightMatrix",
+            17,
+            [[], [], [], [], []],
+            [[16, 20]],
+            [
+                TensorProto.INT64,
+                TensorProto.INT64,
+                TensorProto.INT64,
+                TensorProto.FLOAT,
+                TensorProto.FLOAT,
+            ],
+            initializer=[
+                num_mel_bins,
+                dft_length,
+                sample_rate,
+                lower_edge_hertz,
+                upper_edge_hertz,
+            ],
+        )
+
+    def test_CenterCropPad(self) -> None:
+        input_ = helper.make_tensor(
+            "input",
+            TensorProto.FLOAT,
+            dims=[2, 4],
+            vals=np.array([1, 2, 3, 4, 5, 6, 7, 8]),
+        )
+        shape = helper.make_tensor(
+            "shape", TensorProto.INT64, dims=[2], vals=np.array([3, 3])
+        )
+        self._test_op_upgrade(
+            "CenterCropPad",
+            18,
+            [[], []],
+            [[3, 3]],
+            [TensorProto.FLOAT, TensorProto.INT64],
+            initializer=[input_, shape],
+        )
+
+    def test_BitwiseNot(self) -> None:
+        self._test_op_upgrade(
+            "BitwiseNot",
+            18,
+            [[2, 3]],
+            [[2, 3]],
+            [TensorProto.INT32],
+            [TensorProto.INT32],
+        )
+
+    def test_BitwiseAnd(self) -> None:
+        self._test_op_upgrade(
+            "BitwiseAnd",
+            18,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.INT16, TensorProto.INT16],
+            [TensorProto.INT16],
+        )
+
+    def test_BitwiseOr(self) -> None:
+        self._test_op_upgrade(
+            "BitwiseOr",
+            18,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.INT16, TensorProto.INT16],
+            [TensorProto.INT16],
+        )
+
+    def test_BitwiseXor(self) -> None:
+        self._test_op_upgrade(
+            "BitwiseXor",
+            18,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+            [TensorProto.INT16, TensorProto.INT16],
+            [TensorProto.INT16],
+        )
+
+    def test_GroupNormalization(self) -> None:
+        self._test_op_upgrade(
+            "GroupNormalization",
+            21,
+            [[3, 4, 2, 2], [4], [4]],
+            [[3, 4, 2, 2]],
+            attrs={"epsilon": 1e-5, "num_groups": 2},
+        )
+
+    def test_StringConcat(self) -> None:
+        self._test_op_upgrade(
+            "StringConcat",
+            20,
+            [[2, 3], [2, 3]],
+            [[2, 3]],
+        )
+
+    def test_RegexFullMatch(self) -> None:
+        self._test_op_upgrade(
+            "RegexFullMatch",
+            20,
+            [[2, 3]],
+            [[2, 3]],
+            [TensorProto.STRING],
+            [TensorProto.BOOL],
+        )
+
+    def test_TensorScatter(self) -> None:
+        self._test_op_upgrade(
+            "TensorScatter",
+            24,
+            [
+                [2, 3, 4, 5],
+                [2, 3, 2, 5],
+                [
+                    2,
+                ],
+            ],
+            [[2, 3, 4, 5]],
+            [TensorProto.FLOAT, TensorProto.FLOAT, TensorProto.INT64],
+            [TensorProto.FLOAT],
+        )
+
+    def test_ops_tested(self) -> None:
+        # NOTE: This test is order dependent and needs to run last in this class
+        all_schemas = onnx.defs.get_all_schemas()
+        all_op_names = {schema.name for schema in all_schemas if schema.domain == ""}
+        excluded_ops = {
+            # Sequence-based and Optional-based ops disabled because
+            # the version converter doesn't play nicely with sequences
+            "ConcatFromSequence",
+            "SequenceAt",
+            "SequenceConstruct",
+            "SequenceEmpty",
+            "SequenceErase",
+            "SequenceInsert",
+            "SequenceLength",
+            "SequenceMap",
+            "SplitToSequence",
+            "Optional",
+            "OptionalGetElement",
+            "OptionalHasElement",
+            "StringSplit",
+        }
+        expected_tested_ops = all_op_names - excluded_ops
+
+        untested_ops = expected_tested_ops - set(self.tested_ops)
+        self.assertEqual(untested_ops, set())
+
+
+if __name__ == "__main__":
+    unittest.main()

pythonProject/.venv/Lib/site-packages/onnxscript/converter.py

@@ -0,0 +1,1462 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+from __future__ import annotations
+
+import ast
+import logging
+from typing import (
+    TYPE_CHECKING,
+    Any,
+    Dict,
+    List,
+    NoReturn,
+    Optional,
+    Sequence,
+    Tuple,
+    Union,
+)
+
+import onnx
+
+import onnxscript
+from onnxscript import irbuilder, onnx_types, sourceinfo, values
+from onnxscript import type_annotation as ta
+from onnxscript._internal import analysis, ast_utils, autocast, param_manipulation
+
+logger = logging.getLogger("onnxscript")
+
+
+# Python-to-IR converter:
+
+
+def not_allowed(construct):
+    return f"{construct}not supported."
+
+
+class TranslationError(Exception):
+    def __init__(self, *args: object) -> None:
+        super().__init__(*args)
+
+
+def warn(msg):
+    logger.warning(msg)
+
+
+def fail(msg) -> NoReturn:
+    raise TranslationError(msg)
+
+
+def fail_if(cond, msg):
+    if cond:
+        raise TranslationError(msg)
+
+
+def ignore(cond, msg):
+    if cond:
+        warn(msg)
+
+
+# map from python operators to ONNX ops
+primop_map = {
+    ast.Add: "Add",
+    ast.And: "And",
+    ast.BitAnd: "And",
+    ast.BitOr: "Or",
+    ast.Div: "Div",
+    ast.Eq: "Equal",
+    ast.Gt: "Greater",
+    ast.GtE: "GreaterOrEqual",
+    ast.Lt: "Less",
+    ast.LtE: "LessOrEqual",
+    ast.MatMult: "MatMul",
+    ast.Mod: "Mod",
+    ast.Mult: "Mul",
+    ast.Not: "Not",
+    ast.NotEq: "NotEqual",
+    ast.Or: "Or",
+    ast.Pow: "Pow",
+    ast.Sub: "Sub",
+    ast.USub: "Neg",
+}
+
+
+class Variable:
+    """Represents an ONNX variable.
+
+    TODO(rama): Consider merging this with IRVar. However, "castable" is specific to this
+    converter.
+    """
+
+    def __init__(self, name: str, castable: bool = False):
+        """Initialize the instance.
+
+        Args:
+           name: Name of the ONNX variable
+           castable: Whether this variable is castable to a desired target type.
+              Used for ONNX variables representing constants created from python values
+              like 0 or 1 or 0.5 which are treated as polymorphic values castable to other
+              types as needed.
+        """
+        self.name = name
+        self.is_castable = castable
+
+    def __str__(self) -> str:
+        return self.name
+
+
+if TYPE_CHECKING:
+    # The type-alias LocalSymValue represents the types of values that local names in a
+    # script-function may be bound to during translation, (ONNX IR values).
+    # TODO(rama): Rationalize this and values.SymbolValue
+
+    LocalSymValue = Union[values.SymbolValue, irbuilder.IRFunction]
+
+    # The type-alias PyValue is used to represent the types of python values that may be used
+    # in an ONNX Script function.
+    # TODO(rama): Flesh out the set of valid types here. These include values such as
+    # 1 (int), 1.0 (float), [2, 4], [1.0], etc. which will be converted to ONNX, for
+    # use as value-parameters or attribute-parameters in an ONNX call (Node).
+
+    PyValue = Any
+
+    # The type-alias SymValue denotes values that an identifier may be bound to during
+    # translation. A local name will be bound to a LocalSymValue, while a global name
+    # will be bound to a PyValue.
+
+    SymValue = Union[LocalSymValue, PyValue]
+
+    # PreferredName is a type-alias used to represent the preferred name used in the generated
+    # ONNX for a value returned by an expression. There is no guarantee that the specified
+    # name will be used exactly. The converter will modify the name (with a suffix),
+    # if necesssary, to ensure that it is unique (to ensure ONNX's SSA requirement).
+
+    PreferredName = str
+
+    # The type-alias OnnxVar indicates variable names used in the generated ONNX.
+    OnnxVarName = str
+
+
+class Converter:
+    """Main class to translate python code into ONNX operators.
+
+    Args:
+        ir_builder: convert AST node into ONNX structures, if None,
+            class :class:`onnxscript.irbuilder.IRBuilder` is used
+
+    The class uses logger `onnxscript`. Logging can be enabled with the following code:
+
+    ::
+
+        import logging
+        logging.basicConfig(level=logging.DEBUG)
+
+    Or if you need to enable only the logger used by this module:
+
+    ::
+
+        import logging
+        logger = logging.getLogger('onnxscript')
+        logger.setLevel(logging.DEBUG)
+        console = logging.StreamHandler()
+        logger.addHandler(console)
+    """
+
+    def __init__(
+        self,
+        ir_builder: Optional[irbuilder.IRBuilder] = None,
+        opset: Optional[values.Opset] = None,
+        global_names: Optional[dict[str, Any]] = None,
+        source: Optional[str] = None,
+        default_opset: Optional[values.Opset] = None,
+    ):
+        self.ir_builder = ir_builder or irbuilder.IRBuilder()
+        self.source = source
+        if global_names is not None:
+            # We make a copy in case function eval modifies it.
+            self.globals = global_names.copy()
+        self.this_module = opset
+        self.default_opset_ = default_opset
+
+        # States initialized by `_init_function_translation`
+        self._outer: List[irbuilder.IRFunction] = []
+        self._current_fn: irbuilder.IRFunction = None
+        self._nextvar: int = 0
+        self._used_vars: set[str] = set()
+        self._locals: List[Dict[str, LocalSymValue]] = [{}]
+
+    @property
+    def default_opset(self) -> values.Opset:
+        if self.default_opset_ is None:
+            raise RuntimeError(
+                "default_opset must be specified in script for functions "
+                "that do not contain any use of an ONNX opset."
+            )
+        return self.default_opset_
+
+    def _set_default_opset(self, opset: values.Opset, node: ast.AST) -> None:
+        if opset.domain != "":
+            return
+        if self.default_opset_ is not None:
+            if (
+                opset.domain != self.default_opset_.domain
+                or opset.version != self.default_opset_.version
+            ):
+                self.fail(
+                    node, f"Two distincts opset were used ({opset} != {self.default_opset_})."
+                )
+        else:
+            self.default_opset_ = opset
+
+    def _find_onnx_opset(self, node: ast.AST) -> Optional[values.Opset]:
+        """Find the (first) ONNX opset used in the function, if any."""
+        # Search for a Call expression of form "op.OpName(...)"
+        if isinstance(node, ast.Call):
+            if isinstance(node.func, ast.Attribute):
+                opset_expr = node.func.value
+                if isinstance(opset_expr, ast.Name):
+                    if opset_expr.id in self.globals:
+                        opset = self.globals[opset_expr.id]
+                        if isinstance(opset, values.Opset) and opset.domain == "":
+                            return opset
+        for child in ast.iter_child_nodes(node):
+            res = self._find_onnx_opset(child)
+            if res is not None:
+                return res
+        return None
+
+    def _init_function_translation(self) -> None:
+        """Initialize self for translating a new (top-level) function."""
+        self._outer = []
+        self._current_fn: Optional[irbuilder.IRFunction] = None
+        self._nextvar = 0
+        self._used_vars = set()
+        self._locals: List[Dict[str, LocalSymValue]] = [{}]
+
+    def _source_of(self, node: ast.AST) -> sourceinfo.SourceInfo:
+        return sourceinfo.SourceInfo(node, self.source, self._current_fn.name)
+
+    def _message(self, node: ast.AST, error_msg: str) -> str:
+        """Constructs an error _message containing source information about an ast node."""
+        return self._source_of(node).msg(error_msg)
+
+    def warn(self, node: ast.AST, error_msg: str) -> None:
+        warn(self._message(node, error_msg))
+
+    def fail(self, node: ast.AST, error_msg: str) -> NoReturn:
+        fail(self._message(node, error_msg))
+
+    # Name resolution and namescopes: This component handles the following aspects:
+    # * Name-scopes are different in Python and the generated ONNX:
+    #   - Control-flow blocks (a loop body or the then-or-else block of an if-stmt)
+    #     form part of the same name-scope in Python, but will be mapped to a nested
+    #     name-scope (as a sub-graph) in ONNX.
+    # * Script-time name-value tracking: Name _lookup during script-time returns
+    #   statically-known information about the value the name will have at runtime.
+    def _enter_scope(self, name: str, parent_node: ast.AST):
+        """Enter a control-flow block (a loop body or if-then-else branch).
+        The block is translated into a nested-scope in ONNX.
+        """
+        self._outer.insert(0, self._current_fn)
+        self._current_fn = self.ir_builder.new_function(name)
+        self._locals.insert(0, {})
+        logger.debug("Converter:_enter_scope:%d:node:%s", len(self._locals), type(parent_node))
+
+    def _exit_scope(self) -> irbuilder.IRFunction:
+        """Exit from a control-flow block (a loop body or if-then-else branch)."""
+        logger.debug("Converter:_exit_scope:%d", len(self._locals))
+        graph = self._current_fn
+        self._current_fn = self._outer.pop(0)
+        self._locals.pop(0)
+        return graph
+
+    def _current_scope(self) -> Dict[str, LocalSymValue]:
+        return self._locals[0]
+
+    def _bind(self, name: str, val: LocalSymValue) -> None:
+        logger.debug("Converter:_bind:%s", name)
+        self._locals[0][name] = val
+
+    def _lookup(
+        self, name: str, info: sourceinfo.SourceInfo, raise_exception: bool = True
+    ) -> SymValue:
+        for scope in self._locals:
+            if name in scope:
+                return scope[name]
+        if name in self.globals:
+            return self.globals[name]
+        if raise_exception:
+            raise ValueError(info.msg(f"Unbound name: {name}."))
+        return None
+
+    def generate_unique_name(self, candidate: str = "tmp") -> str:
+        # TODO(justinchuby): Can we reduce the O complexity of this function?
+        r = candidate
+        while r in self._used_vars:
+            r = f"{candidate}_{self._nextvar}"
+            self._nextvar = self._nextvar + 1
+        self._used_vars.add(r)
+        return r
+
+    def _make_onnx_attr(
+        self, attrname: str, attrval: Any, attrtype: int | None = None
+    ) -> irbuilder.IRAttributeValue:
+        def tensor_name_generator() -> str:
+            """Return name to be used for tensor, if we need to create one."""
+            return self.generate_unique_name(f"attr_{attrname}")
+
+        proto = autocast.pyvalue_to_onnx_attribute(
+            attrname, attrval, tensor_name_generator, attrtype
+        )
+        return self.ir_builder.make_attr(proto)
+
+    def _to_onnx_attr_ref(
+        self, val: values.AttrRef, info: Optional[sourceinfo.SourceInfo]
+    ) -> irbuilder.IRAttributeValue:
+        pytype = val.typeinfo
+        attrtype = ta.pytype_to_attrtype(pytype)
+        attrname = None
+        if attrtype is onnx.AttributeProto.FLOAT:
+            attrname = "value_float"
+        elif attrtype is onnx.AttributeProto.INT:
+            attrname = "value_int"
+        elif attrtype is onnx.AttributeProto.STRING:
+            attrname = "value_string"
+        elif attrtype is onnx.AttributeProto.INTS:
+            attrname = "value_ints"
+        else:
+            msg = f"Unsupported attribute type {pytype!r}."
+            fail(info.msg(msg) if info else msg)
+        return self.ir_builder.make_attr_ref(attrname, val.value, pytype)
+
+    def _to_onnx_var(
+        self,
+        val: values.SymbolValue | PyValue,
+        target: Optional[PreferredName] = None,
+        info: Optional[sourceinfo.SourceInfo] = None,
+    ) -> Variable:
+        if isinstance(val, values.AttrRef):
+            # promote attribute to value
+            result = self.generate_unique_name(target or "tmp")
+            attr = self._to_onnx_attr_ref(val, info)
+            self.emit([result], values.Op(self.default_opset, "Constant"), [], [attr])
+            if ta.base_type_is_bool(val.typeinfo):
+                # ONNX attributes use an int-encoding for bools, but ONNX tensor types
+                # distinguish between int and bool. So we cast the int tensor to a bool tensor,
+                # to promote a (python) bool attribute to a ONNX bool tensor.
+                result_as_bool = self.generate_unique_name(result + "_as_bool")
+                cast_attr = self._make_onnx_attr("to", onnx_types.BOOL.dtype)
+                self.emit(
+                    [result_as_bool],
+                    values.Op(self.default_opset, "Cast"),
+                    [result],
+                    [cast_attr],
+                )
+                return Variable(result_as_bool, True)
+            return Variable(result, True)
+        if isinstance(val, values.Dynamic):
+            return Variable(val.value)
+        # Assume value is a python-value convertible to a tensor
+        # TODO: check if value is convertible to a TensorProto, so that we can
+        # produce a better error _message otherwise
+        return self._emit_const(val, target or "tmp", info)
+
+    def _py_var_to_onnx_var(self, py_var: str, info: sourceinfo.SourceInfo) -> Variable:
+        return self._to_onnx_var(self._lookup(py_var, info), target=py_var, info=info)
+
+    def emit(
+        self,
+        outputs: Sequence[str],
+        callee: values.Op | str,
+        inputs: Sequence[Optional[str]],
+        attrs: Optional[Sequence[irbuilder.IRAttributeValue]] = None,
+        sub_functions: Optional[dict[str, onnx.FunctionProto]] = None,
+    ):
+        if not isinstance(callee, values.Op):
+            callee = values.Op(self.default_opset, callee)
+        if attrs is None:
+            attrs = []
+        if sub_functions is None:
+            sub_functions = {}
+        self.ir_builder.add_stmt(
+            self._current_fn,
+            outputs,
+            callee,
+            inputs,
+            attrs,
+            sub_functions,
+        )
+
+    def _emit_const(
+        self,
+        pyvalue: PyValue,
+        suggested_name: Optional[PreferredName],
+        info: sourceinfo.SourceInfo,
+    ) -> Variable:
+        if suggested_name is None:
+            if isinstance(pyvalue, int):
+                if pyvalue >= 0:
+                    suggested_name = f"int64_{pyvalue}"
+                else:
+                    suggested_name = f"int64_m{abs(pyvalue)}"
+            elif (
+                isinstance(pyvalue, list) and len(pyvalue) == 1 and isinstance(pyvalue[0], int)
+            ):
+                if pyvalue[0] >= 0:
+                    suggested_name = f"int64_{pyvalue[0]}_1d"
+                else:
+                    suggested_name = f"int64_m{abs(pyvalue[0])}_1d"
+            else:
+                suggested_name = "const"
+        ovar = self.generate_unique_name(suggested_name)
+        try:
+            tensor = autocast.pyvalue_to_onnx_tensor(ovar, pyvalue)
+        except ValueError as e:
+            fail(info.msg(str(e)))
+        attr = self._make_onnx_attr("value", tensor)
+        self.emit([ovar], values.Op(self.default_opset, "Constant"), [], [attr])
+        return Variable(ovar, True)
+
+    def _emit_copy(self, original_var: str, suggested_name: str) -> str:
+        """Emits a copy statement, using the ONNX Identity operator."""
+        new_var = self.generate_unique_name(suggested_name)
+        self.emit([new_var], "Identity", [original_var])
+        return new_var
+
+    def _is_constant_expr(self, node: ast.AST) -> None:
+        if isinstance(node, ast.UnaryOp):
+            return self._is_constant_expr(node.operand)
+        if isinstance(
+            node,
+            (
+                ast.Call,
+                ast.BinOp,
+                ast.UnaryOp,
+                ast.Compare,
+                ast.Attribute,
+                ast.List,
+                ast.Load,
+                ast.Constant,
+            ),
+        ):
+            return all(self._is_constant_expr(c) for c in ast.iter_child_nodes(node))
+        return False
+
+    def _eval_constant_expr(self, expr: ast.AST) -> PyValue:
+        """Evaluates a sub-expression that is assumed to represent a constant value.
+        The expression can refer only to global names (inherited from the scope
+        where the script is evaluated) and cannot refer to local names defined
+        within the script.) Further, these expressions are assumed to be constants.
+        Thus, any subsequent mutation of any state/variables (used in computing
+        this constant value) will potentially lead to unexpected behavior (such
+        as divergence between eager-mode execution and evaluation of the ONNX
+        function.)
+        """
+        # TODO: assert (self._is_constant_expr(expr))
+        # TODO: Refine types
+        locals: dict[Any, Any] = {}
+        expr = ast.Expression(expr, lineno=expr.lineno, col_offset=expr.col_offset)
+        cpl = compile(expr, filename="<ast>", mode="eval")
+        try:
+            return eval(cpl, self.globals, locals)  # pylint: disable=eval-used
+        except NameError as e:
+            raise NameError(
+                self._message(
+                    expr,
+                    f"Missing names, globals contains {list(self.globals)!r}, "
+                    f"locals {list(locals)!r}.",
+                )
+            ) from e
+
+    def _translate_attr(
+        self,
+        attr_name: str,
+        expr: ast.AST,
+        attr_meta: Optional[onnx.defs.OpSchema.Attribute] = None,
+    ) -> Optional[irbuilder.IRAttributeValue]:
+        """Translate an attribute-value specification of the form `attr_name=<expr>`
+        in a call to an op. expr is an AST. The following cases are supported:
+        * Expr evaluates to a script-time constant (a python-value) that can be mapped
+        into an ONNX attribute value, or
+        * Expr evaluates to None, in which case None is returned, or
+        * Expr must be an attribute-reference, that is a name representing an
+        attribute-parameter of a containing function.
+        """
+
+        if isinstance(expr, ast.Name):
+            val = self._lookup(expr.id, self._source_of(expr))
+            if isinstance(val, values.AttrRef):
+                attr_ref = self.ir_builder.make_attr_ref(attr_name, val.value, val.typeinfo)
+                if attr_meta is not None and (attr_ref.type != attr_meta.type):
+                    self.fail(
+                        expr,
+                        f"Attribute type '{attr_ref.type}' does not match expected type '{attr_meta.type}'",
+                    )
+                return attr_ref
+            if isinstance(val, irbuilder.IRFunction):
+                # Check that outer-scope variables referenced by function have same value
+                # at function-definition site and use-as-attribute site, to avoid errors.
+                for pyvar, previous in val.outer_scope_variables:
+                    current = self._lookup(pyvar, self._source_of(expr))
+                    if current.value != previous.value:
+                        self.fail(
+                            expr,
+                            f"Outer scope variable '{pyvar}' referenced by function "
+                            f"'{expr.id!r}' modified.",
+                        )
+
+                # Create GraphProto attribute
+                val = val.to_graph_proto()
+        else:
+            val = self._eval_constant_expr(expr)
+
+        # In ONNX, there is no way to explicitly specify a None value for an attribute.
+        # Instead, the attribute must be omitted from the attribute list.
+        # Hence, we do not create an attribute-proto if the value is None.
+        # The caller is responsible for omitting such attribute-values from the list of attributes
+        # in a NodeProto.
+        if val is None:
+            if attr_meta and attr_meta.required:
+                self.fail(expr, f"Attribute '{attr_name}' is required.")
+            return None
+        attr_type = int(attr_meta.type) if attr_meta else None
+        attr = self._make_onnx_attr(attr_name, val, attrtype=attr_type)
+        if attr_meta and (attr.type != attr_meta.type):
+            self.fail(
+                expr,
+                f"Attribute type '{attr.type}' does not match expected type '{attr_meta.type}'",
+            )
+        return attr
+
+    def _translate_docstring(self, node: ast.Expr) -> None:
+        if hasattr(node.value, "value"):
+            # python 3.8+
+            return self.ir_builder.add_docstring(self._current_fn, node.value.value)
+        raise TypeError(
+            f"Unexpected type {type(node)!r} for node. Unsupoorted version of python."
+        )
+
+    def _translate_expr(
+        self, node: ast.AST, target: Optional[PreferredName] = None
+    ) -> Variable:
+        """Expression-translation generates "IR statements/nodes" that compute the value of
+        the expression into a target-variable, and returns the variable that is
+        assigned this value.
+        """
+        if isinstance(node, ast.Call):
+            r = self._translate_call_expr(node)
+        elif isinstance(node, (ast.BinOp, ast.BitAnd, ast.BitOr)):
+            r = self._translate_binary_op_expr(node)
+        elif isinstance(node, ast.UnaryOp):
+            r = self._translate_unary_op_expr(node)
+        elif isinstance(node, ast.Compare):
+            r = self._translate_compare_expr(node)
+        elif isinstance(node, ast.Name):
+            r = self._translate_name_expr(node)
+        elif isinstance(node, ast.Subscript):
+            r = self._translate_subscript_expr(node, target)
+        elif self._is_constant_expr(node):
+            r = self._emit_const(self._eval_constant_expr(node), target, self._source_of(node))
+        else:
+            raise ValueError(
+                self._message(node, f"Unsupported expression type {type(node)!r}.")
+            )
+        if isinstance(r, Variable):
+            return r
+        callee, args, attrs = r
+        target = "tmp" if target is None else target
+        assert isinstance(target, str)
+        result = self.generate_unique_name(target)
+        self.emit([result], callee, args, attrs)
+        return Variable(result)
+
+    def _translate_opt_expr(self, node: ast.expr) -> Optional[Variable]:
+        """Translation of an expression where "None" is permitted (eg., for an optional argument).
+        None is represented as a Constant in Python 3.9+.
+        """
+        if isinstance(node, ast.Constant) and (node.value is None):
+            return None
+        return self._translate_expr(node)
+
+    def _translate_subscript_expr(
+        self, node: ast.Subscript, target: Optional[PreferredName]
+    ) -> Variable:
+        """List of supported syntaxes is below.
+        `A` is a tensor or an expression equivalent to a tensor.
+
+        ::
+
+            A[:, 1]
+            A[:2, 0]
+            A[:2, :1]
+            A[2:0:-1]
+            A[1:]
+            A[:2]
+            A[1:-1]
+            A[1:2]
+            A[-1]
+            A[0]
+            A[:0:-1]
+
+        *i* is a tensor holding one integer.
+
+        ::
+
+            A[i]
+            A[i+1:i+2]
+
+        Fully supported for python 3.9+.
+
+        ::
+
+            A[i:i+j, k]
+
+        Not supported:
+
+        ::
+
+            A[::-1]
+        """
+        var = self._translate_expr(node.value)
+        var_name = var.name
+        if target is None:
+            target = f"{var_name}_subscripted"
+        target = self.generate_unique_name(target)
+        indices = ast_utils.normalize_subscript_expr(node)
+        info = self._source_of(node.slice)
+
+        # Create cached int constants:
+        # TODO: Do this at a graph-scope level.
+        cached_int_consts = {}
+
+        def const_1d(value, name: Optional[str] = None):
+            nonlocal cached_int_consts
+            if value not in cached_int_consts:
+                cached_int_consts[value] = self._emit_const([value], name, info)
+            return cached_int_consts[value]
+
+        def one_1d():
+            return const_1d(1)
+
+        # Max/min 64-bit int values are used to represent default values for start/stop in Slice.
+        maxint = (1 << 63) - 1
+        minint = -(1 << 63)
+
+        def translate_slice_component(
+            node_arg, default_value: Optional[int] = None
+        ) -> tuple[str, Optional[int]]:
+            """Translate optional start/stop/step component of a Slice expression."""
+            if node_arg is None:
+                if default_value is None:
+                    # TODO: Emit "Where(step > 0, pos_default, neg_default)"
+                    raise RuntimeError(
+                        "Default start/stop not supported when step direction is unknown."
+                    )
+                return const_1d(default_value), default_value
+
+            if self._is_constant_expr(node_arg):
+                cst = self._eval_constant_expr(node_arg)
+                if isinstance(cst, int):
+                    return const_1d(cst), cst
+                else:
+                    raise RuntimeError(f"Slice component type must be int, not {type(cst)}")
+            else:
+                name = self._translate_expr(node_arg).name
+                reshaped = self.generate_unique_name(f"{name}_reshaped")
+                self.emit(
+                    [reshaped],
+                    values.Op(self.default_opset, "Reshape"),
+                    [name, one_1d().name],
+                    [],
+                )
+                return reshaped, None
+
+        def translate_slice(slice_expr: ast.Slice) -> tuple[str, str, str]:
+            """Translate slice-expression of the form from:to:step."""
+            step_name, step = translate_slice_component(slice_expr.step, 1)
+            if step is None:
+                # Step direction unknown.
+                # TODO: Handle default-values using runtime check on sign of step.
+                lower_name, _ = translate_slice_component(slice_expr.lower, None)
+                upper_name, _ = translate_slice_component(slice_expr.upper, None)
+            elif step > 0:
+                lower_name, _ = translate_slice_component(slice_expr.lower, 0)
+                upper_name, _ = translate_slice_component(slice_expr.upper, maxint)
+            else:
+                lower_name, _ = translate_slice_component(slice_expr.lower, maxint)
+                upper_name, _ = translate_slice_component(slice_expr.upper, minint)
+            return (lower_name, upper_name, step_name)
+
+        # An input like X[2] is translated into a Gather op.
+        # An input like X[1:5:2] is translated into a Slice op.
+        # An input like X[2, 3] is translated into a Slice + Squeeze (instead of two Gathers),
+        #   as an optimization.
+        # An input like X[I, J] is translated into two Gathers (which is correct whatever the
+        #   rank of I and J)
+        # To replace multiple Gathers by the Slice we need to know that the index-values
+        # are scalars.
+
+        # As the first step, we partition the index elements into four kinds: Slice (eg., 1:5:2),
+        # known-to-be-scalar (eg., 2), other-tensor (eg., I), skip/no-op (that is, just ":")
+        sliced_indices: List[Tuple[int, ast.expr]] = []
+        scalar_indices: List[Tuple[int, ast.expr]] = []
+        non_scalar_indices: List[Tuple[int, ast.expr]] = []
+        for axis, elt in enumerate(indices):
+            if isinstance(elt, ast.Slice):
+                # Add to sliced_indices, unless it is "::", which is a no-op.
+                if not (elt.lower is None and elt.upper is None and elt.step is None):
+                    sliced_indices.append((axis, elt))
+            elif self._is_constant_expr(elt) and isinstance(
+                self._eval_constant_expr(elt), int
+            ):
+                scalar_indices.append((axis, elt))
+            else:
+                non_scalar_indices.append((axis, elt))
+        if not (sliced_indices or scalar_indices or non_scalar_indices):
+            # Edge case: no index specified. Eg. A[:, :]
+            self.emit([target], "Identity", [var_name])
+            return Variable(target)
+        if sliced_indices or len(scalar_indices) > 1:
+            # We emit a Slice operation if we have any indices like 1:5:2 or if the number of
+            # scalar indices (like 2) is more than 1.
+            starts = []
+            ends = []
+            axes = []
+            steps = []
+            squeezed_axes = []
+            for axis, expr in scalar_indices:
+                # Treat a scalar index i as slice "i:i+1:1", but squeeze the axis finally.
+                # TODO: handle negative i
+                index = self._eval_constant_expr(expr)
+                squeezed_axes.append(axis)
+                kwargs = dict(
+                    lineno=getattr(expr, "lineno", node.lineno),
+                    col_offset=getattr(expr, "col_offset", node.col_offset),
+                )
+                element = ast.Slice(
+                    ast.Constant(index, **kwargs),
+                    ast.Constant(index + 1, **kwargs),
+                    ast.Constant(1, **kwargs),
+                )
+                sliced_indices.append((axis, element))
+            scalar_indices = []
+            for axis, element in sliced_indices:
+                axis_var = const_1d(axis)
+                inputs = translate_slice(element)
+                starts.append(inputs[0])
+                ends.append(inputs[1])
+                axes.append(axis_var.name)
+                steps.append(inputs[2])
+
+            if len(starts) > 1:
+                axis_0_attr = self._make_onnx_attr("axis", 0)
+                start_name = self.generate_unique_name(f"{var_name}_start")
+                self.emit([start_name], "Concat", starts, [axis_0_attr])
+
+                end_name = self.generate_unique_name(f"{var_name}_end")
+                self.emit([end_name], "Concat", ends, [axis_0_attr])
+
+                axes_name = self.generate_unique_name(f"{var_name}_axis")
+                self.emit([axes_name], "Concat", axes, [axis_0_attr])
+
+                steps_name = self.generate_unique_name(f"{var_name}_step")
+                self.emit([steps_name], "Concat", steps, [axis_0_attr])
+            else:
+                start_name = starts[0]
+                end_name = ends[0]
+                axes_name = axes[0]
+                steps_name = steps[0]
+
+            if squeezed_axes:
+                sliced_name = self.generate_unique_name(f"{var_name}_sliced")
+                self.emit(
+                    [sliced_name],
+                    "Slice",
+                    [var_name, start_name, end_name, axes_name, steps_name],
+                )
+                squeezed_axes = self._emit_const(squeezed_axes, "squeezed_axes", info)
+
+                if non_scalar_indices:  # use temporary to store result of squeeze
+                    result = self.generate_unique_name(f"{var_name}_squeezed")
+                else:  # store squeezed result in final target
+                    result = target
+
+                self.emit([result], "Squeeze", [sliced_name, squeezed_axes])
+            else:
+                if non_scalar_indices:  # use temporary to store result of Slice
+                    result = self.generate_unique_name(f"{var_name}_sliced")
+                else:  # store result of Slice in final target
+                    result = target
+                slice_inputs = [var_name, start_name, end_name, axes_name, steps_name]
+                self.emit([result], "Slice", slice_inputs)
+        else:
+            result = var_name
+        non_scalar_indices.extend(scalar_indices)
+        if non_scalar_indices:
+            last_axis, _ = non_scalar_indices[-1]
+        else:
+            # TODO(justinchuby): Clarify what last_axis should be when non_scalar_indices is False
+            last_axis = None
+        for axis, index_expr in non_scalar_indices:
+            index_value = self._translate_expr(index_expr)
+            axis_attr = self._make_onnx_attr("axis", axis)
+            # use Gather to perform indexing
+            # Assign gathered value to either temporary or final target
+            if axis != last_axis:  # use temporary to store result of Gather
+                gathered = self.generate_unique_name(f"{var_name}_axis_{axis}")
+            else:  # store result of Gather in final target
+                gathered = target
+            self.emit([gathered], "Gather", [str(result), index_value], [axis_attr])
+            result = gathered
+
+        return Variable(result)
+
+    def _translate_call_expr(self, node: ast.Call):
+        """Translates a call-expression."""
+        callee = self._translate_callee_expr(node.func)
+        param_schemas = callee.param_schemas()
+        # If the callee's schema is available, we use it to determine the inputs and attributes.
+        # Otherwise, we map named arguments to attributes and positional arguments to inputs.
+        if param_schemas:
+            kwargs = {x.arg: x.value for x in node.keywords}
+            args, attrs = param_manipulation.separate_input_attributes_from_arguments(
+                param_schemas, node.args, kwargs, fill_defaults=False
+            )
+            args = [self._translate_opt_expr(x) for x in args]
+            attrs = [
+                self._translate_attr(x, y, callee.op_schema.attributes[x])
+                for x, y in attrs.items()
+            ]
+        else:
+            args = [self._translate_opt_expr(x) for x in node.args]
+            attrs = [self._translate_attr(x.arg, x.value) for x in node.keywords]
+        args = autocast.static_cast_inputs(self, callee.op_schema, args)
+
+        # In ONNX, there is no way to explicitly specify a None value for an attribute.
+        # Instead, the attribute must be omitted from the attribute list.
+        # Hence, we do not create an attribute-proto if the value is None.
+        attrs = [attr for attr in attrs if attr is not None]
+        return callee, args, attrs
+
+    def _cast_like_binary_expression(self, op, left, right):
+        schema = op.op_schema
+        return autocast.static_cast_inputs(self, schema, (left, right))
+
+    def _translate_binary_op_expr(self, node: ast.BinOp):
+        op = type(node.op)
+        if op not in primop_map:
+            raise ValueError(self._message(node, f"Unsupported operator {op!r}."))
+
+        attr = []
+        if isinstance(node.op, ast.Mod) and self._is_constant_expr(node.right):
+            # specific case X % f where f is a float.
+            # attribute fmod=1 is added in that case.
+            cst = self._eval_constant_expr(node.right)
+            if isinstance(cst, float):
+                attr = [self._make_onnx_attr("fmod", 1)]
+
+        op = values.Op(self.default_opset, primop_map[op])
+        left, right = self._cast_like_binary_expression(
+            op, self._translate_expr(node.left), self._translate_expr(node.right)
+        )
+        return op, [left, right], attr
+
+    def _translate_unary_op_expr(self, node):
+        op = type(node.op)
+        if op not in primop_map:
+            raise ValueError(self._message(node, self).msg(f"Unsupported operator {op!r}."))
+        if self._is_constant_expr(node.operand):
+            # This function changed the constant node.operand
+            # and returns it. The function calling this one
+            # should intercept this call and replace node
+            # by node.operand.
+            # This mechanism does not handle somthing like `(-(-5))`.
+            if hasattr(node.operand, "value"):
+                # python 3.8+
+                val = node.operand.value
+            else:
+                raise TypeError(
+                    f"Unable to guess constant value from type {type(node.operand)!r} "
+                    f"and attributes {dir(node.operand)!r}."
+                )
+            if op == ast.USub:
+                cst = ast.Constant(-val, lineno=node.lineno, col_offset=node.col_offset)
+                return self._translate_expr(cst)
+            if op == ast.UAdd:
+                return self._translate_expr(node.operand)
+        opname = primop_map[op]
+        operand = self._translate_expr(node.operand)
+        return values.Op(self.default_opset, opname), [operand], []
+
+    def _translate_compare_expr(self, node):
+        # TODO: handle multiple comparisons in one expression
+        assert len(node.ops) == 1
+        assert len(node.comparators) == 1
+        op = type(node.ops[0])
+        if op not in primop_map:
+            raise ValueError(self._message(node, f"Unsupported operator {op!r}."))
+        opname = primop_map[op]
+        left = self._translate_expr(node.left)
+        right = self._translate_expr(node.comparators[0])
+
+        # NotEqual is not a standard ONNX op, and needs to be translated into
+        # an Equal op/node followed by a Not op/node.
+        op = values.Op(self.default_opset, opname if opname != "NotEqual" else "Equal")
+        left, right = self._cast_like_binary_expression(op, left, right)
+        if opname == "NotEqual":
+            tmp = self.generate_unique_name()
+            self.emit([tmp], op, [left, right])
+            not_op = values.Op(self.default_opset, "Not")
+            return not_op, [tmp], []
+
+        return op, [left, right], []
+
+    def _translate_name_expr(self, node: ast.Name) -> Variable:
+        return self._py_var_to_onnx_var(node.id, self._source_of(node))
+
+    # pylint: disable=inconsistent-return-statements
+    def _translate_opset_expr(self, node: ast.Attribute) -> values.Opset:
+        """Return an Opset"""
+        if isinstance(node, ast.Name):
+            val = self._lookup(node.id, self._source_of(node), raise_exception=False)
+            if isinstance(val, values.Opset):
+                return val
+            self.fail(node, f"'{node.id}' is not an instance of type Opset but {type(val)}.")
+        elif isinstance(node, ast.Attribute):
+            self.fail(node, "Nested module unimplemented.")  # TODO
+        else:
+            self.fail(node, "Invalid opset expression.")
+
+    # pylint: enable=inconsistent-return-statements
+    def _translate_callee_expr(self, node: ast.AST) -> values.Op:  # pylint: disable=R1710
+        """Return an Op"""
+        if isinstance(node, ast.Attribute):
+            module = self._translate_opset_expr(node.value)
+            self._set_default_opset(module, node)
+            opname = node.attr
+            if opname in module:
+                return values.Op(module, node.attr)
+            return values.Op(module, node.attr)
+        if isinstance(node, ast.Name):
+            function_name = node.id
+            found = self._lookup(function_name, self._source_of(node), raise_exception=False)
+            if isinstance(found, onnxscript.OnnxFunction):
+                self._current_fn.add_called_function(found)
+                return found
+            if isinstance(found, values.Op):
+                return found
+            if not found:
+                if function_name not in self.default_opset:
+                    warn(
+                        f"Unknown function name {function_name!r}. "
+                        f"The ONNX graph may not work."
+                    )
+                return values.Op(self.default_opset, function_name)
+        self.fail(node, "Invalid callee")
+
+    def _translate_stmt(self, node: ast.stmt, index_of_stmt=None) -> None:
+        """Statement translation: A single Python statement is mapped into a
+        sequence of IR statements.
+        """
+        if isinstance(node, ast.Assign):
+            return self._translate_assign_stmt(node)
+        if isinstance(node, ast.AnnAssign):
+            return self._translate_assign_stmt(node)
+        if isinstance(node, ast.Return):
+            if index_of_stmt is not None:
+                return self._translate_return_stmt(node)
+            raise ValueError(
+                self._message(
+                    node, "Return statements are not permitted inside control-flow statements."
+                )
+            )
+        if isinstance(node, ast.If):
+            return self._translate_if_stmt(node)
+        if isinstance(node, (ast.For, ast.While)):
+            return self._translate_loop_stmt(node)
+        if ast_utils.is_doc_string(node):
+            if index_of_stmt == 0:
+                return self._translate_docstring(node)
+            return None
+        if isinstance(node, ast.FunctionDef):
+            return self._translate_nested_function_def(node)
+        if ast_utils.is_print_call(node):
+            return None
+        raise ValueError(self._message(node, f"Unsupported statement type '{type(node)!r}'."))
+
+    def _translate_assign_stmt(self, stmt: Union[ast.Assign, ast.AnnAssign]) -> None:
+        def assign(lhs: ast.AST, rhs: ast.AST) -> None:
+            if isinstance(lhs, ast.Name):
+                # Assignments of the form "x = SomeExpression"
+                info = self._source_of(lhs)
+                lhs = lhs.id
+                t = self._translate_expr(rhs, lhs).name
+                if isinstance(stmt, ast.AnnAssign):
+                    typeinfo = self._eval_constant_expr(stmt.annotation)
+                else:
+                    typeinfo = None
+                var = values.Dynamic(t, values.DynamicKind.Intermediate, info, typeinfo)
+                self._bind(lhs, var)
+            elif isinstance(lhs, ast.Tuple):
+                # Assignments of the form "x, y, z = op.SomeOp(...)"
+                if not isinstance(rhs, ast.Call):
+                    self.fail(
+                        rhs,
+                        f"RHS must be a Call expression for unpacking, found: '{type(rhs)!r}'",
+                    )
+                callee, inputs, attrs = self._translate_call_expr(rhs)
+
+                def generate_onnx_name(x: ast.AST):
+                    if not isinstance(x, ast.Name):
+                        self.fail(x, f"LHS must be a Name for unpacking, found: '{type(x)!r}'")
+                    onnx_name = self.generate_unique_name(x.id)
+                    self._bind(
+                        x.id,
+                        values.Dynamic(
+                            onnx_name, values.DynamicKind.Intermediate, self._source_of(x)
+                        ),
+                    )
+                    return onnx_name
+
+                outputs = [generate_onnx_name(x) for x in lhs.elts]
+                self.emit(outputs, callee, inputs, attrs)
+            else:
+                self.fail(lhs, f"Unsupported construct in LHS of assignment: '{type(lhs)!r}'")
+
+        if isinstance(stmt, ast.Assign):
+            targets = stmt.targets
+        else:
+            targets = [stmt.target]
+        if len(targets) != 1:
+            # Assignments of the form "x = y = SomeExpression"
+            self.fail(stmt, "Multi-assignment not supported.")
+        lhs = targets[0]
+        rhs = stmt.value
+        if isinstance(rhs, ast.Tuple):
+            # Assignments of the form "... = Expression1, Expression2"
+            if not isinstance(lhs, ast.Tuple):
+                # Assignments of the form "single_var = Expression1, Expression2".
+                # We do not support tuple-typed variables.
+                self.fail(lhs, f"Left term must be a tuple not '{type(lhs)!r}'.")
+            # Parallel assignments of the form "x, y = Expression1, Expression2"
+            if len(lhs.elts) != len(rhs.elts):
+                self.fail(
+                    stmt, "Expected same number of elements on lhs and rhs of assignments."
+                )
+            for p, r in zip(lhs.elts, rhs.elts):
+                assign(p, r)
+        else:
+            assign(lhs, rhs)
+
+    def _translate_return_stmt(self, stmt: ast.Return) -> None:
+        def check_num_outputs(n):
+            if self.returntype is not None:
+                if n != len(self.returntype):
+                    raise SyntaxError(
+                        self._message(
+                            stmt,
+                            f"Mismatch in number of return values and types. Keyword "
+                            f"'return' cannot be used in a subgraph (test, loop).  "
+                            f"returntype is {self.returntype!r}, num_outputs={n!r}.",
+                        )
+                    )
+
+        def ret(exp, i, suffix):
+            preferred_name = f"return_val{suffix}"
+            return_var = self._translate_expr(exp, preferred_name).name
+            val = self._lookup(return_var, self._source_of(exp), False)
+            if val and val.kind == values.DynamicKind.Input:
+                # In ONNX, a graph-input cannot be an output of the graph.
+                # We need to insert a copy.
+                return_var = self._emit_copy(return_var, preferred_name)
+            for prev_output in self._current_fn.outputs:
+                if prev_output.name == return_var:
+                    # ONNX does not allow duplicate output names.
+                    return_var = self._emit_copy(return_var, f"{return_var}_copy")
+                    break
+            if self.returntype is None:
+                t = None
+            else:
+                t = self.returntype[i]
+            self.ir_builder.add_output(self._current_fn, return_var, t, self._source_of(stmt))
+            return return_var
+
+        val = stmt.value
+        assert val is not None, "Return statement without return-value not supported."
+        if isinstance(val, ast.Tuple):
+            check_num_outputs(len(val.elts))
+            return [ret(exp, i, str(i)) for i, exp in enumerate(val.elts)]
+        check_num_outputs(1)
+        return ret(val, 0, "")
+
+    def _translate_if_stmt(self, stmt: ast.If) -> None:
+        if hasattr(stmt, "live_out"):
+            live_defs = list(
+                stmt.live_out.intersection(analysis.assigned_vars(stmt, self._message))
+            )
+        else:
+            live_defs = list(analysis.assigned_vars(stmt, self._message))
+        test = self._translate_expr(stmt.test, "cond").name
+        lineno = self._source_of(stmt).lineno
+        thenGraph, sub_fct_then = self._translate_block(
+            stmt.body, f"thenGraph_{lineno}", live_defs, parent_stmt=stmt
+        )
+        thenAttr = self._make_onnx_attr("then_branch", thenGraph)
+        elseGraph, sub_fct_else = self._translate_block(
+            stmt.orelse, f"elseGraph_{lineno}", live_defs, parent_stmt=stmt
+        )
+        elseAttr = self._make_onnx_attr("else_branch", elseGraph)
+
+        def rename(x):
+            r = self.generate_unique_name(x)
+            self._bind(
+                x,
+                values.Dynamic(r, values.DynamicKind.Intermediate, self._source_of(stmt)),
+            )
+            return r
+
+        # no break condition
+        renamed = [rename(x) for x in live_defs]
+        if not renamed:
+            self.fail(stmt, "A subgraph for a test do not have any output variable.")
+
+        sub_functions = {}
+        sub_functions.update(sub_fct_then)
+        sub_functions.update(sub_fct_else)
+        if renamed == [test]:
+            self.fail(stmt, f"Input and output cannot be the same {renamed!r}.")
+        self.emit(
+            renamed,
+            values.Op(self.default_opset, "If"),
+            [test],
+            [thenAttr, elseAttr],
+            sub_functions=sub_functions,
+        )
+
+    def _translate_loop_stmt(self, loop_stmt: Union[ast.For, ast.While]) -> None:
+        # loop-variable
+        if isinstance(loop_stmt, ast.For):
+            if not isinstance(loop_stmt.target, ast.Name):
+                self.fail(loop_stmt, "For loop target must be a single variable.")
+            p_loop_var = loop_stmt.target.id
+            # iter
+            iter = loop_stmt.iter
+            assert isinstance(iter, ast.Call), "Loop bound not a call."
+            if not isinstance(iter.func, ast.Name):
+                self.fail(loop_stmt, f"Unsupported loop bound {iter.func!r}.")
+            if iter.func.id != "range":
+                self.fail(
+                    loop_stmt, "Unsupported loop bound, only function 'range' is allowed."
+                )
+            if not iter.args or len(iter.args) != 1:
+                self.fail(loop_stmt, "Unsupported loop bound, it should be 'range(?)'.")
+            assert not iter.keywords, "Unsupported loop bound."
+            o_loop_bound = self._translate_expr(iter.args[0], "loop_bound").name
+            o_cond_var = self.generate_unique_name("cond_in")
+            i_cond_var = o_cond_var
+            cond_while = None
+            o_loop_condition = ""  # No condition for a for loop.
+        elif isinstance(loop_stmt, ast.While):
+            test = loop_stmt.test
+            if not isinstance(test, ast.Name):
+                self.fail(
+                    loop_stmt,
+                    "Unexpected condition type {type(loop_stmt)!r} for a while loop, "
+                    "it should be 'while <condition_name>:'.",
+                )
+            p_loop_var = "infinite_loop"
+            o_loop_bound = ""
+            i_cond_var = test.id
+            cond_while = test.id
+            o_cond_var = None
+            o_loop_condition = self._translate_name_expr(test)
+            # we need to go through all the instructions to see
+            # which instruction defines the condition test.id
+        else:
+            self.fail(loop_stmt, f"Unexpected loop type {type(loop_stmt)!r}.")
+        # analyze loop body
+        exposed_uses = analysis.exposed_uses(loop_stmt.body, self._message)
+        vars_def_in_loop = analysis.assigned_vars(loop_stmt.body, self._message)
+        loop_state_vars = vars_def_in_loop.intersection(exposed_uses | loop_stmt.live_out)
+        scan_outputs = set()  # TODO
+        outputs = list(loop_state_vars | scan_outputs)
+
+        # loop-condition:
+        # o_loop_condition = self._emit_const(True, "true", self._source_of(loop_stmt))
+
+        # build loop_body
+        self._enter_scope("loop_body", loop_stmt)
+        o_loop_var = self.generate_unique_name(p_loop_var)
+        self.ir_builder.add_input(
+            self._current_fn,
+            o_loop_var,
+            onnx_types.INT64,
+            self._source_of(loop_stmt),
+        )
+        self._bind(
+            p_loop_var,
+            values.Dynamic(o_loop_var, values.DynamicKind.Loop, self._source_of(loop_stmt)),
+        )
+
+        self.ir_builder.add_input(
+            self._current_fn,
+            i_cond_var,
+            onnx_types.BOOL,
+            self._source_of(loop_stmt),
+        )
+
+        for pv in loop_state_vars:
+            ov = self.generate_unique_name(pv)
+            # TODO: retrieve the annotation for variable pv is any is specified.
+            # typeinfo = self._eval_constant_expr(pv.annotation)
+            typeinfo = None
+            self.ir_builder.add_input(
+                self._current_fn, ov, typeinfo, self._source_of(loop_stmt)
+            )
+            self._bind(
+                pv,
+                values.Dynamic(ov, values.DynamicKind.Loop, self._source_of(loop_stmt)),
+            )
+
+        condition_name = None
+        operator_name = "Identity"
+        for i, s in enumerate(loop_stmt.body):
+            # We first need to intercept a break instruction in test block.
+            # It must be something like `if <condition_name>: break`.
+            # This instruction must be the last of the loop body.
+            if isinstance(s, ast.If) and len(s.body) == 1 and isinstance(s.body[0], ast.Break):
+                if not isinstance(s.test, ast.Name):
+                    self.fail(
+                        s,
+                        f"Instruction break can be introduced with test but it must be "
+                        f"if <condition>: break. However condition is of type "
+                        f"{type(s.test)!r}.",
+                    )
+                if i != len(loop_stmt.body) - 1:
+                    self.fail(s, "Instruction break must be the last one of the loop.")
+
+                current_scope = self._current_scope()
+                if s.test.id not in current_scope:
+                    self.fail(
+                        loop_stmt,
+                        f"Unable to find condition variable {s.test.id!r} in known "
+                        f"variables {list(current_scope)!r}.",
+                    )
+                condition_name = current_scope[s.test.id].value
+                operator_name = "Not"
+                continue
+            self._translate_stmt(s)
+
+        o_cond_out = self.generate_unique_name("cond_out")
+
+        if cond_while is not None:
+            # Loop while
+            current_scope = self._current_scope()
+            if cond_while not in current_scope:
+                self.fail(
+                    loop_stmt,
+                    f"Unable to find condition variable {cond_while!r} in known "
+                    f"variables {list(current_scope)!r}.",
+                )
+            o_cond_var = current_scope[cond_while].value
+
+        self.emit(
+            [o_cond_out],
+            values.Op(self.default_opset, operator_name),
+            [condition_name or o_cond_var],
+            [],
+        )
+
+        self.ir_builder.add_output(
+            self._current_fn,
+            o_cond_out,
+            onnx_types.BOOL,
+            self._source_of(loop_stmt),
+        )
+        for pv in loop_state_vars:
+            ov = self._py_var_to_onnx_var(pv, self._source_of(loop_stmt)).name
+            if ov not in self._current_fn.assigned_names:
+                # When converting the loop-body into a graph, we need to handle
+                # identity assignments of the form "x = y" inside the loop body
+                # specially if y represents a value computed outside the loop body.
+                # In this case, we create a copy of y, treating the statement as
+                # shorthand for "x = op.Identity(y)".
+                ov = self._emit_copy(ov, pv)
+            # TODO: retrieve variable type for the annotation if any.
+            typeinfo = None
+            self.ir_builder.add_output(
+                self._current_fn, ov, typeinfo, self._source_of(loop_stmt)
+            )
+        body = self._exit_scope()
+        inputs = [o_loop_bound, o_loop_condition] + [
+            self._py_var_to_onnx_var(pv, self._source_of(loop_stmt)).name
+            for pv in loop_state_vars
+        ]
+        graph, sub_functions = body.to_graph_and_functions()
+        attrs = [self._make_onnx_attr("body", graph)]
+        info = self._source_of(loop_stmt)
+
+        def rename(x):
+            r = self.generate_unique_name(x)
+            self._bind(x, values.Dynamic(r, values.DynamicKind.Output, info))
+            return r
+
+        onnx_outputs = [rename(x) for x in outputs]
+        self.emit(
+            onnx_outputs,
+            "Loop",
+            inputs,
+            attrs,
+            sub_functions=sub_functions,
+        )
+
+    def _translate_block(
+        self,
+        stmts: Sequence[ast.stmt],
+        name: str,
+        live_defs: Sequence[str],
+        parent_stmt: ast.stmt,
+    ):
+        """Translation of a statement-block to GraphProto attribute."""
+        info_stmt = stmts[0] if len(stmts) > 0 else parent_stmt
+        source = self._source_of(info_stmt)
+        self._enter_scope(name, None)
+        for s in stmts:
+            self._translate_stmt(s)
+        for pvar in live_defs:
+            if pvar in self._current_scope():
+                pv_val = self._current_scope()[pvar]
+                output = self._to_onnx_var(pv_val, pvar).name
+                if output not in self._current_fn.assigned_names:
+                    # To return an outer-scope variable, an ONNX Graph has to
+                    # use an explicit copy via Identity.
+                    output = self._emit_copy(output, pvar)
+                self.ir_builder.add_output(
+                    self._current_fn,
+                    output,
+                    pv_val.typeinfo,
+                    source,
+                )
+            else:
+                pv_val = None
+                for scope in self._locals:  # TODO: skip _current_scope
+                    if pvar in scope:
+                        pv_val = scope[pvar]
+                        break
+                if pv_val is None:
+                    self.fail(
+                        stmts[0],
+                        f"Variable {pvar} is not assigned a value along a conditional "
+                        f"branch, known variables: {list(self._locals)}.",
+                    )
+                # introduce a copy
+                ovar = self._emit_copy(self._to_onnx_var(pv_val, pvar).name, pvar)
+
+                # TODO: retrieve the annotation if any.
+                typeinfo = None
+                self.ir_builder.add_output(self._current_fn, ovar, typeinfo, source)
+        graph = self._exit_scope()
+        return graph.to_graph_and_functions()
+
+    def _translate_nested_function_def(self, fn: ast.FunctionDef) -> None:
+        """Translate a nested function definition."""
+        self._enter_scope(fn.name, fn)
+        self._translate_function_def_common(fn)
+        function_ir = self._exit_scope()
+        outer_scope_vars = analysis.outer_scope_variables(fn, self._message)
+        function_ir.outer_scope_variables = [
+            (var, self._lookup(var, self._source_of(fn))) for var in outer_scope_vars
+        ]
+        self._bind(fn.name, function_ir)
+        # TODO: Does not yet handle nested functions within nested functions.
+        self._current_fn.add_nested_function(function_ir)
+
+    def _translate_function_signature_common(
+        self, fn: ast.FunctionDef
+    ) -> irbuilder.IRFunction:
+        """Translate a function signature (top-level or nested)."""
+        args = fn.args
+        if args.vararg or args.kwonlyargs or args.kw_defaults or args.kwarg:
+            warn(f"{fn.name}: Unsupported feature in function signature.")
+        for i, x in enumerate(args.args):
+            arg_with_default_start_index = len(args.args) - len(args.defaults)
+            if args.defaults and i >= arg_with_default_start_index:
+                default_value = self._eval_constant_expr(
+                    args.defaults[i - arg_with_default_start_index]
+                )
+            else:
+                default_value = None
+            if x.annotation:
+                typeinfo = self._eval_constant_expr(x.annotation)
+                if not ta.is_valid_type(typeinfo):
+                    self.warn(
+                        x.annotation,
+                        f"Unsupported type annotation for argument {x.arg}.",
+                    )
+                    typeinfo = None
+            else:
+                # The code can only be exported as a function.
+                typeinfo = None
+            if typeinfo and ta.is_attr_type(typeinfo):
+                self.ir_builder.add_attr_parameter(
+                    self._current_fn,
+                    x.arg,
+                    ta.pytype_to_attrtype(typeinfo),
+                    default_value,
+                )
+                self._bind(x.arg, values.AttrRef(x.arg, typeinfo, self._source_of(x)))
+            else:
+                self.ir_builder.add_input(
+                    self._current_fn, x.arg, typeinfo, self._source_of(x)
+                )
+                self._used_vars.add(x.arg)
+                self._bind(
+                    x.arg,
+                    values.Dynamic(x.arg, values.DynamicKind.Input, self._source_of(x)),
+                )
+        if fn.returns:
+            type_annotation = self._eval_constant_expr(fn.returns)
+            self.returntype = ta.get_return_types(type_annotation)
+            invalid = False
+            for t in self.returntype:
+                if not ta.is_valid_type(t):
+                    self.warn(
+                        fn.returns,
+                        f"Unsupported type annotation for return value {t}.",
+                    )
+                    invalid = True
+            if invalid:
+                self.returntype = None
+        else:
+            self.returntype = None
+
+        return self._current_fn
+
+    def _translate_function_def_common(self, fn: ast.FunctionDef) -> irbuilder.IRFunction:
+        """Translate a function definition, including the signature and its body."""
+        logger.debug("Converter:_translate_function_def_common:%s", fn.name)
+        _ = self._translate_function_signature_common(fn)
+        for i, s in enumerate(fn.body):
+            self._translate_stmt(s, index_of_stmt=i)
+        return self._current_fn
+
+    def translate_function_def(self, stmt: ast.FunctionDef) -> irbuilder.IRFunction:
+        if isinstance(stmt, ast.FunctionDef):
+            self._init_function_translation()
+            if self.default_opset_ is None:
+                opset = self._find_onnx_opset(stmt)
+                if opset:
+                    self._set_default_opset(opset, stmt)
+            domain = self.this_module.domain
+            self._current_fn = self.ir_builder.new_function(stmt.name, domain, True)
+            analysis.do_liveness_analysis(stmt, self._message)
+            fn_ir = self._translate_function_def_common(stmt)
+            fn_ir.debug_print()
+            self.this_module.add_function_def(fn_ir)
+            return fn_ir
+        raise ValueError(f"Unsupported top-level statement type {type(stmt)!r}.")
+
+    def translate_function_signature(self, fn: ast.FunctionDef) -> irbuilder.IRFunction:
+        """Translate a (top-level) function signature."""
+        domain = self.this_module.domain
+        self._current_fn = self.ir_builder.new_function(fn.name, domain, True)
+        return self._translate_function_signature_common(fn)

pythonProject/.venv/Lib/site-packages/onnxscript/evaluator.py

@@ -0,0 +1,619 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+from __future__ import annotations
+
+import abc
+import contextlib
+import pprint
+from typing import (
+    Any,
+    Callable,
+    Mapping,
+    Optional,
+    Protocol,
+    Sequence,
+    TypeVar,
+    Union,
+    runtime_checkable,
+)
+
+import numpy as np
+import onnx
+import onnx.defs
+import onnx.reference
+from typing_extensions import TypeAlias
+
+from onnxscript import irbuilder, onnx_opset, tensor, values
+from onnxscript._internal import autocast, param_manipulation, utils
+
+UserModeValue: TypeAlias = Union[Optional[np.ndarray], Sequence["UserModeValue"]]
+
+EagerModeValue: TypeAlias = Union[Optional["tensor.Tensor"], Sequence["EagerModeValue"]]
+
+ExtendedModeValue: TypeAlias = Union[
+    Optional["tensor.Tensor"],
+    Sequence["ExtendedModeValue"],
+    np.ndarray,
+    int,
+    float,
+    bool,
+    str,
+]
+
+_T = TypeVar("_T")
+
+
+def _adapt_to_eager_mode(inputs: ExtendedModeValue) -> tuple[EagerModeValue, bool]:
+    """Adapts inputs into representation used by onnxscript eager mode.
+
+    This does the following transformations:
+    * It adds an onnxscript Tensor wrapper around numpy arrays, which
+    allows the use of overloaded operators like + to be controlled by onnxscript.
+    * It also provides a promotion of scalars into tensors as a convenience.
+    This is needed to complement the similar promotion supported by the
+    onnxscript converter (for example, when an attribute is promoted and used
+    as an input argument).
+
+    Args:
+        inputs: a list/tuple of inputs to an ONNX function
+
+    Returns:
+        a pair (wrapped_inputs, flag) where flag indicates whether any numpy array
+        was wrapped into a Tensor.
+    """
+    has_array = False
+
+    def adapt(input: ExtendedModeValue) -> EagerModeValue:
+        if isinstance(input, np.ndarray):
+            nonlocal has_array
+            has_array = True
+            return tensor.Tensor(input)
+        if isinstance(input, tensor.Tensor):
+            return input
+        if isinstance(input, (bool, float)):
+            return tensor.Tensor(np.array(input))
+        if isinstance(input, int):
+            return tensor.Tensor(np.array(input, dtype=np.int64))
+        if input is None:
+            return None
+        if isinstance(input, list):
+            return [adapt(elt) for elt in input]
+        if isinstance(input, tuple):
+            return tuple(adapt(elt) for elt in input)
+        raise TypeError(f"Unexpected input type {type(input)}.")
+
+    result = adapt(inputs)
+    return result, has_array
+
+
+def _adapt_to_user_mode(output: ExtendedModeValue) -> UserModeValue:
+    """Unwraps Tensor wrapper around numpy arrays.
+
+    Args:
+        output: output of an ONNX function, which can be either a single
+            onnx value or a list/tuple of onnx values.
+
+    Returns:
+        unwrapped output
+    """
+    if isinstance(output, tensor.Tensor):
+        return output.value
+    if output is None:
+        return None
+    if isinstance(output, list):
+        return [_adapt_to_user_mode(elt) for elt in output]
+    if isinstance(output, tuple):
+        return tuple(_adapt_to_user_mode(elt) for elt in output)
+    if isinstance(output, np.ndarray):
+        return output
+    raise TypeError(f"Unexpected type {type(output)}.")
+
+
+def _unwrap_tensors_in_kwargs(kwargs: Mapping[str, Any]) -> dict[str, Any]:
+    """Unwrap tensors in a mapping to numpy arrays."""
+    new_kwargs = {}
+    for k, v in kwargs.items():
+        new_kwargs[k] = v
+        if isinstance(v, tensor.Tensor):
+            new_kwargs[k] = v.value
+
+    return new_kwargs
+
+
+@runtime_checkable
+class Evaluator(Protocol):
+    """Protocol for evaluating ONNX ops."""
+
+    def eval(
+        self,
+        schema: onnx.defs.OpSchema,
+        inputs: Sequence[ExtendedModeValue],
+        attributes: Mapping[str, Any],
+    ):
+        """Evaluates an ONNX op.
+
+        Args:
+            schema: The OpSchema of the operator to evaluate.
+            inputs: The ONNX inputs to the op.
+            attributes: The ONNX attributes to the op.
+        """
+
+    def eval_function(
+        self,
+        function: values.OnnxFunction,
+        args: Sequence[ExtendedModeValue],
+        kwargs: Mapping[str, ExtendedModeValue],
+    ):
+        """Evaluates an OnnxFunction.
+
+        Args:
+            function: The OnnxFunction to evaluate.
+            args: The positional arguments to the function.
+            kwargs: The keyword arguments to the function.
+        """
+
+
+class BaseEvaluator(Evaluator, abc.ABC):
+    """Base class for evaluation of ONNX ops.
+
+    The execution of onnxscript functions in eager-mode is dispatched to an Evaluator
+    instance (or, more precisely, to the eval method of the Evaluator instance).
+    The evaluator is expected to transform the input/output/attribute representation
+    supported by onnxscript to those expected by a particular backend.
+    """
+
+    def __init__(self, ignore_unknown_function_kwargs: bool = False):
+        """Initializes a BaseEvaluator.
+
+        Args:
+            ignore_unknown_function_kwargs: Whether to ignore unknown keyword arguments
+                when evaluating an OnnxFunction. This is useful when using the
+                evaluator to validate operators programmatically, where
+                additional keyword arguments that is not part of the signature
+                may be provided to the function.
+        """
+        self._ignore_unknown_function_kwargs = ignore_unknown_function_kwargs
+
+    def eval(
+        self,
+        schema: onnx.defs.OpSchema,
+        inputs: Sequence[ExtendedModeValue],
+        attributes: Mapping[str, Any],
+    ):
+        """Evaluates an ONNX op.
+
+        Args:
+            schema: The OpSchema of the operator to evaluate.
+            inputs: The ONNX inputs to the op.
+            attributes: The ONNX attributes to the op.
+        """
+        attributes = _unwrap_tensors_in_kwargs(attributes)
+        attributes, closure = self.adapt_attributes(schema, attributes)
+        inputs = self.adapt_inputs(schema, inputs)
+        outputs = self._eval(schema, inputs, attributes, closure)
+        return self.adapt_outputs(schema, outputs)
+
+    def adapt_inputs(self, schema: onnx.defs.OpSchema, inputs: Sequence[ExtendedModeValue]):
+        """Transform inputs to the expected format for the evaluator.
+
+        Enables some syntactic sugar, such as the use of Python scalars,
+        in a manner consistent with the translator. See autocast.py for details.
+        """
+        return autocast.dynamic_cast_inputs(schema, inputs)
+
+    def adapt_attributes(
+        self, schema: onnx.defs.OpSchema, attributes: Mapping[str, ExtendedModeValue]
+    ) -> tuple[dict[str, ExtendedModeValue], dict[str, ExtendedModeValue]]:
+        """Transform attributes to the expected format for the evaluator.
+
+        Returns:
+            A closure that can be used to evaluate graph-valued attributes.
+        """
+        use_graph_attribute = self.use_graph_attribute(schema)
+        closure: dict[Any, Any] = {}
+        adapted_attributes = {}
+        for k, v in attributes.items():
+            if isinstance(v, values.OnnxClosure):
+                if use_graph_attribute:
+                    adapted_attributes[k] = v.function_ir.to_graph_proto()
+                    for pyvar, onnxvar in v.function_ir.outer_scope_variables:
+                        closure[onnxvar.value] = v.frame.f_locals[pyvar]
+                else:
+                    adapted_attributes[k] = v.function
+            elif callable(v):
+                raise TypeError(
+                    f"Error: function-valued attribute {v.__name__} has no graph_proto"
+                    "attribute. Did you forget to decorate it with @graph?"
+                )
+            else:
+                adapted_attributes[k] = v
+        return adapted_attributes, closure
+
+    def adapt_outputs(self, schema: onnx.defs.OpSchema, outputs: Sequence[EagerModeValue]):
+        """Adapt evaluator's output to convention used in onnxscript.
+
+        Onnxscript uses a tuple/sequence only when number of outputs > 1.
+        """
+        del schema  # unused
+        return outputs[0] if len(outputs) == 1 else outputs
+
+    def use_graph_attribute(self, schema: onnx.defs.OpSchema):
+        del schema  # unused
+        return True
+
+    @abc.abstractmethod
+    def _eval(
+        self,
+        schema: onnx.defs.OpSchema,
+        inputs: Sequence[ExtendedModeValue],
+        attributes: Mapping[str, ExtendedModeValue],
+        closure: Mapping[str, ExtendedModeValue],
+    ) -> EagerModeValue:
+        """Evaluates an ONNX op given its schema and inputs/attributes.
+
+        Args:
+            schema: The schema of the op to evaluate.
+            inputs: The ONNX inputs to the op.
+            attributes: The ONNX attributes to the op.
+            closure: The closure to use when evaluating graph-valued attributes.
+        """
+
+    def eval_function(
+        self,
+        function: values.OnnxFunction,
+        args: Sequence[ExtendedModeValue],
+        kwargs: Mapping[str, ExtendedModeValue],
+    ):
+        """Evaluates a function in eager mode.
+
+        Override this function to change the evaluator's behavior for functions.
+
+        Args:
+            function: The OnnxFunction to evaluate.
+            args: The positional arguments to the function.
+            kwargs: The keyword arguments to the function.
+        """
+        param_schemas = function.param_schemas()
+        # Split happens in the evaluator instead of the OnnxFunction __call__ method
+        # so that evaluators can control behaviors like whether to fill in default values for attributes.
+        tagged_args, tagged_kwargs = param_manipulation.tag_arguments_with_param_schemas(
+            param_schemas,
+            args,
+            kwargs,
+            fill_defaults=False,
+            allow_extra_kwargs=self._ignore_unknown_function_kwargs,
+        )
+
+        adapted_args: list[ExtendedModeValue] = []
+        adapted_kwargs: dict[str, ExtendedModeValue] = {}
+        has_array = False
+        for arg, param_schema in tagged_args:
+            if param_schema.is_input:
+                adapted_arg, has_array_ = _adapt_to_eager_mode(arg)
+                has_array = has_array or has_array_
+                adapted_args.append(adapted_arg)
+            else:
+                adapted_args.append(arg)
+
+        for key, (arg, param_schema) in tagged_kwargs.items():
+            if param_schema.is_input:
+                adapted_arg, has_array_ = _adapt_to_eager_mode(arg)
+                has_array = has_array or has_array_
+                adapted_kwargs[key] = adapted_arg
+            else:
+                adapted_kwargs[key] = arg
+
+        result = function.function(*adapted_args, **adapted_kwargs)
+
+        # We use a heuristic to decide whether to return output values as
+        # numpy arrays or tensor.Tensors. If the function has at least one
+        # numpy array as input, we return numpy arrays. Otherwise, we return
+        # tensor.Tensors. We could use a user-specified flag to control this
+        # or explicitly track whether this is a top-level function-call or
+        # a nested function-call.
+
+        return _adapt_to_user_mode(result) if has_array else result
+
+
+# Utilities for evaluation using ORT:
+
+
+class EagerModeError(RuntimeError):
+    pass
+
+
+def _rename_io(prefix, i, arg):
+    if arg is None:
+        return ""
+    return f"{prefix}{i}"
+
+
+def compute_num_outputs(
+    schema: onnx.defs.OpSchema, args: Sequence[Any], kwargs: Mapping[str, Any]
+) -> int:
+    """Returns the number of outputs expected."""
+
+    # TODO: Use ONNX type inference to replace the special-case handling below.
+    if schema.domain == "":
+        if schema.name == "BatchNormalization":
+            if not kwargs.get("training_mode", 0):
+                return 1
+        if schema.name == "LSTM":
+            return 3
+        if schema.name == "Split":
+            if len(args) == 1 and "num_outputs" not in kwargs:
+                raise EagerModeError(
+                    "Operator Split: the number of expected outputs defines the split. "
+                    "This information is unknown here."
+                )
+            if len(args) == 2:  # has argument(split)
+                return len(args[1])
+            else:  # no argument(split), check attribute(num_outputs)
+                return kwargs["num_outputs"]
+        if schema.name == "Scan":
+            scan_body = kwargs["body"]
+            return len(scan_body.output)
+        if schema.name == "Loop":
+            loop_body = kwargs["body"]
+            return len(loop_body.output) - 1
+    return len(schema.outputs)
+
+
+def _onnxscript_to_numpy_value(v):
+    """Converts an onnxscript encoding of an ONNX value into the numpy encoding used by runtimes."""
+    if isinstance(v, tensor.Tensor):
+        return v.value
+    if isinstance(v, list):
+        return [_onnxscript_to_numpy_value(x) for x in v]
+    if isinstance(v, tuple):
+        if len(v) > 0 and type(v[0]) is int:  # pylint: disable=unidiomatic-typecheck
+            return np.array(v, dtype=np.int64)
+        return np.array(v)
+    if v is None:
+        # Treated as a static-optional value.
+        # Dynamic optional None not yet supported.
+        return v
+    if isinstance(v, np.ndarray):
+        return v
+    raise TypeError(
+        f"Unexpected onnxscript value type '{type(v)}'."
+        "Valid value types are 'Tensor | list[Tensor] | None | np.ndarray | list[np.ndarray]'"
+    )
+
+
+def _numpy_to_onnxscript_value(
+    v: np.ndarray | np.generic | list[np.ndarray] | list[np.generic],
+):
+    """Converts an ORT encoding of an ONNX value into the encoding used by onnxscript."""
+    if isinstance(v, np.ndarray):
+        # ORT may reuse buffers when the output numpy array is provided back as input.
+        # We need to make a copy to ensure that the tensor is not modified in-place.
+        return tensor.Tensor(v.copy())
+    if issubclass(type(v), np.generic):
+        # Numpy scalar types that are not ndarray
+        # https://numpy.org/doc/stable/reference/arrays.scalars.html
+        return tensor.Tensor(np.array(v))
+    if isinstance(v, list):
+        return [_numpy_to_onnxscript_value(x) for x in v]
+    if v is None:
+        raise TypeError("Dynamic optional values not yet supported.")
+    raise TypeError(
+        f"Unexpected runtime value type '{type(v)}'."
+        "Valid types are: 'np.ndarray | np.generic | list[np.ndarray] | list[np.generic]'"
+    )
+
+
+def _prepare_model_and_inputs_for_eager(
+    schema: onnx.defs.OpSchema,
+    args: Sequence[Any],
+    kwargs: Mapping[str, Any],
+    implicit_args: Optional[Mapping[str, Any]],
+):
+    implicit_args = implicit_args or {}
+    # Convert input values to ORT representation-type:
+    args = [_onnxscript_to_numpy_value(x) for x in args]
+    implicit_args = {k: _onnxscript_to_numpy_value(v) for k, v in implicit_args.items()}
+
+    # Utility to convert kwarg to ONNX AttributeProto:
+    def make_attr(key: str, value: Any) -> onnx.AttributeProto:
+        def make_tensor_name() -> str:
+            return f"attr_{key}"
+
+        return autocast.pyvalue_to_onnx_attribute(
+            key, value, make_tensor_name, int(schema.attributes[key].type)
+        )
+
+    # Construct ONNX model with a single op call:
+    inputs = [_rename_io("input", i, arg) for i, arg in enumerate(args)]
+
+    num_outputs = compute_num_outputs(schema, args, kwargs)
+    outputs = [f"output{i}" for i in range(num_outputs)]
+
+    node = onnx.helper.make_node(schema.name, inputs, outputs, domain=schema.domain)  # noqa: TID251
+    node.attribute.extend(
+        make_attr(key, value) for key, value in kwargs.items() if value is not None
+    )
+    input_value_infos = utils.values_to_value_infos(zip(inputs, args))
+    implicit_value_infos = utils.values_to_value_infos(implicit_args.items())
+    output_value_infos = [
+        onnx.helper.make_value_info(name, onnx.TypeProto())  # noqa: TID251
+        for name in outputs
+    ]
+
+    graph = onnx.helper.make_graph(  # noqa: TID251
+        [node], "node_graph", input_value_infos + implicit_value_infos, output_value_infos
+    )
+    opset_id = onnx.helper.make_opsetid(schema.domain, schema.since_version)  # noqa: TID251
+    model = onnx.helper.make_model(  # noqa: TID251
+        graph,
+        opset_imports=[opset_id],
+        ir_version=irbuilder.select_ir_version(schema.since_version, domain=schema.domain),
+    )
+    model = onnx.shape_inference.infer_shapes(model)
+
+    session_run_input = {name: arg for name, arg in zip(inputs, args) if name != ""}
+    session_run_input.update(implicit_args)
+
+    return model, session_run_input, inputs
+
+
+def _call_ort(
+    schema: onnx.defs.OpSchema,
+    args: Sequence[Any],
+    kwargs: Mapping[str, Any],
+    implicit_args: Optional[Mapping[str, Any]],
+):
+    # Delay import onnxruntime so that onnxscript can be used without
+    # installing onnxruntime.
+    import onnxruntime as ort  # pylint: disable=import-outside-toplevel
+    from onnxruntime.capi.onnxruntime_pybind11_state import (  # pylint: disable=import-outside-toplevel
+        Fail,
+        InvalidArgument,
+        InvalidGraph,
+    )
+
+    model, session_run_input, inputs = _prepare_model_and_inputs_for_eager(
+        schema, args, kwargs, implicit_args
+    )
+
+    try:
+        session = ort.InferenceSession(
+            model.SerializeToString(), providers=("CPUExecutionProvider",)
+        )
+    except (Fail, InvalidGraph, InvalidArgument) as e:
+        raise EagerModeError(
+            f"Unable to create onnxruntime InferenceSession "
+            f"for executing {schema.domain}.{schema.name} op "
+            f"with onnx model\n{onnx.printer.to_text(model)}"
+        ) from e
+
+    try:
+        result = session.run(None, session_run_input)
+    except (RuntimeError, Fail) as e:
+        raise EagerModeError(
+            f"Unable to execute model operator {schema.name!r} due to {e!r}"
+            f"\ninput types:\n"
+            f"{pprint.pformat({k: type(v) for k, v in zip(inputs, args)})}"
+            f"\nmodified input types:\n"
+            f"{pprint.pformat({k: type(v) for k, v in session_run_input.items()})}"
+            f"\ninputs:\n{pprint.pformat(session_run_input)}\n{model}"
+        ) from e
+
+    # Map ORT output values to the onnxscript representation-type.
+    return [_numpy_to_onnxscript_value(x) for x in result]
+
+
+def _schema_id(schema: onnx.defs.OpSchema) -> tuple[str, str, int]:
+    return schema.name, schema.domain, schema.since_version
+
+
+class ORTEvaluator(BaseEvaluator):
+    """Evaluates ONNX ops using ONNX Runtime."""
+
+    def _eval(self, schema, inputs, attributes, closure):
+        return _call_ort(schema, inputs, attributes, closure)
+
+
+class OnnxReferenceRuntimeEvaluator(BaseEvaluator):
+    """Evaluates ONNX ops using ONNX Runtime."""
+
+    def _eval(self, schema, inputs, attributes, closure):
+        model, session_run_input, adapted_inputs = _prepare_model_and_inputs_for_eager(
+            schema, inputs, attributes, closure
+        )
+        session = onnx.reference.ReferenceEvaluator(model)
+        try:
+            result = session.run(None, session_run_input)
+        except RuntimeError as e:
+            raise EagerModeError(
+                f"Unable to execute model operator {schema.name!r} due to {e!r}"
+                f"\ninput types:\n"
+                f"{pprint.pformat({k: type(v) for k, v in zip(adapted_inputs, inputs)})}"
+                f"\nmodified input types:\n"
+                f"{pprint.pformat({k: type(v) for k, v in session_run_input.items()})}"
+                f"\ninputs:\n{pprint.pformat(session_run_input)}\n{model}"
+            ) from e
+
+        return [_numpy_to_onnxscript_value(x) for x in result]
+
+
+ort_evaluator = ORTEvaluator()
+
+
+class ORTMixedEvaluator(ORTEvaluator):
+    """Evaluates ONNX ops using ONNX Runtime, unless an overriding python implementation is registered.
+
+    This is useful for higher-order ops such as Scan and SequenceMap, allowing for
+    python-based debugging.
+    """
+
+    def __init__(self) -> None:
+        super().__init__()
+        self._python_ops: dict[tuple[str, str, int], Any] = {}
+
+    def use_graph_attribute(self, schema: onnx.defs.OpSchema) -> bool:
+        return _schema_id(schema) not in self._python_ops
+
+    def _eval(self, schema, inputs, attributes, closure):
+        schemaid = _schema_id(schema)
+        if schemaid in self._python_ops:
+            return self._python_ops[schemaid](inputs, attributes)
+        else:
+            return super()._eval(schema, inputs, attributes, closure)
+
+    def register(self, opset: values.Opset) -> Callable[[_T], _T]:
+        assert opset is not None
+
+        def decorator(function: _T) -> _T:
+            schema = opset[function.__name__]
+            self._python_ops[_schema_id(schema)] = function
+            return function
+
+        return decorator
+
+
+ort_mixed_evaluator = ORTMixedEvaluator()
+
+
+@ort_mixed_evaluator.register(opset=onnx_opset.opset18)
+def SequenceMap(inputs: Sequence[Any], attributes: Mapping[str, Any]):
+    """Evaluates a SequenceMap op."""
+    fun = attributes["body"]
+
+    def get_input_of(input_index, iter_num):
+        input = inputs[input_index]
+        if isinstance(input, list):
+            return input[iter_num]
+        return input
+
+    def get_input(iter_num):
+        return [get_input_of(input_index, iter_num) for input_index in range(len(inputs))]
+
+    return [fun(*(get_input(i))) for i in range(len(inputs[0]))]
+
+
+# Used to control the default evaluator instance. A simple approach for now.
+
+_default_evaluator: Evaluator = ort_evaluator
+
+
+def default() -> Evaluator:
+    """Returns the default Evaluator default."""
+    return _default_evaluator
+
+
+def set_default(new_default: Evaluator) -> None:
+    """Sets the current Evaluator default."""
+    global _default_evaluator  # pylint: disable=global-statement
+    _default_evaluator = new_default
+
+
+@contextlib.contextmanager
+def default_as(temp_default: Evaluator):
+    """Context manager that temporarily switches the default evaluator."""
+    old_default = _default_evaluator
+    set_default(temp_default)
+    try:
+        yield
+    finally:
+        set_default(old_default)

pythonProject/.venv/Lib/site-packages/onnxscript/irbuilder.py

@@ -0,0 +1,561 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+# ruff: noqa: TID251
+from __future__ import annotations
+
+import dataclasses
+import io
+import logging
+import warnings
+from typing import Any, Optional, Protocol, Sequence, Union
+
+import onnx
+from onnx import ValueInfoProto, helper
+from onnx.defs import onnx_opset_version
+
+import onnxscript
+from onnxscript import type_annotation as ta
+from onnxscript import values
+from onnxscript._internal import version_utils
+from onnxscript.onnx_types import ONNXType
+from onnxscript.sourceinfo import SourceInfo
+
+# A simple IR (Function, Stmt, Attr, Var):
+
+logger = logging.getLogger("onnxscript")
+
+
+def _format(seq: Sequence[Any], prefix: str, sep: str, suffix: str, formatter=str):
+    """Formats a sequence of objects into a string."""
+    return prefix + sep.join([formatter(x) for x in seq]) + suffix
+
+
+def select_ir_version(version: int, domain: str = "") -> int:
+    """Selects a suitable ONNX ir_version for a given opset version."""
+    if domain == "":
+        domain = "ai.onnx"
+    if (domain, version) not in helper.OP_SET_ID_VERSION_MAP:
+        return max(v for k, v in helper.OP_SET_ID_VERSION_MAP.items() if k[0] == "ai.onnx")
+    return helper.OP_SET_ID_VERSION_MAP[domain, version]
+
+
+class IRType:
+    def __init__(self):
+        self.onnx_type = onnx.TypeProto()
+
+    def to_type_proto(self):
+        return self.onnx_type
+
+    def __repr__(self) -> str:
+        return "IRType()"
+
+
+class IRTensorType(IRType):
+    def __init__(self, elem_type: onnx.TensorProto.DataType) -> None:
+        super().__init__()
+        self.onnx_type.tensor_type.elem_type = elem_type
+
+    def __repr__(self) -> str:
+        return f"IRTensorType({self.onnx_type.tensor_type.elem_type})"
+
+
+class IRTypeLike(Protocol):
+    def to_type_proto(self) -> onnx.TypeProto:
+        """Converts IR type representation to onnx.TypeProto"""
+
+
+class IRVar:
+    """A variable (representing a formal parameter)."""
+
+    def __init__(self, varname: str, typeinfo: IRTypeLike, sourceinfo: SourceInfo) -> None:
+        if not isinstance(varname, str):
+            raise TypeError(f"varname must be a string not {type(varname)!r}.")
+        self.name = varname
+        self.info = sourceinfo
+        self.typeinfo = typeinfo
+
+    def __str__(self):
+        return self.name
+
+    def __repr__(self):
+        return f"{self.__class__.__name__}({self.name!r}, {self.typeinfo!r})"
+
+    def typed_str(self):
+        return f"{self.name} : {self.typeinfo}"
+
+    def to_value_info(self, use_default_type: bool = True):
+        """Converts the content of this class into :class:`onnx.ValueInfoProto`.
+
+        Args:
+            use_default_type: if True, use a default type if an explicit type
+                is not known. Otherwise, returns a ValueInfoProto without type.
+
+        Returns:
+            an instance of :class:`onnx.ValueInfoProto`
+        """
+        if self.name is None:
+            raise ValueError(self.info.msg("name cannot be None."))
+        value_info_proto = ValueInfoProto()
+        value_info_proto.name = self.name
+        if self.typeinfo is not None:
+            value_info_proto.type.CopyFrom(self.typeinfo.to_type_proto())
+        elif use_default_type:
+            value_info_proto.type.CopyFrom(IRType().to_type_proto())
+        return value_info_proto
+
+
+def _opt_var_to_str(x):
+    return "" if x is None else str(x)
+
+
+class IRAttributeValue:
+    """An attribute value (representing an actual parameter).
+
+    Attributes:
+        name: The name of the attribute.
+        type: The type of the attribute.
+        attr_proto: The attribute proto.
+    """
+
+    def __init__(self, attrproto: onnx.AttributeProto) -> None:
+        self.attr_proto = attrproto
+
+    def __str__(self):
+        if self.attr_proto.HasField("ref_attr_name"):
+            return f"{self.attr_proto.name} = @{self.attr_proto.ref_attr_name}"
+        # self.name + " = " + self.value
+        return helper.printable_attribute(self.attr_proto)
+
+    @property
+    def name(self) -> str:
+        return self.attr_proto.name
+
+    @property
+    def type(self) -> onnx.AttributeProto.AttributeType:
+        return self.attr_proto.type
+
+
+@dataclasses.dataclass(frozen=True)
+class IRAttributeParameter:
+    """An attribute parameter (representing a formal parameter).
+
+    It may or may not carry a default value.
+
+    Attributes:
+        name: The name of the attribute.
+        type: The type of the attribute.
+        default_value: The default value of the attribute.
+        has_default: Whether the attribute has a default value.
+        attr_proto: The attribute proto.
+    """
+
+    name: str
+    type: onnx.AttributeProto.AttributeType
+    default_value: str | int | float | None = None
+
+    # TODO(justinchuby): Validate the default_value is the same type as specified in AttributeType.
+
+    def __str__(self):
+        if self.has_default:
+            return helper.printable_attribute(self.attr_proto)
+        # TODO(justinchuby): Include a readable type name.
+        return self.name
+
+    @property
+    def has_default(self):
+        return self.default_value is not None
+
+    @property
+    def attr_proto(self) -> onnx.AttributeProto:
+        if not self.has_default:
+            raise ValueError(
+                "Attribute has no default value. Only attributes with default "
+                "values can be converted to AttributeProto."
+            )
+        if version_utils.onnx_older_than("1.15"):
+            # TODO(after 1.14 is deprecated): Remove this branch.
+            # Argument 'attr_type' was added after version 1.14.
+            return helper.make_attribute(self.name, self.default_value)
+        # pylint: disable=unexpected-keyword-arg
+        return helper.make_attribute(self.name, self.default_value, attr_type=self.type)  # type: ignore[call-arg]
+        # pylint: enable=unexpected-keyword-arg
+
+
+class IRStmt:
+    def __init__(
+        self,
+        result: Sequence[str],
+        callee: values.Op,
+        args: Sequence[Optional[str]],
+        attrs: Sequence[IRAttributeValue],
+        sub_functions=None,
+    ) -> None:
+        if not isinstance(callee, values.Op):
+            raise TypeError(f"Unexpected type {type(callee)} for callee.")
+        self.result = result
+        self.callee = callee
+        self.args = args
+        self.attrs = attrs
+        self.functions = sub_functions or {}
+
+    def __str__(self):
+        if isinstance(self.result, str):
+            logger.debug("unexpected str type for self.result where type(self)=%r", type(self))
+        lhs = ", ".join(self.result)
+        attrs = ""
+        if self.attrs:
+            attrs = _format(self.attrs, "<", ", ", ">")
+
+        args = _format(self.args, "(", ", ", ")", _opt_var_to_str)
+        domain = self.callee.opset.domain
+        opname = self.callee.name
+        callee = f"{domain}.{opname}" if (domain != "") else opname
+        return f"{lhs} = {callee} {attrs}{args}"
+
+    def debug_print(self):
+        if logger.isEnabledFor(logging.DEBUG):
+            logger.debug("%s: %s", type(self), str(self))
+
+    def to_node_proto(self, node_name: str) -> onnx.NodeProto:
+        n = helper.make_node(
+            self.callee.name,
+            [_opt_var_to_str(x) for x in self.args],
+            [str(x) for x in self.result],
+            domain=self.callee.opset.domain,
+            name=node_name,
+        )
+        for a in self.attrs:
+            n.attribute.append(a.attr_proto)
+        return n
+
+    @property
+    def output_names(self) -> Sequence[str]:
+        """Returns the list of variables assigned to by this statement."""
+        return [str(x) for x in self.result]
+
+
+class IRFunction:
+    """Represents a function in the IR."""
+
+    def __init__(self, name: str, domain: str = "") -> None:
+        self.domain = domain
+        self.name = name
+        self.outputs: list[IRVar] = []
+        self.stmts: list[IRStmt] = []
+        self.called_functions: dict[str, onnx.FunctionProto] = {}
+        self.docstring: str = ""
+        # a dictionary of nested function-definitions
+        self.nested_functions: dict[str, IRFunction] = {}
+        self.outer_scope_variables: dict[Any, Any] = {}
+        self.ordered_inputs_and_attrs: list[Union[IRVar, IRAttributeParameter]] = []
+
+    @property
+    def assigned_names(self) -> Sequence[str]:
+        """Returns the list of variables assigned to by this function."""
+        return [v for stmt in self.stmts for v in stmt.output_names]
+
+    @property
+    def inputs(self) -> Sequence[IRVar]:
+        return [var for var in self.ordered_inputs_and_attrs if isinstance(var, IRVar)]
+
+    @property
+    def attrs(self) -> Sequence[IRAttributeParameter]:
+        return [
+            attr
+            for attr in self.ordered_inputs_and_attrs
+            if isinstance(attr, IRAttributeParameter)
+        ]
+
+    def __str__(self):
+        attrs = _format(self.attrs, "<", ", ", ">") if self.attrs else ""
+        inputs = _format([x.typed_str() for x in self.inputs], "(", ", ", ")")
+        outputs = _format([x.typed_str() for x in self.outputs], "(", ", ", ")")
+        stmts = _format(self.stmts, "\n{\n   ", "\n   ", "\n}\n")
+        return f"{self.name} {attrs}{inputs} => {outputs}{stmts}"
+
+    def append_docstring(self, docstring):
+        self.docstring += docstring
+
+    def append_stmt(self, stmt: IRStmt) -> None:
+        self.stmts.append(stmt)
+
+    def append_input(self, name: IRVar) -> None:
+        self.ordered_inputs_and_attrs.append(name)
+
+    def append_output(self, name: IRVar) -> None:
+        self.outputs.append(name)
+
+    def add_attr_parameter(self, attr: IRAttributeParameter) -> None:
+        self.ordered_inputs_and_attrs.append(attr)
+
+    def debug_print(self):
+        if logger.isEnabledFor(logging.DEBUG):
+            st = io.StringIO()
+            for s in self.stmts:
+                for attr in s.attrs:
+                    if attr.attr_proto.HasField("g"):
+                        st.write(helper.printable_graph(attr.attr_proto.g))
+                        st.write("\n")
+
+    def add_called_function(self, fun: values.OnnxFunction) -> None:
+        for name, fct in fun.function_ir.called_functions.items():
+            if name in self.called_functions:
+                continue
+            self.called_functions[name] = fct
+        if fun.name in self.called_functions:
+            # Already added.
+            return
+        try:
+            proto = fun.to_function_proto()
+        except (TypeError, AttributeError) as e:
+            raise TypeError(f"Issue with type f{type(fun)}.") from e
+        self.called_functions[fun.name] = proto
+
+    def add_nested_function(self, fun: IRFunction) -> None:
+        self.nested_functions[fun.name] = fun
+
+    def to_model_proto(
+        self,
+        functions=None,
+        io_types: Optional[ONNXType] = None,
+        input_types: Optional[Sequence[ONNXType]] = None,
+        output_types: Optional[Sequence[ONNXType]] = None,
+        value_infos: dict[str, ONNXType] | None = None,
+        **kwargs,
+    ) -> onnx.ModelProto:
+        """Converts this instance into a `onnx.ModelProto`.
+
+        Args:
+            functions: A list of functions to include in the model.
+                By default, all functions called at least once are included.
+            io_types: When specified, all the inputs/outputs of the model
+                are set to be of this type.
+            input_types: When specified, all the inputs of the model
+                are set to be of the corresponding type in this list.
+            output_types: When specified, all the outputs of the model
+                are set to be of the corresponding type in this list.
+            value_infos: A dictionary mapping intermediate variable names to ONNX types.
+                Used to set value_info for intermediate variables.
+            kwargs: Additional parameters given to function :func:`onnx.helper.make_model`.
+
+        Returns:
+            An instance of :class:`onnx.ModelProto`.
+        """
+        value_infos = (
+            [
+                onnx.helper.make_value_info(name, type.to_type_proto())
+                for name, type in value_infos.items()
+            ]
+            if value_infos
+            else None
+        )
+        graph, sub_functions = self.to_graph_and_functions(
+            use_default_type=False, value_infos=value_infos
+        )
+        if io_types is not None:
+            for input in graph.input:
+                if not input.HasField("type"):
+                    input.type.CopyFrom(io_types.to_type_proto())
+            for output in graph.output:
+                if not output.HasField("type"):
+                    output.type.CopyFrom(io_types.to_type_proto())
+        if input_types is not None:
+            for input, type in zip(graph.input, input_types):
+                input.type.CopyFrom(type.to_type_proto())
+        if output_types is not None:
+            for output, type in zip(graph.output, output_types):
+                output.type.CopyFrom(type.to_type_proto())
+        if functions is None:
+            functions = sub_functions.values()
+        else:
+
+            def to_proto(f):
+                if isinstance(f, onnx.FunctionProto):
+                    return f
+                if isinstance(f, onnxscript.OnnxFunction):
+                    return f.to_function_proto()
+                raise TypeError("Expected a value of type FunctionProto of OnnxFunction")
+
+            functions = [to_proto(f) for f in functions]
+
+        opsets = {}
+        for n in self.stmts:
+            if n.callee.opset.domain not in opsets:
+                opsets[n.callee.opset.domain] = n.callee.opset.version
+
+        for proto in functions:
+            if proto.domain not in opsets:
+                opsets[proto.domain] = 1
+            # TODO(rama): Handle conflicts with appropriate error/warning message.
+            for opset in proto.opset_import:
+                if opset.domain not in opsets:
+                    opsets[opset.domain] = opset.version
+
+        if "" not in opsets:
+            # No operator is using the standard opset.
+            # A default value is given.
+            opsets[""] = onnx_opset_version()
+
+        if "ir_version" not in kwargs:
+            kwargs["ir_version"] = select_ir_version(opsets[""])
+        opset_imports = [
+            onnx.helper.make_opsetid(domain, version) for domain, version in opsets.items()
+        ]
+
+        return helper.make_model(
+            graph, opset_imports=opset_imports, functions=functions, **kwargs
+        )
+
+    def to_graph_and_functions(
+        self,
+        use_default_type: bool = True,
+        value_infos: Sequence[ValueInfoProto] | None = None,
+    ) -> tuple[onnx.GraphProto, dict[str, onnx.FunctionProto]]:
+        """Converts this instance into a `onnx.GraphProto` and a map from
+        function-name to `onnx.FunctionProto`.
+
+        Args:
+            use_default_type: if True, the function uses a default type
+                for inputs and outputs that do not have a type
+            value_infos: a sequence of :class:`onnx.ValueInfoProto` to be added
+                to the graph.
+
+        Returns:
+            a pair of a :class:`onnx.GraphProto` and list of :class:`onnx.FunctionProto`
+        """
+        called_functions: dict[str, onnx.FunctionProto] = {}
+        for s in self.stmts:
+            called_functions.update(s.functions)
+        called_functions.update(self.called_functions)
+        graph = helper.make_graph(
+            [s.to_node_proto(f"n{i}") for i, s in enumerate(self.stmts)],
+            self.name,
+            [x.to_value_info(use_default_type) for x in self.inputs],
+            [y.to_value_info(use_default_type) for y in self.outputs],
+            value_info=value_infos,
+        )
+        return graph, called_functions
+
+    def to_graph_proto(self, use_default_type: bool = True) -> onnx.GraphProto:
+        """Converts this instance into a `onnx.GraphProto`.
+
+        Args:
+            use_default_type: if True, the function uses a default type
+                for inputs and outputs that do not have a type
+
+        Returns:
+            an instance of :class:`onnx.GraphProto`
+        """
+        graph, _ = self.to_graph_and_functions(use_default_type=use_default_type)
+        return graph
+
+    def get_opset_import(self) -> dict[str, int]:
+        func_opset_imports = {}
+        for s in self.stmts:
+            if s.callee.opset.domain not in func_opset_imports:
+                func_opset_imports[s.callee.opset.domain] = s.callee.opset.version
+            elif func_opset_imports[s.callee.opset.domain] != s.callee.opset.version:
+                warnings.warn(
+                    f"There is a version conflict in domain: {s.callee.opset.domain!r}, "
+                    f"with {self.name!r}.",
+                    category=UserWarning,
+                    stacklevel=1,
+                )
+        return func_opset_imports
+
+    def to_function_proto(self) -> onnx.FunctionProto:
+        """Converts this instance into a `onnx.FunctionProto`.
+
+        Note: Default values for attributes are an experimental feature in ONNX.
+        Conversion ignores default values for attributes if the ONNX version installed
+        doesn't support it.
+        """
+        opsets = self.get_opset_import()
+        nodes = [s.to_node_proto(f"n{i}") for i, s in enumerate(self.stmts)]
+        for n in nodes:
+            if n.domain not in opsets:
+                opsets[n.domain] = 1  # TODO: how to get n.version?
+        opset_imports = [
+            onnx.helper.make_opsetid(domain, version) for domain, version in opsets.items()
+        ]
+
+        attribute_names = [attr.name for attr in self.attrs if not attr.has_default]
+
+        f = helper.make_function(
+            self.domain,
+            self.name,
+            inputs=[x.name for x in self.inputs],
+            outputs=[y.name for y in self.outputs],
+            nodes=nodes,
+            opset_imports=opset_imports,  # TODO
+            attributes=attribute_names,
+            doc_string=self.docstring,
+        )
+        # In protobuf 4.x fields aren't defined as class attribute so it should check instance attribute instead
+        if hasattr(f, "attribute_proto"):
+            f.attribute_proto.extend(
+                [attr.attr_proto for attr in self.attrs if attr.has_default]
+            )
+        return f
+
+
+# IRBuilder: abstracts out details of the IR in the python-to-IR converter
+
+
+class IRBuilder:
+    def __init__(self):
+        self.functions = {}
+
+    def new_function(self, name: str, domain: str = "", register: bool = False) -> IRFunction:
+        if register and (domain, name) in self.functions:
+            raise RuntimeError(f"Function '{name}' already exists in domain '{domain}'.")
+        function = IRFunction(name, domain)
+        if register:
+            self.functions[domain, name] = function
+        return function
+
+    def add_docstring(self, fn: IRFunction, docstring: str):
+        fn.append_docstring(docstring)
+
+    def add_stmt(
+        self,
+        fn: IRFunction,
+        results: Sequence[str],
+        callee: values.Op,
+        args: Sequence[Optional[str]],
+        attrs: Sequence[IRAttributeValue],
+        sub_functions=None,
+    ) -> None:
+        stmt = IRStmt(results, callee, args, attrs, sub_functions=sub_functions)
+        fn.append_stmt(stmt)
+
+    def add_input(
+        self, fn: IRFunction, varname: str, type: IRTypeLike, info: SourceInfo
+    ) -> None:
+        var = IRVar(varname, type, info)
+        fn.append_input(var)
+
+    def add_attr_parameter(
+        self,
+        fn: IRFunction,
+        varname: str,
+        attribute_type: onnx.AttributeProto.AttributeType,
+        default_value: int | float | str | None,
+    ) -> None:
+        fn.add_attr_parameter(IRAttributeParameter(varname, attribute_type, default_value))
+
+    def add_output(self, fn: IRFunction, varname: str, typeinfo, sourceinfo) -> None:
+        var = IRVar(varname, typeinfo, sourceinfo)
+        fn.append_output(var)
+
+    def make_attr(self, attrproto: onnx.AttributeProto) -> IRAttributeValue:
+        return IRAttributeValue(attrproto)
+
+    def make_attr_ref(self, attrname: str, refname: str, pytype: type) -> IRAttributeValue:
+        proto = onnx.AttributeProto()
+        proto.name = attrname
+        proto.ref_attr_name = refname
+        attr_type = ta.pytype_to_attrtype(pytype)
+        assert attr_type is not None
+        proto.type = attr_type
+        return IRAttributeValue(proto)

pythonProject/.venv/Lib/site-packages/onnxscript/main.py

@@ -0,0 +1,176 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+# pylint disable: protected-access
+from __future__ import annotations
+
+import ast
+import inspect
+import sys
+from typing import Any, Callable, Optional, Sequence, TypeVar
+
+from typing_extensions import ParamSpec
+
+import onnxscript
+from onnxscript import converter, ir, irbuilder, values
+from onnxscript._internal import ast_utils
+
+_R = TypeVar("_R")
+_P = ParamSpec("_P")
+
+
+def script_check(
+    f: ast.FunctionDef,
+    opset: values.Opset,
+    global_names: dict[str, Any],
+    source: str,
+    default_opset: Optional[values.Opset] = None,
+) -> irbuilder.IRFunction:
+    """Check that a function falls into the ONNXScript subset of Python."""
+    # See if conversion succeeds.
+    # TODO: cleanup Converter interface/API, separating checker from
+    # converter
+    convert = converter.Converter(
+        opset=opset,
+        global_names=global_names,
+        source=source,
+        default_opset=default_opset,
+    )
+    return convert.translate_function_def(f)
+
+
+def script(
+    opset: Optional[values.Opset] = None,
+    default_opset: Optional[values.Opset] = None,
+    **kwargs: Any,
+) -> Callable[[Callable[_P, _R]], onnxscript.OnnxFunction[_P, _R]]:
+    """Main decorator. Declares a function as an onnx function.
+
+    Args:
+        opset: Opset the function belongs to (see :ref:`l-api-opsets`).
+        default_opset: Opset to use for operators not in the function's opset.
+        kwargs: Additional keyword arguments.
+
+    Returns:
+        an instance of :class:`onnxscript.values.OnnxFunction`
+
+    Example:
+    ::
+
+        @script()
+        def log2(x):
+            one = op.Constant(value=make_tensor('one', TensorProto.FLOAT, [1], [1]))
+            return op.Div(op.Log(x), op.CastLike(op.Log(cst), x))
+
+    Or:
+
+    ::
+
+        from onnxscript.onnx_opset import opset16
+
+        @script(opset16)
+        def log2(x):
+            one = op.Constant(value=make_tensor('one', TensorProto.FLOAT, [1], [1]))
+            return op.Div(op.Log(x), op.CastLike(op.Log(cst), x))
+    """
+    opset = opset or values.Opset("this", 1)
+    if not isinstance(opset, values.Opset):
+        raise TypeError(
+            "Script parameter must be an opset. Did you use @script instead of @script()?"
+        )
+
+    def transform(f: Callable[_P, _R]) -> onnxscript.OnnxFunction[_P, _R]:
+        if not inspect.isfunction(f):
+            raise TypeError("The ONNXScript decorator should be applied to functions only.")
+
+        src, f_ast = ast_utils.get_src_and_ast(f)
+        # The script should be compiled using the globals/locals at the definition site.
+        # This allows the script to reference names defined outside the script,
+        # which is used for a few different purposes.
+        # The following is an approximate solution that works for normal use.
+        module = inspect.getmodule(f)
+        closure = inspect.getclosurevars(f)
+        env = module.__dict__.copy()
+        env.update(closure.nonlocals)
+        result = script_check(f_ast, opset, env, src, default_opset=default_opset)
+        # TODO: add transformations.
+        return onnxscript.OnnxFunction(opset, f, result, src, kwargs)
+
+    return transform
+
+
+def graph() -> Callable[[Callable], values.OnnxClosure]:
+    """A parametric decorator used to annotate nested-functions that are used
+    as graph-attributes.
+
+    Returns:
+        A decorator that returns its input function, but attaches a graph_proto
+        attribute representing the input function. The translation is not
+        done at this time, but previously when the outer-level function
+        was translated to an OnnxFunction. The decorator just looks up
+        and retrieves the GraphProto representation previously generated.
+
+    Example:
+    ::
+
+        @script()
+        def cumulative_sum(X: INT64['N']):
+
+            # Translation of cumulative_sum by @script will also translate Sum
+            # into a GraphProto, which will be stored in the OnnxFunction generated
+            # for cumulative_sum. At run-time (in eager-mode), the @graph decorator
+            # retrieves the pre-computed GraphProto and attaches it to the Sum function.
+            @graph()
+            def Sum(sum_in, next):
+                sum_out = sum_in + next
+                scan_out = op.Identity(sum_out)
+                return sum_out, scan_out
+            zero = op.Constant(value_int=0)
+            # The call to higher-order operator Scan below uses the above function
+            # Sum as a graph-attribute.
+            all_sum, result = op.Scan (zero, X, body=Sum, num_scan_inputs=1)
+            return result
+
+    """
+    # This is a bit fragile. We want to get the ONNXFunction object representing
+    # the outer-scope ONNXScript function from the execution stack. The caller of
+    # @graph is the original script function (cumulative_sum in the above example),
+    # and the caller of that function is the wrapper function/method in the
+    # corresponding OnnxFunction object.
+    # Currently, there is no support for eager-mode execution of nested functions,
+    # so we don't need to handle doubly nested functions (e.g., a function defined
+    # inside Sum in the above example).
+
+    function_frame = sys._getframe(1)  # pylint: disable=protected-access
+    wrapper_frame = sys._getframe(3)  # pylint: disable=protected-access
+    onnx_function = wrapper_frame.f_locals["self"]
+    nested_functions = onnx_function.function_ir.nested_functions
+
+    def transform(f: Callable) -> values.OnnxClosure:
+        return values.OnnxClosure(nested_functions[f.__name__], function_frame, f)
+
+    return transform
+
+
+def is_converted_fun(f: Any) -> bool:
+    """Return True if f is a function converted by onnxscript decorator."""
+    return isinstance(f, onnxscript.OnnxFunction)
+
+
+def export_onnx_lib(functions: Sequence[values.OnnxFunction], filename: str) -> None:
+    # Since we don't yet have LibProto defined, we use a ModelProto as a temporary
+    # container for the list of functions exported as a library, with an empty graph
+    # and dummy opset_imports.
+
+    # TODO(justinchuby): This function is not well supported. We should consider removing it
+    model = ir.Model(
+        ir.Graph(
+            inputs=[],
+            outputs=[],
+            nodes=[],
+            opset_imports={"": 15},
+        ),
+        functions=[ir.serde.deserialize_function(f.to_function_proto()) for f in functions],
+        ir_version=10,
+        producer_name="p2o",
+    )
+    ir.save(model, filename)

pythonProject/.venv/Lib/site-packages/onnxscript/onnx_types.py

@@ -0,0 +1,237 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+from __future__ import annotations
+
+import abc
+from typing import ClassVar, Optional, Tuple, Union
+
+import onnx
+import onnx_ir as ir
+
+_DType = ir.DataType
+_DimType = Union[int, str, type(None)]
+_ShapeType = Union[Tuple[_DimType, ...], _DimType, type(Ellipsis)]
+
+_tensor_type_shape_cache: dict[_DType, TensorType] = {}
+tensor_type_registry: dict[_DType, TensorType] = {}
+
+
+def _check_dim(dim):
+    if not isinstance(dim, (int, str, type(None))):
+        raise TypeError(f"Invalid dimension {dim}")
+
+
+def _check_shape(shape):
+    if isinstance(shape, tuple):
+        for dim in shape:
+            _check_dim(dim)
+    elif shape != Ellipsis:
+        _check_dim(shape)
+
+
+class TensorType(abc.ABC):
+    """ONNX Script representation of a tensor type supporting shape annotations.
+
+    A scalar-tensor of rank 0:
+    ::
+
+        tensor: FLOAT
+
+    A tensor of unknown rank:
+    ::
+
+        tensor: FLOAT[...]
+
+    A tensor of rank 2 of unknown dimensions, with symbolic names:
+    ::
+
+        tensor: FLOAT['M', 'N']
+
+    A tensor of rank 2 of known dimensions:
+    ::
+
+        tensor: FLOAT[128, 1024]
+    """
+
+    dtype: ClassVar[_DType]
+    shape: ClassVar[Optional[_ShapeType]]
+
+    def __new__(cls):
+        raise NotImplementedError("TensorTypes cannot be instantiated")
+
+    def __init_subclass__(cls, dtype: _DType, shape: Optional[_ShapeType] = None):
+        cls.dtype = dtype
+        cls.shape = shape
+        if shape is None:
+            existing_cls = tensor_type_registry.get(dtype)
+            if existing_cls is not None:
+                raise ValueError(
+                    f"Invalid usage: subclass {existing_cls!r} "
+                    f"already defined for dtype={dtype}"
+                )
+            tensor_type_registry[dtype] = cls
+        else:
+            _check_shape(shape)
+
+    def __class_getitem__(cls, shape: Optional[_ShapeType]) -> type[TensorType]:
+        if cls.shape is not None:
+            raise ValueError("Invalid usage: shape already specified.")
+        if shape is None:
+            # Treat FLOAT[NONE] as 1-dimensional tensor with unknown dimension
+            shape = (None,)
+        key = (cls.dtype, shape)
+        shaped_type = _tensor_type_shape_cache.get(key)
+        if shaped_type is None:
+            shaped_type = type(cls.__name__, (TensorType,), {}, dtype=cls.dtype, shape=shape)
+            _tensor_type_shape_cache[key] = shaped_type
+        return shaped_type
+
+    @classmethod
+    def to_type_proto(cls) -> onnx.TypeProto:
+        if cls.shape is None:
+            shape = ()  # "FLOAT" is treated as a scalar
+        elif cls.shape is Ellipsis:
+            shape = None  # "FLOAT[...]" is a tensor of unknown rank
+        elif isinstance(cls.shape, tuple):
+            shape = cls.shape  # example: "FLOAT[10,20]"
+        else:
+            shape = [cls.shape]  # example: "FLOAT[10]"
+        return onnx.helper.make_tensor_type_proto(cls.dtype, shape)  # noqa: TID251
+
+    @classmethod
+    def to_string(cls) -> str:
+        return f"tensor({cls.__name__.lower()})"
+
+
+class FLOAT(TensorType, dtype=ir.DataType.FLOAT):
+    pass
+
+
+class UINT8(TensorType, dtype=ir.DataType.UINT8):
+    pass
+
+
+class INT8(TensorType, dtype=ir.DataType.INT8):
+    pass
+
+
+class UINT16(TensorType, dtype=ir.DataType.UINT16):
+    pass
+
+
+class INT16(TensorType, dtype=ir.DataType.INT16):
+    pass
+
+
+class INT32(TensorType, dtype=ir.DataType.INT32):
+    pass
+
+
+class INT64(TensorType, dtype=ir.DataType.INT64):
+    pass
+
+
+class STRING(TensorType, dtype=ir.DataType.STRING):
+    pass
+
+
+class BOOL(TensorType, dtype=ir.DataType.BOOL):
+    pass
+
+
+class FLOAT16(TensorType, dtype=ir.DataType.FLOAT16):
+    pass
+
+
+class DOUBLE(TensorType, dtype=ir.DataType.DOUBLE):
+    pass
+
+
+class UINT32(TensorType, dtype=ir.DataType.UINT32):
+    pass
+
+
+class UINT64(TensorType, dtype=ir.DataType.UINT64):
+    pass
+
+
+class COMPLEX64(TensorType, dtype=ir.DataType.COMPLEX64):
+    pass
+
+
+class COMPLEX128(TensorType, dtype=ir.DataType.COMPLEX128):
+    pass
+
+
+class BFLOAT16(TensorType, dtype=ir.DataType.BFLOAT16):
+    pass
+
+
+class FLOAT8E4M3FN(TensorType, dtype=ir.DataType.FLOAT8E4M3FN):
+    pass
+
+
+class FLOAT8E4M3FNUZ(TensorType, dtype=ir.DataType.FLOAT8E4M3FNUZ):
+    pass
+
+
+class FLOAT8E5M2(TensorType, dtype=ir.DataType.FLOAT8E5M2):
+    pass
+
+
+class FLOAT8E5M2FNUZ(TensorType, dtype=ir.DataType.FLOAT8E5M2FNUZ):
+    pass
+
+
+class INT4(TensorType, dtype=ir.DataType.INT4):
+    pass
+
+
+class UINT4(TensorType, dtype=ir.DataType.UINT4):
+    pass
+
+
+class FLOAT4E2M1(TensorType, dtype=ir.DataType.FLOAT4E2M1):
+    pass
+
+
+def onnx_type_to_onnxscript_repr(onnx_type: onnx.TypeProto, *, reversible: bool = True) -> str:
+    """Converts an onnx type into the string representation of the type in *onnxscript*.
+
+    Args:
+        onnx_type: an instance of onnx TypeProto
+        reversible: if True, the conversion produces only types that are
+            recognized by the onnxscript converter.
+
+    Returns:
+        The string representation of the type in onnxscript
+
+    Raises:
+        ...
+    """
+    if onnx_type.HasField("tensor_type"):
+        elem_type = onnx_type.tensor_type.elem_type
+        name = onnx.TensorProto.DataType.Name(elem_type)
+        if onnx_type.tensor_type.HasField("shape"):
+            shape = []
+            for d in onnx_type.tensor_type.shape.dim:
+                if d.HasField("dim_value"):
+                    shape.append(str(d.dim_value))
+                elif d.HasField("dim_param"):
+                    shape.append(repr(d.dim_param))
+                else:
+                    shape.append("None")
+            if not shape:
+                return name
+            return f"{name}[{','.join(shape)}]"
+        return f"{name}[...]"
+    if not reversible:
+        if onnx_type.HasField("sequence_type"):
+            elem_type = onnx_type.sequence_type.elem_type
+            return f"List[{onnx_type_to_onnxscript_repr(elem_type)}]"
+    raise NotImplementedError(f"Unable to translate type {onnx_type!r} into onnxscript type.")
+
+
+# Currently, only tensor types are supported. Need to expand support for other ONNX types.
+ONNXType = TensorType

pythonProject/.venv/Lib/site-packages/onnxscript/py.typed

@@ -0,0 +1 @@
+# Marker file for PEP-561 (inline types)

pythonProject/.venv/Lib/site-packages/onnxscript/sourceinfo.py

@@ -0,0 +1,59 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# -------------------------------------------------------------------------
+
+"""Source code information used for diagnostic messages."""
+
+from __future__ import annotations
+
+import ast
+from typing import Callable, Optional
+
+
+class SourceInfo:
+    """Information about onnxscript source fragment, used for diagnostic messages."""
+
+    def __init__(
+        self,
+        ast_node: ast.AST,
+        code: Optional[str] = None,
+        function_name: Optional[str] = None,
+    ):
+        self.ast_node = ast_node
+        self.code = code
+        self.function_name = function_name
+
+    @property
+    def lineno(self):
+        return self.ast_node.lineno
+
+    def msg(self, error_message: str) -> str:
+        lineno = self.lineno
+        if self.function_name:
+            source_loc = f"Function '{self.function_name}', line {lineno}"
+        else:
+            source_loc = f"Line {lineno}"
+
+        if self.code:
+            lines = self.code.split("\n")
+            line = lines[lineno - 1]
+            marker_prefix = " " * (self.ast_node.col_offset)
+            source_line = f"{line}\n{marker_prefix}^\n"
+        else:
+            source_line = ""
+
+        return f"ERROR: {error_message}\nat: {source_loc}\n{source_line}"
+
+    def __str__(self) -> str:
+        raise ValueError("Cannot happen!")
+
+
+Formatter = Callable[[ast.AST, str], str]
+
+
+def formatter(source_code: Optional[str]) -> Formatter:
+    def format(node: ast.AST, message: str) -> str:
+        return SourceInfo(node, source_code).msg(message)
+
+    return format

pythonProject/.venv/Lib/site-packages/onnxscript/tensor.py

@@ -0,0 +1,225 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
+from __future__ import annotations
+
+from typing import Any, Optional
+
+import numpy as np
+
+from onnxscript import ir, onnx_opset
+from onnxscript._internal import autocast
+
+
+class Tensor:
+    """An implementation of ONNX Tensors, based on a wrapper around numpy arrays.
+    Serves to define overloaded ops with an ONNX/ONNXScript semantics.
+    """
+
+    def __init__(self, nparray: Optional[np.ndarray], opset=None):
+        if nparray is not None and not isinstance(nparray, np.ndarray):
+            raise TypeError(
+                f"Unexpected type {type(nparray)}. It must be a numpy array or None."
+            )
+
+        self._nparray = nparray
+        # FIXME(justinhuby): Create a better way to determine the opset version
+        self._opset: Any = opset or onnx_opset.opset18
+
+    @property
+    def value(self) -> np.ndarray:
+        if self._nparray is None:
+            raise ValueError("Tensor does not have a value.")
+        return self._nparray
+
+    @property
+    def rank(self) -> int:
+        return len(self.value.shape)
+
+    @property
+    def is_scalar(self) -> bool:
+        return self.rank == 0
+
+    @property
+    def shape(self) -> tuple[int, ...]:
+        return self.value.shape
+
+    @property
+    def dtype(self) -> np.dtype:
+        return self.value.dtype
+
+    @property
+    def onnx_dtype(self) -> int:
+        return ir.DataType.from_numpy(self.dtype)
+
+    def __repr__(self) -> str:
+        return f"{self.__class__.__name__}({self.value!r})"
+
+    def __bool__(self) -> bool:
+        return bool(self.value)
+
+    def __int__(self) -> int:
+        return int(self.value)
+
+    def __float__(self) -> float:
+        return float(self.value)
+
+    def __len__(self) -> int:
+        return self.shape[0]
+
+    def __index__(self) -> int:
+        return self.value.__index__()
+
+    def __getitem__(self, index):
+        op = self._opset
+        if op.version < 13:
+            raise RuntimeError("Indexing requires opset 13 or later.")
+        if not isinstance(index, tuple):
+            # Normalize representation to a tuple.
+            # A single index-value is equivalent to a tuple with a single element.
+            index = (index,)
+        if len(index) > self.rank:
+            raise ValueError(
+                f"Number of indices {len(index)} is greater than rank {self.rank}"
+            )
+
+        # Promote integer indices to tensors of rank 0
+        index = [autocast.cast_pyvalue_to_os_tensor(x) for x in index]
+        # Process all elements in index
+        shape = self.shape
+        sliced_indices = []
+        scalar_indices = []
+        to_squeeze = []
+        non_scalar_indices = []
+        for axis_, s in enumerate(index):
+            if isinstance(s, slice):
+                if s.start is None and s.stop is None and s.step is None:
+                    continue
+                if s.step is None or s.step > 0:
+                    sliced_indices.append(
+                        [
+                            s.start or 0,
+                            s.stop if s.stop is not None else shape[axis_],
+                            axis_,
+                            s.step or 1,
+                        ]
+                    )
+                else:
+                    sliced_indices.append(
+                        [
+                            s.start if s.start is not None else (shape[axis_] - 1),
+                            s.stop if s.stop is not None else -(shape[axis_] + 1),
+                            axis_,
+                            s.step,
+                        ]
+                    )
+            elif isinstance(s, Tensor):
+                if s.is_scalar:
+                    scalar_indices.append([s, s + 1, axis_, 1])
+                    to_squeeze.append(axis_)
+                else:
+                    non_scalar_indices.append((axis_, s))
+            else:
+                raise TypeError(f"Unexpected type {type(s)}: slice or int expected.")
+
+        # Non-scalar-indexing requires the use of ONNX Gather operation.
+        # Slicing can be implemented efficiently using ONNX's Slice operation.
+        # Scalar-indexing can be implemented using either Gather or with the Slice operation.
+        # We map scalar-indexing into the Slice operation, except in the special case
+        # of a single scalar-index (with no other sliced_index), which we map directly
+        # to a Gather.
+
+        if not (sliced_indices or scalar_indices or non_scalar_indices):
+            # Edge case: no index specified. Eg. A[:, :]
+            return op.Identity(self)
+        if not sliced_indices and len(scalar_indices) == 1:
+            # Special case of indexing along a single axis: A[i], A[:, i], A[:, :, i] etc.
+            # promote integer input to tensor
+            axis = to_squeeze[0]
+            index_value = index[axis]
+            # use Gather to perform indexing
+            result = op.Gather(self, index_value, axis=axis)
+        elif sliced_indices or scalar_indices:
+            sliced_indices = sliced_indices + scalar_indices
+            indices = np.array(sliced_indices, dtype=np.int64).T
+            starts = Tensor(indices[0])
+            ends = Tensor(indices[1])
+            axes = Tensor(indices[2])
+            steps = Tensor(indices[3])
+            result = op.Slice(self, starts, ends, axes, steps)
+            if to_squeeze:
+                result = Tensor(np.squeeze(result.value, axis=tuple(to_squeeze)))
+        else:
+            result = self
+        for axis, value in non_scalar_indices:
+            result = op.Gather(result, value, axis=axis)
+
+        return result
+
+    def __mod__(self, other):
+        if self.onnx_dtype in {
+            ir.DataType.FLOAT,
+            ir.DataType.DOUBLE,
+            ir.DataType.FLOAT16,
+            ir.DataType.BFLOAT16,
+        }:
+            return self._opset.Mod(self, other, fmod=1)
+        return self._opset.Mod(self, other)
+
+    def __ne__(self, other):
+        temp = self._opset.Equal(self, other)
+        return self._opset.Not(temp)
+
+    def __neg__(self):
+        return self._opset.Neg(self)
+
+    def __add__(self, other):
+        return self._opset.Add(self, other)
+
+    def __radd__(self, other):
+        return self._opset.Add(other, self)
+
+    def __and__(self, other):
+        return self._opset.And(self, other)
+
+    def __rand__(self, other):
+        return self._opset.And(other, self)
+
+    def __mul__(self, other):
+        return self._opset.Mul(self, other)
+
+    def __rmul__(self, other):
+        return self._opset.Mul(other, self)
+
+    def __matmul__(self, other):
+        return self._opset.MatMul(self, other)
+
+    def __or__(self, other):
+        return self._opset.Or(self, other)
+
+    def __pow__(self, other):
+        return self._opset.Pow(self, other)
+
+    def __sub__(self, other):
+        return self._opset.Sub(self, other)
+
+    def __rsub__(self, other):
+        return self._opset.Sub(other, self)
+
+    def __truediv__(self, other):
+        return self._opset.Div(self, other)
+
+    def __lt__(self, other):
+        return self._opset.Less(self, other)
+
+    def __le__(self, other):
+        return self._opset.LessOrEqual(self, other)
+
+    def __eq__(self, other):
+        return self._opset.Equal(self, other)
+
+    def __ge__(self, other):
+        return self._opset.GreaterOrEqual(self, other)
+
+    def __gt__(self, other):
+        return self._opset.Greater(self, other)

pythonProject/.venv/Lib/site-packages/onnxscript/type_annotation.py

@@ -0,0 +1,288 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+from __future__ import annotations
+
+import collections
+import inspect
+import typing
+from typing import Optional, Sequence, Union
+
+import onnx
+
+from onnxscript import onnx_types
+from onnxscript._internal import version_utils
+
+# TypeAnnotationValue represents the (value of) valid type-annotations recognized
+# by ONNX Script. TODO: Flesh out a formal definition. Currently, it supports
+# - float, int, str (primitive attribute types)
+# - Sequence[float], Sequence[int], Sequence[str] (attribute types)
+# - Tensor types
+# - Sequence[Tensor] types
+# - Union of above 2
+# - TypeVars with above bounds
+# - Above types with annotation attached
+TypeAnnotationValue = typing.Any
+
+# Map from python type to corresponding ONNX AttributeProto type
+_PYTYPE_TO_ATTRTYPE_MAP = {
+    float: onnx.AttributeProto.FLOAT,
+    int: onnx.AttributeProto.INT,
+    str: onnx.AttributeProto.STRING,
+    bool: onnx.AttributeProto.INT,  # experimental
+}
+
+# Map from python type to corresponding ONNX AttributeProto type,
+# for repeated (i.e., list of) values
+_LISTTYPE_TO_ATTRTYPE_MAP = {
+    float: onnx.AttributeProto.FLOATS,
+    int: onnx.AttributeProto.INTS,
+    str: onnx.AttributeProto.STRINGS,
+    bool: onnx.AttributeProto.INTS,  # experimental
+}
+
+_LIST_CONSTRUCTORS = frozenset([list, typing.List, typing.Sequence, collections.abc.Sequence])
+
+# Map from ONNX AttributeProto type to its representation (in ONNX Script).
+_ATTRTYPE_TO_REPR = {
+    onnx.AttributeProto.FLOAT: "float",
+    onnx.AttributeProto.INT: "int",
+    onnx.AttributeProto.STRING: "str",
+    onnx.AttributeProto.FLOATS: "Sequence[float]",
+    onnx.AttributeProto.INTS: "Sequence[int]",
+    onnx.AttributeProto.STRINGS: "Sequence[str]",
+}
+
+
+def onnx_attr_type_to_onnxscript_repr(attr_type: onnx.AttributeProto.AttributeType) -> str:
+    if attr_type not in _ATTRTYPE_TO_REPR:
+        supported = ", ".join(
+            f"'{onnx.AttributeProto.AttributeType.Name(v)}'" for v in _ATTRTYPE_TO_REPR
+        )
+        raise ValueError(f"Unsupported attribute type {attr_type}: only {supported} allowed.")
+    return _ATTRTYPE_TO_REPR[attr_type]
+
+
+# A sorted list of all type strings used in an OpSchema
+ALL_TENSOR_TYPE_STRINGS = tuple(
+    sorted(
+        tensor_type.to_string()
+        for tensor_type in onnx_types.tensor_type_registry.values()
+        # Skip FLOAT4E2M1 for versions older than 1.18
+        # TODO(after onnx requirement bump): Remove this check
+        if not (version_utils.onnx_older_than("1.18") and tensor_type == onnx_types.FLOAT4E2M1)
+    )
+)
+
+
+def _remove_annotation(typeinfo: TypeAnnotationValue) -> TypeAnnotationValue:
+    """Remove Annotated wrapper if present, otherwise return typeinfo as is."""
+    if hasattr(typing, "Annotated"):
+        # Present in Python 3.9+
+        if typing.get_origin(typeinfo) is typing.Annotated:
+            return typing.get_args(typeinfo)[0]
+    return typeinfo
+
+
+def _is_primitive_attr_type(typeinfo: TypeAnnotationValue) -> bool:
+    return typeinfo in _PYTYPE_TO_ATTRTYPE_MAP
+
+
+def pytype_to_attrtype(
+    pytype: TypeAnnotationValue,
+) -> Optional[onnx.AttributeProto.AttributeType]:
+    pytype = _remove_annotation(pytype)
+    if pytype in _PYTYPE_TO_ATTRTYPE_MAP:
+        return _PYTYPE_TO_ATTRTYPE_MAP[pytype]
+    type_constructor = typing.get_origin(pytype)
+    # Remove Optional wrapper if present, which is represented as an Union[..., type(None)]
+    if type_constructor is typing.Union:
+        # Filter out type(None), since typing.Optional[X] evaluates to Union[X, type(None)]
+        args = [x for x in typing.get_args(pytype) if x is not type(None)]
+        if len(args) == 1:
+            return pytype_to_attrtype(args[0])
+    if type_constructor in _LIST_CONSTRUCTORS:
+        elt_type = typing.get_args(pytype)[0]
+        if elt_type in _LISTTYPE_TO_ATTRTYPE_MAP:
+            return _LISTTYPE_TO_ATTRTYPE_MAP[elt_type]
+    return None
+
+
+def base_type_is_bool(pytype: TypeAnnotationValue) -> bool:
+    """Returns True if base type of pytype is bool, False otherwise."""
+    pytype = _remove_annotation(pytype)
+    if pytype in _PYTYPE_TO_ATTRTYPE_MAP:
+        return pytype is bool
+    type_constructor = typing.get_origin(pytype)
+    if type_constructor in _LIST_CONSTRUCTORS:
+        element_type = typing.get_args(pytype)[0]
+        return element_type is bool
+    # Remove Optional wrapper if present:
+    if type_constructor is Optional or type_constructor is Union:
+        # In Python < 3.10, Optional[X] is represented as Union[X, type(None)]
+        # so we filter out type(None) if present
+        args = [x for x in typing.get_args(pytype) if x is not type(None)]
+        if len(args) == 1:
+            return base_type_is_bool(args[0])
+
+    return False
+
+
+def _is_tensor_type(typeinfo: TypeAnnotationValue) -> bool:
+    if isinstance(typeinfo, onnx_types.TensorType):
+        return True
+    if inspect.isclass(typeinfo) and issubclass(typeinfo, onnx_types.TensorType):
+        return True
+    return False
+
+
+def is_value_type(typeinfo: TypeAnnotationValue) -> bool:
+    """Returns True if typeinfo represents a value type, False if it is an attribute type.
+    Raises ValueError if typeinfo is not a supported type annotation.
+    """
+    typeinfo = _remove_annotation(typeinfo)
+    if _is_tensor_type(typeinfo):
+        return True
+    if _is_primitive_attr_type(typeinfo):
+        return False
+    type_constructor = typing.get_origin(typeinfo)
+    # Handle List-like type-constructor
+    # Eg. List[INT32] is a value type, while List[int] is an attribute type
+    if type_constructor in _LIST_CONSTRUCTORS:
+        elt_type = typing.get_args(typeinfo)[0]
+        return is_value_type(elt_type)
+    # Handle Union and Optional type-constructors
+    if type_constructor is typing.Union:
+        # Filter out None, since typing.Optional[X] evaluates to Union[X, None]
+        args = [x for x in typing.get_args(typeinfo) if x is not type(None)]
+        args_value_check = [is_value_type(x) for x in args]
+        if all(args_value_check):
+            # Handles cases like Optional[INT32] as well as Union[FLOAT16, FLOAT, DOUBLE]
+            return True
+        elif (len(args) == 1) and args_value_check[0] is False:
+            # Handle the case of optional attribute: eg. Optional[int]
+            # Note that we do not allow Union[int, float] for attributes.
+            return False
+        else:
+            raise ValueError(f"Unsupported type annotation '{typeinfo}'")
+    # Handle TypeVars:
+    if isinstance(typeinfo, typing.TypeVar):
+        if hasattr(typeinfo, "__bound__"):
+            bound = typeinfo.__bound__
+            return is_value_type(bound)
+    raise ValueError(f"Unsupported type annotation {typeinfo}")
+
+
+def is_attr_type(pytype: TypeAnnotationValue):
+    return is_value_type(pytype) is False
+
+
+def is_valid_type(typeinfo: TypeAnnotationValue):
+    try:
+        return is_value_type(typeinfo) in {True, False}
+    except ValueError:
+        return False
+
+
+def is_optional(pytype) -> bool:
+    """Returns whether a pytype is an Optional."""
+    if typing.get_origin(pytype) is Union and type(None) in typing.get_args(pytype):
+        # Python < 3.10
+        return True
+    if typing.get_origin(pytype) is Optional:
+        # Python >= 3.10
+        return True
+    return False
+
+
+def get_return_types(typeinfo: type | Sequence[type]) -> Sequence[type]:
+    """Converts return-type annotation into a sequence of types.
+
+    The return type annotation can be either a single type (for a single output)
+    or a Tuple type (for multiple outputs). This function normalizes the
+    representation so that it is always a sequence of types, even for a single
+    output.
+    """
+    if isinstance(typeinfo, typing.Sequence):
+        return typeinfo
+    if typing.get_origin(typeinfo) is tuple:
+        return typing.get_args(typeinfo)
+    return (typeinfo,)
+
+
+def pytype_to_type_strings(pytype: TypeAnnotationValue) -> list[str]:
+    """Returns a list of type-strings corresponding to a given type annotation.
+
+    Args:
+        pytype: A type annotation.
+
+    Returns:
+        A list of all supported input types for the given type annotation.
+        Ensures that the list is sorted in the same order as ALL_TYPE_STRINGS.
+    """
+    if pytype is None:
+        return list(ALL_TENSOR_TYPE_STRINGS)
+    if pytype is onnx_types.TensorType:
+        return list(ALL_TENSOR_TYPE_STRINGS)
+    if isinstance(pytype, type) and issubclass(pytype, onnx_types.TensorType):
+        return [pytype.to_string()]
+    if isinstance(pytype, onnx_types.TensorType):
+        return [pytype.to_string()]
+    if isinstance(pytype, typing.TypeVar):
+        constraints = pytype.__constraints__
+        if constraints:
+            return pytype_to_type_strings(Union.__getitem__(constraints))  # pylint: disable=unnecessary-dunder-call
+        bound = pytype.__bound__
+        if bound is None:
+            return list(ALL_TENSOR_TYPE_STRINGS)
+        return pytype_to_type_strings(bound)
+    if typing.get_origin(pytype) is Union:
+        options = []
+        subtypes = typing.get_args(pytype)
+        # A None type in a Union is equivalent to an optional type
+        optional = is_optional(pytype)
+        for subtype in subtypes:
+            if subtype is type(None):
+                # Skip None type because we are handling it with is_optional
+                continue
+            if optional:
+                options += [
+                    *pytype_to_type_strings(subtype),
+                    *[f"optional({s})" for s in pytype_to_type_strings(subtype)],
+                ]
+            else:
+                options += pytype_to_type_strings(subtype)
+        # Remove duplicates
+        return sorted(set(options))
+    if typing.get_origin(pytype) in _LIST_CONSTRUCTORS:
+        subtypes = typing.get_args(pytype)
+        return [f"seq({s})" for s in pytype_to_type_strings(subtypes[0])]
+
+    raise ValueError(f"Unsupported type: {pytype}")
+
+
+def get_type_constraint_name(pytype: TypeAnnotationValue) -> Optional[str]:
+    """Returns the name of the type constraint for a given type annotation.
+
+    Args:
+        pytype: A type annotation.
+
+    Returns:
+        The name of the type constraint if it is a TypeVar.
+        - Prefixes the name with "Optional_" if the type annotation is Optional[TypeVar].
+        - Prefixes the name with "Sequence_" if the type annotation is a Sequence[].
+        - Returns None if the type annotation does not have a type constraint.
+    """
+    if isinstance(pytype, typing.TypeVar):
+        return pytype.__name__
+    if is_optional(pytype):
+        subtypes = typing.get_args(pytype)
+        for subtype in subtypes:
+            if subtype is type(None):
+                continue
+            type_param_name = get_type_constraint_name(subtype)
+            return f"Optional_{type_param_name}" if type_param_name else None
+    if typing.get_origin(pytype) in _LIST_CONSTRUCTORS:
+        subtypes = typing.get_args(pytype)
+        type_param_name = get_type_constraint_name(subtypes[0])
+        return f"Sequence_{type_param_name}" if type_param_name else None
+    return None