-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Docs][PyOV] Documentation for Custom Python Operation #25615
base: master
Are you sure you want to change the base?
Changes from all commits
205044d
13c0a10
5b7f90e
e0f40f9
8a417a6
95ba862
091af42
3cb000a
9939776
f0c3733
4e20847
a237a03
753562d
9ba1424
d25ac84
1c4732f
110fd42
c4d1de8
815bba4
32d9076
0d0543f
b505644
77040b4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Copyright (C) 2018-2024 Intel Corporation | ||
# SPDX-License-Identifier: Apache-2.0 | ||
# | ||
|
||
# ! [op:common_include] | ||
import numpy as np | ||
from openvino import Op | ||
from openvino.runtime import DiscreteTypeInfo | ||
# ! [op:common_include] | ||
|
||
|
||
|
||
# ! [op:header] | ||
class Identity(Op): | ||
class_type_info = DiscreteTypeInfo("Identity", "extension") | ||
|
||
def get_type_info(self): | ||
return Identity.class_type_info | ||
# ! [op:header] | ||
|
||
# ! [op:ctor] | ||
def __init__(self, inputs): | ||
super().__init__(self) | ||
self.set_arguments(inputs) | ||
self.constructor_validate_and_infer_types() | ||
# ! [op:ctor] | ||
|
||
# ! [op:validate] | ||
def validate_and_infer_types(self): | ||
self.set_output_type(0, self.get_input_element_type(0), self.get_input_partial_shape(0)) | ||
# ! [op:validate] | ||
|
||
# ! [op:copy] | ||
def clone_with_new_inputs(self, new_inputs): | ||
return Identity(new_inputs) | ||
# ! [op:copy] | ||
|
||
# ! [op:evaluate] | ||
def evaluate(self, outputs, inputs): | ||
outputs[0].shape = inputs[0].shape | ||
mlukasze marked this conversation as resolved.
Show resolved
Hide resolved
|
||
inputs[0].copy_to(outputs[0]) | ||
return True | ||
|
||
def has_evaluate(self): | ||
return True | ||
Comment on lines
+44
to
+45
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we avoid doing that by just defining There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As far as I understand this method is obligatory and always goes together with |
||
# ! [op:evaluate] | ||
|
||
# ! [op:visit_attributes] | ||
def visit_attributes(self, visitor): | ||
return True | ||
# ! [op:visit_attributes] |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -110,9 +110,9 @@ The ``Identity`` is a custom operation class defined in :doc:`Custom Operation G | |
:language: cpp | ||
:fragment: [add_frontend_extension] | ||
|
||
When Python API is used, there is no way to implement a custom OpenVINO operation. Even if custom OpenVINO operation is implemented in C++ and loaded into the runtime by a shared library, there is still no way to add a frontend mapping extension that refers to this custom operation. In this case, use C++ shared library approach to implement both operations semantics and framework mapping. | ||
The Python API now supports implementation of custom OpenVINO operations, enabling direct integration within Python code. Still python custom operations won't be loaded into the runtime, so use the C++ shared library approach to implement both operation semantics and framework mapping that refers to this custom operation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not release notes, I suppose. So There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The meaning of the whole sentence doesn't look correct: "Still python custom operations won't be loaded into the runtime, so use the C++ shared library approach to implement both operation semantics and framework mapping that refers to this custom operation.". By definition of Python custom operation you already have "loaded" it in runtime - just insert it into the graph and it will work. Am I missing something? |
||
|
||
Python can still be used to map and decompose operations when only operations from the standard OpenVINO operation set are used. | ||
Python can still be used to map and decompose operations when operations from the standard OpenVINO operation set and custom operation implemented in Python are used. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But only via There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes |
||
|
||
.. _create_a_library_with_extensions: | ||
|
||
|
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -7,20 +7,54 @@ Custom OpenVINO Operations | |||||
custom operations to support models with operations | ||||||
not supported by OpenVINO. | ||||||
|
||||||
OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box. This capability requires writing code in C++, so if you are using Python to develop your application you need to build a separate shared library implemented in C++ first and load it in Python using ``add_extension`` API. Please refer to :ref:`Create library with extensions <create_a_library_with_extensions>` for more details on library creation and usage. The remaining part of this document describes how to implement an operation class. | ||||||
OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box. A custom operation might be implemented both in C++ and Python (with some known limitations).``` | ||||||
|
||||||
For those who still wish to use C++, the capability to create a shared library with custom operation implementations is maintained. Please refer to :ref:`Create library with extensions <create_a_library_with_extensions>` for more details on library creation and usage. The remaining part of this document describes how to implement an operation class using both the C++ API and Python API. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. C++ extensions and libraries with such extensions are the first class citizen in OV, there shouldn't be any "still" or "is maintained", I think. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
Operation Class | ||||||
############### | ||||||
|
||||||
To add your custom operation, create a new class that extends ``ov::Op``, which is in turn derived from ``ov::Node``, the base class for all graph operations in OpenVINO™. To add ``ov::Op``, include the next file: | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.hpp | ||||||
:language: cpp | ||||||
:fragment: [op:common_include] | ||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.hpp | ||||||
:language: cpp | ||||||
:fragment: [op:common_include] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:common_include] | ||||||
|
||||||
Follow the steps below to add a custom operation: | ||||||
|
||||||
1. Add the ``OPENVINO_OP`` macro which defines a ``NodeTypeInfo`` object that identifies the type of the operation to the graph users and helps with dynamic type resolution. The type info of an operation currently consists of a string operation identifier and a string for operation version. | ||||||
1. Define the operation type information: | ||||||
|
||||||
a. For the C++ API, add the ``OPENVINO_OP`` macro which defines a ``NodeTypeInfo`` object that identifies the type of the operation to the graph users and helps with dynamic type resolution. The type info of an operation currently consists of a string operation identifier and a string for operation version. | ||||||
|
||||||
b. For the Python API, define the ``class_type_info`` attribute in your custom operation class. This attribute should be an instance of ``DiscreteTypeInfo``, encapsulating both the operation name, which serves as a unique string identifier, and the operation version or namespace, represented as a second string. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So should it be a class field and also a function or only one is enough? See another comment in the snippet. |
||||||
|
||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.hpp | ||||||
:language: cpp | ||||||
:fragment: [op:header] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:header] | ||||||
|
||||||
2. Implement default constructor and constructors that optionally take the operation inputs and attributes as parameters. | ||||||
|
||||||
|
@@ -43,43 +77,103 @@ OpenVINO™ operation contains two constructors: | |||||
* Default constructor, which enables you to create an operation without attributes | ||||||
* Constructor that creates and validates an operation with specified inputs and attributes | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:ctor] | ||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:ctor] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:ctor] | ||||||
|
||||||
``validate_and_infer_types()`` | ||||||
++++++++++++++++++++++++++++++ | ||||||
|
||||||
``ov::Node::validate_and_infer_types`` method validates operation attributes and calculates output shapes using attributes of the operation. | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:validate] | ||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:validate] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:validate] | ||||||
|
||||||
``clone_with_new_inputs()`` | ||||||
+++++++++++++++++++++++++++ | ||||||
|
||||||
``ov::Node::clone_with_new_inputs`` method creates a copy of the operation with new inputs. | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:copy] | ||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:copy] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:copy] | ||||||
|
||||||
``visit_attributes()`` | ||||||
++++++++++++++++++++++ | ||||||
|
||||||
``ov::Node::visit_attributes`` method enables you to visit all operation attributes. | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:visit_attributes] | ||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:visit_attributes] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:visit_attributes] | ||||||
|
||||||
``evaluate() and has_evaluate()`` | ||||||
+++++++++++++++++++++++++++++++++ | ||||||
|
||||||
``ov::Node::evaluate`` method enables you to apply constant folding to an operation. | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:evaluate] | ||||||
.. tab-set:: | ||||||
|
||||||
.. tab-item:: C++ | ||||||
:sync: cpp | ||||||
|
||||||
.. doxygensnippet:: src/core/template_extension/identity.cpp | ||||||
:language: cpp | ||||||
:fragment: [op:evaluate] | ||||||
|
||||||
.. tab-item:: Python | ||||||
:sync: py | ||||||
|
||||||
.. doxygensnippet:: docs/articles_en/assets/snippets/ov_custom_op.py | ||||||
:language: python | ||||||
:fragment: [op:evaluate] | ||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it required to name it
class_type_info
, or is it just an example?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I omit "extension" part?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just an example
No, right now DiscreteTypeInfo doesn't have a default value for the second
version_id
argument