Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NPU] Use zero tensor to get correct data #27980

Merged
merged 9 commits into from
Jan 13, 2025
Merged

[NPU] Use zero tensor to get correct data #27980

merged 9 commits into from
Jan 13, 2025

Conversation

pereanub
Copy link
Contributor

@pereanub pereanub commented Dec 9, 2024 •
edited
Loading

Details:

  • Can not deallocate and re-allocate a newer memory for tensor if update mutable command list is not supported. Newer memory should be updated in the graph and this can be done only using the updating command list feature
  • In order to check if the memory was or wasn't re-allocated we are checking the unique ID provided by the driver when memory is created

Tickets:

  • E#134453

@github-actions github-actions bot added the category: NPU OpenVINO NPU plugin label Dec 9, 2024
@pereanub pereanub changed the title Use zero tensor to get correct data [NPU] Use zero tensor to get correct data Dec 9, 2024
@pereanub pereanub changed the title [NPU] Use zero tensor to get correct data [DO NOT MERGE][NPU] Use zero tensor to get correct data Dec 10, 2024
@pereanub pereanub marked this pull request as ready for review December 10, 2024 13:37
@pereanub pereanub requested review from a team as code owners December 10, 2024 13:37
@pereanub pereanub force-pushed the use_correct_tensor branch 11 times, most recently from c718ff5 to b1afd2c Compare December 16, 2024 14:55
@pereanub pereanub changed the title [DO NOT MERGE][NPU] Use zero tensor to get correct data [NPU] Use zero tensor to get correct data Dec 18, 2024
razvanapetroaie
razvanapetroaie reviewed Dec 18, 2024
View reviewed changes
Copy link
Contributor

@razvanapetroaie razvanapetroaie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Halfway through, nothing major so far.

src/plugins/intel_npu/src/backend/include/zero_infer_request.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/include/zero_tensor.hpp Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_infer_request.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/common/include/intel_npu/common/sync_infer_request.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_tensor.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_tensor.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_infer_request.cpp Outdated Show resolved Hide resolved
razvanapetroaie
razvanapetroaie reviewed Dec 20, 2024
View reviewed changes
Copy link
Contributor

@razvanapetroaie razvanapetroaie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MirceaDan99 Please let us know if you have anything before merging this.

src/plugins/intel_npu/src/backend/src/zero_infer_request.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_infer_request.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_infer_request.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_infer_request.cpp Outdated Show resolved Hide resolved
@pereanub pereanub force-pushed the use_correct_tensor branch 6 times, most recently from f505588 to 1b6d092 Compare January 8, 2025 10:25
@pereanub pereanub force-pushed the use_correct_tensor branch 4 times, most recently from bd6c07b to e83292e Compare January 9, 2025 09:39
razvanapetroaie
razvanapetroaie approved these changes Jan 9, 2025
View reviewed changes
src/plugins/intel_npu/src/backend/include/zero_pipeline.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/include/zero_tensor.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/include/zero_tensor.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/include/zero_tensor.hpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/backend/src/zero_pipeline.cpp Outdated Show resolved Hide resolved
src/plugins/intel_npu/src/common/src/remote_tensor.cpp Outdated Show resolved Hide resolved
@pereanub
Use zero tensor to get correct data
2f35e58 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Update command list if someone else allocated another tensor
95109d5 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Create zero tensor class
1c837ea 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Clean code
24816d3 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Do not set a bigger shape in case of remote tensor
1b0a335 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Add method for checking memory address changes in the zero tensor
4786d62 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Create is_remote_tensor helper
f8b0098 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Command list shall be closed only when all the args are set to a vali…
6e1d738 
…d address

Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub
Update code according to the review
3a38ec7 
Signed-off-by: Bogdan Pereanu <[email protected]>
@pereanub pereanub added this pull request to the merge queue Jan 13, 2025
Merged via the queue into openvinotoolkit:master with commit 2817977 Jan 13, 2025
157 checks passed
@pereanub pereanub deleted the use_correct_tensor branch January 13, 2025 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@razvanapetroaie razvanapetroaie razvanapetroaie approved these changes

@PatrikStepan PatrikStepan PatrikStepan approved these changes

Assignees

@MirceaDan99 MirceaDan99

@razvanapetroaie razvanapetroaie

Labels
category: NPU OpenVINO NPU plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pereanub @PatrikStepan @razvanapetroaie @MirceaDan99