Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Intel NPU utilization #985

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

m-falkowski
Copy link

Hello,

This commit adds proof of concept support for the Intel NPU inference accelerator 123
that is present in Meteor and Lunar Lake CPUs.

I emphasize that this is a PoC implementation to show that NPU device implementation may reuse GPU codebase.
These two are asynchronous coprocessors that expose similar statistics (utilization, VRAM/RAM memory, clocks, power stats).

For now, the NPU device is added into vector and based on is_npu_device info to generate distinct strings for GPU/NPU at the drawing stage.

To make a complete implementation there are a couple of ways I see and I am not sure which is appropriate:

  1. Continue the current way but filter NPU devices from gpu-average etc.
  2. Duplicate or maybe alias gpu_info into npu_info, create separate container for vector<npu_info> and reuse drawing functions,
    keeping is_npu_device in supported_functions to get the device string in a drawing.
  3. Duplicate fully gpu_info and drawing utilities.

I think that the second option seems to be the most optimal and I would ask for some guidance.

Footnotes

  1. NPU explained: https://intel.github.io/intel-npu-acceleration-library/npu.html

  2. Linux driver sources path: root/drivers/accel/ivpu/

  3. NPU UMD sources: https://github.com/intel/linux-npu-driver

This commit adds monitoring of utilization and RAM usage of
NPU devices that are present in Meteor Lake and Lunar Lake CPUs.

Linux driver sources path: root/drivers/accel/ivpu/
UMD sources: https://github.com/intel/linux-npu-driver
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant