How to process Images with tensorflow using DLPU in Artpec8 Axis camera

[rafiomeon]

Hi,

Currently we have a video analytics ACAP application running within Axis Q1615 camera. In this application we pull an internal H.264 stream and do significant amount of preprocessing of the stream using OpenCV. Following this preprocess step, we send a handful of selected images to the TPU chip within Q1615 to perform object classification. We want to port this application to Artpec8 environment. We want to keep our preprocess flow exactly the same - just send the same handful of selected images to Artpec 8 DLPU instead of the TPU chip in Q1615. Can you tell us as to how we invoke the just the DLPU from our existing application to do this.

Thanks
Rafi

[Corallo]

Hello @rafiomeon

Are you using the native sdk or the computer vision sdk?
Assuming you are using the native sdk, to switch from tpu chip and artpec8 chip you need to provide the chip "12" to the setuplarod function.
You can find an example here:
https://github.com/AxisCommunications/acap-native-sdk-examples/blob/main/object-detection/app/object_detection.c#L459
arg.chip comes from the -c parameter that you can see here:
https://github.com/AxisCommunications/acap-native-sdk-examples/blob/main/object-detection/app/manifest.json.artpec8

On an extra note, to run your model on artpec8, it has to be in tflite format, not anymore in edgetpu.tflite.

[rafiomeon]

Hi @Corallo

Thank you responding to our question. However, your response does not address our requirement. We need an direct API for calling the Artpec 8 DLPU (P32687-LV) Inference Engine from within our Application. We just want to swap out our current call for the TPU chip (in Q615) and replace it with the a call to the Artpec 8 DLPU - without rewriting our Application to fit the LAROD framework. We will need directions on how to call the Artpec 8 DLPU libraries directly. This way we will be able run our Application both in Q1615 (invoking TPU libraries) and P3267-LV (invoking DLPU libraries) with minimal changes. Appreciate your help on this.

[Corallo]

I see what you mean.
Although this is theoretically doable, unfortunately we don't support accessing directly the DLPU API.
Among other things, larod take care of the power management of the chip for the inference, which for A8 is more critical.
The only way we provide support for running inference in Artpec8 is via larod.

[rafiomeon]

Hi @Corallo

Its a set back to us if we cant access directly the DLPU API. To resolve this issue we are planning a different approach. We have a pre-processed Image data in our application and want to inference in Object detection using Larod.

The Axis example vdo-larod shows how inference can be done on "video stream" whereas we wanted to do inference based on an Image data. The closest example we could find was larod but this example does only classification using BMP and Not doing Inference of object detection. Could you provide a sample to do Inference using DLPU with "Image data" in object detection which provides location, class and number of detections.

Thanks,
Rafi

[Corallo]

Hi @rafiomeon

We don't have an example that shows exactly what you are trying to do, but we have an object-detector-example that combined with the larod example that you mentioned can achieve what you want to do.
Larod will give to the model whatever you write in the input tensor (larod_input_addr), it doesn't matter if it is an image that you read from a file or a frame captured by the camera.
Then it can also run the model, regardless if it is an image classification like in the case of vdo-larod, or an object detector. The important part is that you allocate the output tensor, and you just read the output from them, as showed both in the vdo-larod example and in the object detector example L652 .