How To Write Back NX AI Manager Custom Postprocessor Output on the Device View
In Progress
Greetings,
Thank you for the great open NX Meta AI Manager, we are so thrilled to integrate our AI using the NX AI Manager.
We managed to create a crowd counting postprocessor on top of the NX AI demo/default 80-Classes Object Detector [320x320] as shown in the first image, by following this guide https://github.com/scailable/sclbl-integration-sdk
Quick overview on how we did it is that: We have our crowd counting AI model as a separate inference server in a form of API. Then on our custom postprocessor, we get the image from SHM and call our crowd counting inference API with the image as the payload. Then we reformat the result exactly as the scailable example here https://github.com/scailable/sclbl-integration-sdk/tree/main/postprocessor-python-example#messagepack-schema
We can provide the snippet of our postprocessor code and the returned result to the communication socket here.
https://drive.google.com/drive/folders/1nTnoQtwbW82T9BvhimMf9_alNqartpMN?usp=sharing
The question are:
- How can I display the object count on the device view? in my output I believe I have put the correct value for the object counts
Even when I try to activate the built-in object count postprocessor, I don't see the object count anywhere on the view
- When I have 2 camera devices, both activate my custom crowd count postprocessor at the same time, only one device can work with the postprocessor, but when one of the device run alone it works well. (Yes I have tested 4 parallel process call on my API and my API supports parallel calls)
Environment:
Python 3.8 (in building the postprocessor)
Nx Meta 6.0.0.39503
Nx Client 6.0.0.39503
-
Hi Benedictus Visto Kartika,
Thanks for your update and question. At the moment, the post-processor has not yet been able to generate/fire the event directly. We do have the plan for this, but there is no solid ETA yet, might be after a few major releases.
For the object count result, it is literally an event you can set in event-rules engine, then you would be able to see the sample result accordingly. (Ex: text overlay on the live video)
The current result is likely you may need the to utilize the generic event feature, which may be able to trigger Nx Generic event in post-processor, then use the Generic event as the trigger to fire an event.
-
continuing question number 2,
"When I have 2 camera devices, both activate my custom crowd count postprocessor at the same time, only one device can work with the postprocessor, but when one of the device run alone it works well. (Yes I have tested 4 parallel process call on my API and my API supports parallel calls)"
the box only showed up if there is only one video that I view, also the result is not showing up on second stream
above
I used 2 RTSP source that has same video file for face recognition
Video Demo 2 inference box not showing
https://drive.google.com/file/d/1r0Haqs3sFPuhyUa9ap1FYh3nv794l4dY/view?usp=sharing
Video Demo 2 inference result not showinghttps://drive.google.com/file/d/1myWvcWVe3SzySxTush_nDJYyXZGuqxBp/view?usp=sharing
-
Hi Erwin Yonata,
Thank you for your questions. Unfortunately, I haven’t been able to reproduce the issue using our sample and Nx AI Manager plugins.
However, I did notice one potential issue in your video: recording has not been enabled, which is essential for the plugin to function continuously. You can see a notice about this in your video.
If recording is not enabled, the plugin will only work when the camera is actively being viewed (in view, selected).
In the video, I noticed that you have one camera selected for display continuously. While I’m not certain if this is intentional, you might consider clearing the filter to allow the desktop client to display “any camera.”
At this point, I’m not entirely clear on the specific issue you’re experiencing. There are several conditions that may not be fully met, and based on the operations shown in the video, the behavior appears to be expected.
The first suggestion here is to enable the recording on the device that you would like to apply Nx AI Manager plugin, the footage is not necessary to be kept, but recording has to be enabled.
Hope this helps.
Thanks.
Please sign in to leave a comment.
Comments
4 comments