Testcamera can not push frame to plugin in Nx 5.0

Answered

Comments

12 comments

  • Avatar
    Jacky

    More detail:

    Testcamera binary I use during this test is the one bundled with nx server.
    Therefore, I suppose the version of testcamera binary is not a problem in my test.

    0
    Comment actions Permalink
  • Avatar
    Jacky

    Furthermore, I also try the previous 5.0 (Linux build 5.0.0.35064) release build from below link.
    https://updates.networkoptix.com/default/

    Both issues (1. Streaming from testcamera does not call pushUncompressedVideoFrame; 2. Assertion message from real camera streaming) exist.

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello,

    I've just rechecked the testcamera with our opencv object detection plugin. It uses pushUncompressedVideoFrame fot precessint frames.

    Here is the video, I tested with 

    http://resources.vmsproxy.com/video_analytics_howto/kitten.mp4

    I guess, the issue is with the video you're using.

    I don't t know if the video I mentioned would suit your case, but could you test your plugin with it?

    >>> ASSERTION FAILED: /home/jenkins/vms.vms_5.0.linux-x64-2/nx_vms/vms/server/nx_vms_server/src/nx/vms/server/analytics/uncompressed_video_frame.cpp:209 (m_avFrame->data[plane]) Null data for plane 0

    Please recheck your plugin requests the desired frame format from the Server.

    0
    Comment actions Permalink
  • Avatar
    Jacky

    Hi Andrey,
    Thanks for your response. 

    The video I tested works perfectly on Nx server 4.2, so I think the video itself might not be an issue.
    But I still give a try to use the kitten.mp4 as the video source, and switch on the enableOutput flag to show if the doPushDataPacket() is called, but unfortunately, I still got the same result (no function call to push frame to our plugin).

     

    For the second issue (assertion message in the real camera stream), I always request the yuv420 format for the uncompressedVideoFrame in the manifest. Actually, as I mentioned in the post, I could get the video frame correctly, but only in the first few minutes (3-5 minutes). If the provided frame format is wrong, I suppose I won't get any correct video frame during the test.
    Here is the response in the engineManifest parsed from /ec2/getFullInfo response. ( the unrelated deviceAgentSettingsModel content is removed in order to focus on the other field.)

    {
    "name": "engineManifest",
    "resourceId": "{b3757aa3-c60a-e80e-567a-f07c42f5df5f}",
    "value": "{\"capabilities\":\"needUncompressedVideoFrames_yuv420\",\"deviceAgentSettingsModel\":{/* ignored * /},\"preferredStream\":\"secondary\",\"streamTypeFilter\":\"uncompressedVideo|metadata|motion\",\"typeLibrary\":{}}"
    }
     
    I also try two different real cameras (the brand is also different), both camera will produce the same assertion message as I post.

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello Jacky,

    As per the issue

    But in Nx version 5.0, our plugin could only receive video frame from real camera (but could only first few minutes, see another issue below). It seems that Nx server does not call pushUncompressedVideoFrame if the stream source is testcamera.

    Please share the following info.

    1. A link to the video file you're using. Just upload the file to any cloud and share the link in a post.

    2. The command you are executing the testcamera with.

    Please, proceed with following actions and share result with us.

    1. Use sample_analytics_pluing of the Metadata SDK with your video file and the testcamera. Does the issue persist? Do you still see assertion messages in the log file?

    I also try two different real cameras (the brand is also different), both camera will produce the same assertion message as I post.

    2. Use sample_analytics_pluing of the Metadata SDK with the cameras in question. Does the issue persist? Do you still see assertion messages in the log file?

    0
    Comment actions Permalink
  • Avatar
    Jacky

    Hi Andrey,
    Thanks for your advice.

    For the testcamera issue, I realize the problem is about the preferred stream.
    After checking the command I used to execute the testcamera, I realize that I have an --no-secondary option in the command but set preferredStream as 'secondary' in the engineManifest.

    It seems that Nx 5.0 fix a bug in the preferred stream. In plugin setting page, user could set the preferred stream, and the default value could be assigned through the engineManifest. In Nx 4.2, the value shown here will always be primary no matter which value is assigned in the manifest (That's why the command I use the testcamera will work in Nx 4.2 !). Therefore, after setting the preferred stream, our plugin could receive video frame as before.

     

    For the second issue, I try the sample_analytics_plugin bundled in the plugins_optional folder, and no assertion message found in the log. 
    So I test some different plugins with Nx server 5.0 in this issue.
    (Yes means assertion message found in 10 minutes)
    0. our plugin -> Yes.
    1. sample_analytics_plugin (build in SDK 4.2) -> No.
    2. stub_analytics_plugin (build in SDK 4.2)  -> No.
    3. stub_analytics_plugin, but request yuv420 in the ini file (build in SDK 4.2) -> Yes.
    4. sample_analytics_plugin (bundled in Nx 5.0 server) -> No.
    5. stub_analytics_plugin_video_frames -> No.
    6. stub_analytics_plugin_video_frames, but request yuv420 in the ini file -> No.

    It seems that there are some breaking changes in the uncompressed video related function from Nx server 4.2 to 5.0, which I don't expect to happen. 

    A properly written Plugin is expected to remain compatible with any future version of the VMS,
    including minor updates like monthly patches.
     
    As described in the dynamic_libraries.md in the SDK, I suppose we could use the same plugin built with Nx 4.2 SDK in any future version of Nx server. Is that still true? Or could you share what is a better approach to support both Nx 4.2 and Nx 5.0 (and for future version) in plugin development?
    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hi,

    It seems that there are some breaking changes in the uncompressed video related function from Nx server 4.2 to 5.0, which I don't expect to happen. 

    It's a wrong conclusion. I asked you to test with sample_analytics_plugin just to demonstrate that the plugin is working not been changed. There is something in your plugin peculiarities. You can see either the sample analytics plugin or stub analytics plugin source code for reference. That's the second reason for referring to sample plugin.

    I suppose we could use the same plugin built with Nx 4.2 SDK in any future version of Nx server. Is that still true? 

    That's true as long as your code corresponds to the plugin interface specification, i.e. SDK. I hope you understand, that it does not comprise plugin internal errors, bugs, peculiarities etc.

    3. stub_analytics_plugin, but request yuv420 in the ini file (build in SDK 4.2) -> Yes.

    For that, could you please provide step-by-step scenario of what you were doing? What was expected result? What was the actual result? Please enable Server debug logging, redo the test and share afterward the log directory of the Server.

    0
    Comment actions Permalink
  • Avatar
    Jacky

    Hi Andrey,
    Here are my step to check the different plugins in Nx 5.0.
    0. Stop Nx server with "systemctl stop networkoptix-mediaserver".
    1. Set server loglevel to DEBUG2 in /opt/networkoptix/mediaserver/etc/mediaserver.conf.
    2. Put the target plugin in /opt/networkoptix/mediaserver/bin/plugins/, makes sure the permission of the plugin file is sufficient for Nx server to load. (with chmod a+rx command)
    3. touch /home/networkoptix/.config/nx_ini/stub_analytics_plugin.ini, assign the following parameters:
         enableOutput=true
         needUncompressedVideoFrames="yuv420"
    4. Start Nx server with "systemctl start networkoptix-mediaserver".
    5. Enable stub_analytics_plugin on one camera.
    6. Wait and tail the mediaserver_stderr.log to check is there any assertion message.

    The plugin I used in the test, logs could be found in the below link.
    https://drive.google.com/drive/folders/1Ed-hTu5cAnxYbtmXoJKrZWfd0oB18LQG?usp=share_link
    The folder structure is shown as below:

    stub_analytics_plugin_log.tar -------
                                                  |____ libstub_analytics_plugin.so

                                                  |____ log/ (from /opt/networkoptix/mediaserver/var/)

                                                  |____ mediaserver_stderr.log (from /home/networkoptix/.config/nx_ini/)

    0
    Comment actions Permalink
  • Avatar
    Jacky

    Hi, Is there any update?

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello,

    According to the mediaserver_stderr.log the Server was started at 4 2022-11-23 18:00:29. The first assert messages appeared at 2022-11-23 18:00:46, i.e. almost instantly after the start.

    I used the plugin binary you've provided, but could not reproduce the issue. No assert messages at least during 20 min. I tested with built-in web camera.

    • Nx: 5.0.0.35745.
    • OS: ubuntu 20.04.5
    • glibc version 2.31.

     

    0
    Comment actions Permalink
  • Avatar
    Jacky

    Yes, that is also what I observed. And this issue could be reproduced even in different physical hosts. 

    Because this phenomenon is not found in Nx server 4.2, please help to check what is the possible cause. 

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hi,

    please help to check what is the possible cause.

    I'd like to, but I can't reproduce the issue, therefore don't have a subject to investigate.

    0
    Comment actions Permalink

Please sign in to leave a comment.