Darknet model integration

Answered

Comments

26 comments

  • Avatar
    Andrey Terentyev

    Hello Riles Ouali,

    Here is the post worth reading on how to organize third library binaries in a plugin.

    https://support.networkoptix.com/hc/en-us/community/posts/360045632473/comments/360011473654

    Do you link you daknet library statically or dynamically? Where the .dll file of darknet is located in your system?

    Could you try to do the following:

    1. configure your IDE to run Server under debugger. Here is an example for Visual Studio

    2. Setup a break point in the plugin code somewhere in Engine::~Engine() or new the line where you call darknet function for the first time.

    3. Build the plugin and copy to server's plugin directory.

    4. Run the Server under debugger in your IDE. The execution should stop at the breakpoint.

    5. Step further in debugger and find out the line at which code fails.

    6. Share with us the code of the function this line belongs to.

    TTPLUGINLIBS

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    Hello Andrey Terentyev

    _ I organized my third library binaries the same way as in this description, but still nothing appears in panel

    _ Do you link you daknet library statically or dynamically? I tried both: statically and dynamically!

    _ Where the .dll file of darknet is located in your system? In the same plug-in directory (the same folder as the plug-in .dll)

    _ Could you try to do the following:

    1. Configure your IDE to run Server under debugger.
    2. Setup a break point in the plugin code somewhere in Engine::~Engine() or new the line where you call darknet function for the first time.
    3. Build the plugin and copy to server's plugin directory.

    I followed this threes instruction, but I couldn’t run the Server under debugger in my IDE. Could you please help me to do it?

    _ Share with us the code of the function this line belongs to. I sent you the code by email! Could you please let me know if you received it?

    Regards!

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello Riles Ouali,

    Yes, I've got your code. Please, allow me some time for investigation.

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello,

    I've got your code archive and had a look into it.

    Thought I got the code compiled successfully a plugin is not loaded with an error

    2020-06-10 14:59:44.539 2c88 ERROR PluginManager(0x2b777e62340): Failed loading Server Plugin [C:/Program Files/Network Optix/Nx MetaVMS/MediaServer/plugins/stub_analytics_plugin.dll]: [cannotLoadLibrary]: Cannot load library C:\Program Files\Network Optix\Nx MetaVMS\MediaServer\plugins\stub_analytics_plugin.dll: The specified module could not be found.

    The resulting dll does not export nxCreatePlugin function, which is to be invoked by the Server to initialize plugin.

    You don't use Plugin helper class in your code normally exporting this function neither you implement and export nxCreatePlugin explicitly.

    Please, see plugin.cpp and plugin.h in stub_analytics_plugin example.

    Please, make sure you are using the latest Metadata SDK available here https://nxvms.com/downloads/patches

     

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    The good practice in developing a plugin is start with an example plugin. stub_analytics_plugin for example.

    1. Build it with the build_samples.bat scripts provided with the SDK. Place it to the server and check if the server loads it successfully.

    2. Configure your IDE and try to build the same original plugin from IDE. Place it to the server and check if the server loads it successfully.

    3. Change some settings values in plugin or engine or device gent manifests. Build a plugin, place it to the server and check if the server loads it successfully.

    4. Make some minor changes to the code. For example, include (just include without invoking functions) your darknet library. Build a plugin, make sure your external library is linked. Place a plugin to the server and check if the server loads it successfully.

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    _ The resulting dll does not export nxCreatePlugin function, which is to be invoked by the Server to initialize plugin.

    • How can I include or create nxCreatePlugin?

    _ You don't have plugin helper class implemented in your code normally exporting this function.

    • I used the same helper class that is provided by the “Media SDK”. Could you please provide me an example of its implementation or guide me to implement my own version?

    _ Please, see plugin.cpp and plugin.h in stub_analytics_plugin example.

    • I did! But it sounds, somehow, can’t figure it out!

    _ Please, make sure you are using the latest Metadata SDK available here

    • The code I sent you is the latest “Metadata SDK” that I found of your website on April first! The only thing that I changed, is including some libraries in the project configuration and creates a static AI_Detector in device_agent.h and a function that invoking that detector “processFrame” used one in pushUncompressedVideoFrame. That’s it!
    • If the function “processFrame” is commented, the plugin’ll appear once again in the client panel!
    • I tried to download the latest “Metadata SDK” with the link you sent me but it’s not working https://nxvms.com/downloads/patches (page not found)

     

    _ Concerning the practice of creating a plug-in, it’s exactly what I did!

    _ Could-you please provide us with more details about debugging the media_SDK when invoking as you suggested it in the following link? : https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011516774

     

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Riles Ouali ,

    1. You should create an account and sign up for Meta VMS. There is a pinned thread "Getting started" on the top of "Developer forum"

    2. log in to meta cloud and download the latest (at the moment 4.1.0.31036 R7) Metadata SDK from here https://meta.nxvms.com/downloads/patches

    Sorry, I provided wrong link in my the last post.

    You would probably want to download the latest MetaVMS build to develop your plugin for.

    3. In Metadata SDK you'll find the "samples" folder which contains two example plugins.

    • sample_analytics_plugin
    • stub_analytics_plugin

    You could use any of them to rebase your code into.

    Please, let me know when you are done with rebasing.

    Please, share your code again if you did not get your plugin working after rebase.

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    _ Could-you please provide us with more details about debugging the media_SDK when invoking as you suggested it in the following link? : https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011516774

    Could you please specify what exactly remains unclear? I would focus on this.

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    Andrey Terentyev

    I created and activated my account successfully. Then, when I tried to download the patches from  https://meta.nxvms.com/downloads/patches I got the same error:

    0
    Comment actions Permalink
  • 0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    _ The latest version I found is (4.0.0.29991 R17)

     

    _ Just subscribed: https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011784293

    _ I would like to step trough the plugin, in debug mode, when developing it to identify and figure out any problem when integrating a third party in meta_SDKS: anwer to https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011784013

     

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    You see, there is "Download Nx Meta VMP" link  at the bottom of the screen.

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    _ I would like to step trough the plugin, in debug mode, when developing it to identify and figure out any problem when integrating a third party in meta_SDKS: anwer to https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011784013

    I don't know which IDE are you using. And frankly speaking this kind of questions is out of scope of Metadata SDK.

    In Visual Studio is just like this.

    Here is an article for Visual Studio for more details on navigation.

    https://docs.microsoft.com/en-us/visualstudio/debugger/navigating-through-code-with-the-debugger?view=vs-2019

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    _ I'm using visual studio 2017!
    _ I followed the instructions as you provided here, and automated the procedure of transferring the plugin with a bat file

    net stop "Network Optix Media Server"

    set CURRENT_DIR=%~dp0

    move %CURRENT_DIR%stub_analytics_plugin.dll "C:\Program Files\Network Optix\Nx Witness\MediaServer\plugins"n

    But when I hit F5 I got this following message

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hi Riles Ouali ,

    It seems VS is trying to start program with the path "F:\Plug-ins... etc"

    Which leads to conclusion

    _ I followed the instructions as you provided here

    seems to be wrong statement.

    Please, once gain look carefully at the screenshot in this post

    https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011516774

    In VS right click on "stub_analytics_plugin" project. Select "Properties".

    In the left part of the opened window select "Debugging" and fill in "Command" and "Command Arguments" as shown at the screenshot.

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali
    Hi Andrey,
     
    _ It's been about three days that we trying to solve the problem in integration, by integrating our code step by step, as you suggested!
     
    _ Hence, we noticed that the Plug-in disappears from the client, as son as the AI_Detector is declared in the code!
     
    _ However We couldn't step trough the debugging process to identify the problem. Hence, we're pretty sure that the problem does not from the library cause we already test it in other projects!
     
    Regards
     
     
    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello,

    I created a ticket. Please check your mail box for messages.

    On resolution, the explanation will be put hereafter in a post.

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Resume.

    The instruction given on configuring Visual Studio to run server under debugger were correct.

    Several corrections for initializing third party Detector class.

    1. Declare a member inside DeviceAgent class in device_agent.h

    2. Use correct syntax of the constructor of this class.

    The resulting correct code is

    Detector m_Detector = Detector(config_path, path);

    Project compiled with no error.

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    Hi Andrey,

    _ Thanks you so much for your help!

    _ We succeeded to compile and generate the Plugin. Hence, the Plugin still does not appear in the panel. Do you have an idea on how to fix this issue?

    _ Once again, thanks a lot!

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hi Riles Ouali ,

    I'd really like to help you, but there is no enough data from your end.

    > Do you have an idea on how to fix this issue?

    Please, try to put questions specific, otherwise there is absolutely no possibility to determine what is the issue.

    The algorithm of description is similar to this
    1. I do these steps, putting this code in the file1, file2 etc.

    2. I expect to have <something to happen>

    3. Actual result is <something has happened>

    Here is the post with detailed instructions. Please, study it through.
    https://support.networkoptix.com/hc/en-us/community/posts/360039290494-How-to-submit-a-support-request-

     

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Regarding technical aspects of your issue.

    As I can judge after the remote session, there is no technical issue related to Metadata SDK itself.

    I'm really sorry, but there is no magical pill making your plugin to have third part library built in.

    You should work though SDK documentation, figure out how helper classes Plugin, Engine, DeviceAgent are organized.
    After that carefully put your code INSIDE one of these classes.

    Now you are able to set breakpoints inside the plugin code, run the mediaserver under debugger in Visual Studio and step trough the plugin code in order to determine a place causing the issue.

    Here are several more guidelines on plugin appearance.

    There are two places where plugin information appears.

    1. "System Administration..."->"Plugins". Here information appears if
      • the Plugin helper class instance was successfully created,
      • Plugin::manifestString() returns a  engineSettingsModel json object,
      • the Engine helper class instance was successfully created.
    2. "Camera settings"->"Plugins". Here information appears if
      • Engine::manifestString() returns a deviceAgentSettingsModel json object,

    Here is a sequence of methods called by the Server when plugin is being loaded.

    1. The Server loads plugin dynamical library and invokes its entry point function createNxPlugin (see i_plugin.h), which returns a pointer to the nx::sdk::IPlugin interface class instance.
    2. The Server invokes  IPlugin::createEngine using the pointer. createEngine calls the  doCreateEngine virtualmethod of the interface class, which in turn  - doObtainEngine. The IPlugin::createEngine  returns a pointer to nx::sdk::analytics::IEngine interface class instance. The  doObtainEngine is overridden in the Plugin helper class.
    3. The Server invokes IEngine::obtainDeviceAgent using the pointer.  obtainDeviceAgent calls  the  doObtainDeviceAgent virtual method of the interface class. The IEngine::obtainDeviceAgent returns a pointer to the nx::sdk::analytics::IDeviceAgent interface class instance. The  doObtainDeviceAgent is overridden in the Engine helper class.
    4. The Server invokes pushDataPacket method on retrieval of every packet. It is the general method for all types of plugins (analytics, video source, storage). It’ is the only method the Server knows. In case of Metadata SDK, pushDataPacket in turn invokes the doPushDataPacket virtual method of the interface class (see src/nx/sdk/analytics/i_consuming_device_agent.h). Network Optix has prepared the ConsumingDeviceAgent helper class (see src/nx/sdk/analytics/helpers/consuming_device_agent.cpp), which besides the reimplementation of doPushDataPacket offers several other methods in order to make plugin development more convenient and efficient:  pushCompressedVideoFrame, pushUncompressedVideoFrame, pullMetadataPackets, pushMetadataPacket etc.

    In general case, the helper classed Plugin, Engine, DeviceAgent are recommended to be used.
    You could set breakpoints in the indicated methods as well as in the constructors of the helper classes Plugin, Engine, DeviceAgent to determine what is going wrong, what fails and restricts a plugin information to appear in the GUI.

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    Here the algorithm that describes the issue:


            1. I create this object, “Detector m_Detector = Detector(config_path, path)” in “DeviceAgent”. Then I compile the plugin without any problem. I copy the DLL in plugin folder "C:\Program Files\Network Optix\Nx Witness\MediaServer\plugins" and run VS in debug mode with several breakpoints;

    1. I expect to that VS will stop at the breakpoints and the plugin will appear in the panel of my client;
    2. Actual result is :

                a_ VS does not stop at the breakpoints (debug mode);

                b_ and the plugin doesn’t appear in the panel

     

    These two notifications I already reported them in my previous posts:

    a_ https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011751293

    b_ https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011558014

     

    Regards,

     

    0
    Comment actions Permalink
  • Avatar
    Riles Ouali

    Hello Andry

    We sent you, by e-mail, a VS ".sln" with adequate configurations. We include the lib and the configuration files of the AI detector.

    Regards

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev
    Guys,
     
    Could you please, kindly respond via ticket system? It is very difficult to follow several communication channel.
    I just lost your email.
    Please, duplicate your last response in the ticket.
     
    Thank you for understanding.
    0
    Comment actions Permalink
  • Avatar
    Riles Ouali
    _ Where can we find ticket system? Could you please provide us with a link to the "ticket system"?
     

     

     

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello,

    Here is the result of our findings.

    The codeTestProject you've sent besides .lib files in the codeTestProject\darknet folder uses pthreadCV2.dll library, located in codeTestProject\x64\Debug and codeTestProject\x64\Release directories.
    These directories are actually Output Directories, configured in your project. (See a screenshot)

    Where
    $(SolutionDir) is codeTestProject
    $(Platform) is x64
    $(Configuration) is Debug or Release
    That means the resulting codeTestProject.exe is placed in these directories during the build process and when is being executed, finds a pthreadVC2.dll nearby and successfully loads it.

    Please, make an experiment.
    1. Open the codeTestProject project in the Visual Studio.
    2. Move pthreadVC2.dll from codeTestProject\x64\Debug and codeTestProject\x64\Release
    3. Build the project.
    4. Press F5.
    You'll get an error.

    This error is the root cause of why your plugin is not loaded and does not appear in the GUI. Plugin's dll just can't find and load the pthreadVC2.dll.
    This failure is detectable by PluginManager message in the server log file

    2020-07-13 20:01:57.574   1fc0    INFO PluginManager(0x210a35403b0): Considering to load Server plugin [C:/Program Files/Network Optix/Nx MetaVMS/MediaServer/plugins/pthreadVC2.dll]
    2020-07-13 20:01:57.602  1fc0  ERROR PluginManager(0x210a35403b0): Failed loading Server Plugin [C:/Program Files/Network Optix/Nx MetaVMS/MediaServer/plugins/pthreadVC2.dll]: [invalidLibrary]: No entry point function createNxPlugin() or old SDK createNXPluginInstance()
    2020-07-13 20:01:57.603  1fc0    INFO PluginManager(0x210a35403b0): Considering to load Server plugin [C:/Program Files/Network Optix/Nx MetaVMS/MediaServer/plugins/sample_analytics_plugin.dll]
    2020-07-13 20:01:57.638  1fc0  ERROR PluginManager(0x210a35403b0): Failed loading Server Plugin [C:/Program Files/Network Optix/Nx MetaVMS/MediaServer/plugins/sample_analytics_plugin.dll]: [cannotLoadLibrary]: Cannot load library C:\Program Files\Network Optix\Nx MetaVMS\MediaServer\plugins\sample_analytics_plugin.dll: The specified module could not be found.

    So, please, organize your plugin binaries as was advised above in this post https://support.networkoptix.com/hc/en-us/community/posts/360046685314/comments/360011516774, i.e. copy your plugin .dll and pthreadVC2.dll to one folder.

    0
    Comment actions Permalink

Please sign in to leave a comment.