Is the Nx Cloud up? Visit our Status Page for the current health and performance of the Nx Cloud.

Status Page

How run metavms with valgrind?

Answered
Permanently deleted user
Edited

Hi,

I would like to use valgrind to pin down possible memory leak in my plugin, how should I do it?

I tried to add "/usr/bin/valgrind --leak-check=yes" before "/opt/networkoptix-metavms/mediaserver/bin/mediaserver" and "/opt/networkoptix-metavms/mediaserver/bin/root-tool-bin", but it doesn't seem to work.

All I got when I stop the service is as follows:

Nov 17 22:17:03 admin1234-BOXER-6842M systemd[1]: Stopped Network Optix Media Server.
Nov 17 22:17:03 admin1234-BOXER-6842M systemd[1]: Started Network Optix Media Server.
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== Memcheck, a memory error detector
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== Command: /opt/networkoptix-metavms/mediaserver/bin/mediaserver
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597==
Nov 17 22:17:03 admin1234-BOXER-6842M systemd[1]: Stopping Network Optix Media Server...
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597==
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== Process terminating with default action of signal 15 (SIGTERM)
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== at 0x537BD5C: open (open64.c:47)
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== by 0x146F16: check_dev_tty (in /bin/bash)
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== by 0x137DFC: main (in /bin/bash)
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597==
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== HEAP SUMMARY:
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== in use at exit: 0 bytes in 0 blocks
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597==
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== All heap blocks were freed -- no leaks are possible
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597==
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== For counts of detected and suppressed errors, rerun with: -v
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11597]: ==11597== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Nov 17 22:17:03 admin1234-BOXER-6842M systemd[1]: Stopped Network Optix Media Server.
Nov 17 22:17:03 admin1234-BOXER-6842M systemd[1]: Started Network Optix Media Server.
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11638]: ==11638== Memcheck, a memory error detector
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11638]: ==11638== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11638]: ==11638== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11638]: ==11638== Command: /opt/networkoptix-metavms/mediaserver/bin/mediaserver
Nov 17 22:17:03 admin1234-BOXER-6842M valgrind[11638]: ==11638==
Nov 17 22:17:03 admin1234-BOXER-6842M systemd[1]: Stopping Network Optix Media Server...
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== Process terminating with default action of signal 15 (SIGTERM)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== at 0x537BD5C: open (open64.c:47)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== by 0x146F16: check_dev_tty (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== by 0x137DFC: main (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== HEAP SUMMARY:
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== in use at exit: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== total heap usage: 0 allocs, 0 frees, 0 bytes allocated
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== All heap blocks were freed -- no leaks are possible
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== For counts of detected and suppressed errors, rerun with: -v
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11638]: ==11638== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Nov 17 22:17:04 admin1234-BOXER-6842M systemd[1]: Stopped Network Optix Media Server.
Nov 17 22:17:04 admin1234-BOXER-6842M systemd[1]: Started Network Optix Media Server.
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11681== Memcheck, a memory error detector
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11681== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11681== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11681== Command: /opt/networkoptix-metavms/mediaserver/bin/mediaserver
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11681==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== HEAP SUMMARY:
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== in use at exit: 41,582 bytes in 563 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== total heap usage: 986 allocs, 423 frees, 82,551 bytes allocated
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== 236 bytes in 1 blocks are definitely lost in loss record 368 of 389
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== by 0x18F0DD: xmalloc (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== by 0x188B3A: set_default_locale (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== by 0x137E66: main (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== LEAK SUMMARY:
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== definitely lost: 236 bytes in 1 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== indirectly lost: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== possibly lost: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== still reachable: 41,346 bytes in 562 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== suppressed: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== Reachable blocks (those to which a pointer was found) are not shown.
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== For counts of detected and suppressed errors, rerun with: -v
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11710== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== HEAP SUMMARY:
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== in use at exit: 40,313 bytes in 525 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== total heap usage: 909 allocs, 384 frees, 80,099 bytes allocated
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== 236 bytes in 1 blocks are definitely lost in loss record 348 of 368
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== at 0x4C31B0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== by 0x18F0DD: xmalloc (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== by 0x188B3A: set_default_locale (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== by 0x137E66: main (in /bin/bash)
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== LEAK SUMMARY:
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== definitely lost: 236 bytes in 1 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== indirectly lost: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== possibly lost: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== still reachable: 40,077 bytes in 524 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== suppressed: 0 bytes in 0 blocks
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== Reachable blocks (those to which a pointer was found) are not shown.
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709==
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== For counts of detected and suppressed errors, rerun with: -v
Nov 17 22:17:04 admin1234-BOXER-6842M valgrind[11681]: ==11709== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== Conditional jump or move depends on uninitialised value(s)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== at 0x53FBDB7: __wcsnlen_avx2 (strlen-avx2.S:264)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x5329CA1: wcsrtombs (wcsrtombs.c:104)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x1A78D3: ??? (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x1A8540: glob_vector (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x1A8BCC: glob_filename (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x1781ED: shell_glob_filename (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x173563: ??? (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x14ECC2: execute_command_internal (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x14FBF1: execute_command (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x13A273: reader_loop (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681== by 0x138C7E: main (in /bin/bash)
Nov 17 22:17:05 admin1234-BOXER-6842M valgrind[11681]: ==11681==

The memory usage over last 6 hours shows a a trend going up without running any plugin, is this normal?

 

 

0

Comments

7 comments

Date Votes
  • Andrey Terentyev
    • Network Optix team

    Hello James Yu ,

    Could you please describe your set-up in grater detail? How many cameras? What is hi,lo stream quality? What is server configuration: OS, RAM, VMS build?

     

    0
  • Permanently deleted user
    Edited

    Hi Andrey,

    Setup is as follows:
    number of stream => 10
    stream quality => 1920x1080, 25fps, H264, Hi-Res
    OS => Ubuntu 18.04.6 LTS
    RAM => 32G
    VMS build => 4.2.0.32836 (both client & server)

    Following is the memory trend for last ~24hrs, "dummy plugin" indicates that DeviceAgent::pushUncompressedVideoFrame() returns immediately when invoked by NX server:

    0
  • Andrey Terentyev
    • Network Optix team

    Hello James,

    I would like to use valgrind to pin down possible memory leak in my plugin, how should I do it?

    1. Here is the pattern of the command to start the Server under valgrind

    sudo valgrind $ARGS $SERVER_BIN -e >$LOG_FILE.out 2>&1

    Pay your attention, you should pass the "-e" argument to the Server.

    2. As $ARGS your the following:

    for leak: "--leak-check=yes --show-leak-kinds=definite,possible --undef-value-errors=no $SUP"
    for massif: "--tool=massif --massif-out-file=$OUT.massif"

    0
  • Andrey Terentyev
    • Network Optix team

    James,

    Following is the memory trend for last ~24hrs, "dummy plugin" indicates that DeviceAgent::pushUncompressedVideoFrame() returns immediately when invoked by NX server:

    Could you share the source code of your plugin, please?

    The memory usage over last 6 hours shows a a trend going up without running any plugin, is this normal?

    It could be expected behavior. Memory utilization tends to stop at some value. It depends on the server configuration and is more about OS memory management.

    0
  • Permanently deleted user

    Hi Andrey,

    which email should I send the files to?

    0
  • Permanently deleted user

    Hi Andrey,

    I tried to start the Server under valgrind with the following command and failed (as root and networkoptix-metavms):

    valgrind --leak-check=yes --show-leak-kinds=definite,possible --undef-value-errors=no $SUP /opt/networkoptix-metavms/mediaserver/bin/mediaserver-bin -e >/tmp/valgrind.out 2>&1


    This is the error logged in /tmp/valgrind.out:

    ==32732== 
    ==32732== Warning: Can't execute setuid/setgid/setcap executable: /opt/networkoptix-metavms/mediaserver/bin/mediaserver-bin
    ==32732== Possible workaround: remove --trace-children=yes, if in effect
    ==32732== 
    valgrind: /opt/networkoptix-metavms/mediaserver/bin/mediaserver-bin: Permission denied
    0
  • Andrey Terentyev
    • Network Optix team
    Edited

    Hello James,

    There is a limitation intentionally set to avoid the server to be run under the valgrind.

    Here is a workaround

    sudo cp /opt/networkoptix-metavms/mediaserver/bin/mediaserver-bin /opt/networkoptix-metavms/mediaserver/bin/mediaserver-valgrind
    sudo chmod 755 /opt/networkoptix-metavms/mediaserver/bin/mediaserver-valgrind
    sudo valgrind --leak-check=yes --show-leak-kinds=definite,possible --undef-value-errors=no /opt/networkoptix-metavms/mediaserver/bin/mediaserver-valgrind -e >/tmp/valgrind.out 2>&1

     

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post