Is the Nx Cloud up? Visit our Status Page for the current health and performance of the Nx Cloud.

Status Page

FAILED Camera request and VMS killed

Answered
Yoshi Sei
Edited

Hi Nx Support,

In one of our environments , We found the unexplained phenomenon. We conldn't have identified which the camera or Nx server has issues in this phenomenon. If you have any similar experince and any comments, please let us share. 

 1. Suddenly it was not able to access one camera from Nx server ,and  Nx server showed the following camera diagnostics messge, and then this camera stream hadn't been recorded, 

In additional comment, We tried to reset power of this camera after that, However this camera hasn't  been working yet. (not recovered)

2. After approximately 50 sec since this camera stream hadn't been recorded, Nx Wintess server restarted while outputting the following syslog & nx debug log & gdb-bt.

  • syslog
2022-04-06T07:22:05.963179+09:00 localhost <daemon.warning> systemd[1]: networkoptix-mediaserver.service: Main process exited, code=killed, status=11/SEGV
2022-04-06T07:22:05.966482+09:00 localhost <daemon.warning> systemd[1]: networkoptix-mediaserver.service: Failed with result 'signal'.
2022-04-06T07:22:06.280112+09:00 localhost <daemon.info> systemd[1]: networkoptix-mediaserver.service: Service hold-off time over, scheduling restart.
2022-04-06T07:22:06.282396+09:00 localhost <daemon.info> systemd[1]: networkoptix-mediaserver.service: Scheduled restart job, restart counter is at 1.
2022-04-06T07:22:06.283527+09:00 localhost <daemon.info> systemd[1]: Stopped Network Optix Media Server.
2022-04-06T07:22:06.290279+09:00 localhost <daemon.info> systemd[1]: Started Network Optix Media Server.
2022-04-06T07:22:06.379626+09:00 localhost <daemon.info> mediaserver[23210]: /usr/bin/gdb
2022-04-06T07:22:06.429024+09:00 localhost <daemon.info> mediaserver[23210]: Generate crash report /root/mediaserver-bin_4.2.0.32840-c56b2bc06b90-default_1649197326.gdb-bt...   BACKGROUND
2022-04-06T07:22:06.506304+09:00 localhost <daemon.info> mediaserver[23210]: --- Set ulimits:
2022-04-06T07:22:06.506684+09:00 localhost <daemon.info> mediaserver[23210]: core file size          (blocks, -c) unlimited
2022-04-06T07:22:06.507017+09:00 localhost <daemon.info> mediaserver[23210]: data seg size           (kbytes, -d) unlimited
2022-04-06T07:22:06.507234+09:00 localhost <daemon.info> mediaserver[23210]: scheduling priority             (-e) 0
2022-04-06T07:22:06.507425+09:00 localhost <daemon.info> mediaserver[23210]: file size               (blocks, -f) unlimited
2022-04-06T07:22:06.507561+09:00 localhost <daemon.info> mediaserver[23210]: pending signals                 (-i) 3791
2022-04-06T07:22:06.507743+09:00 localhost <daemon.info> mediaserver[23210]: max locked memory       (kbytes, -l) 65536
2022-04-06T07:22:06.507864+09:00 localhost <daemon.info> mediaserver[23210]: max memory size         (kbytes, -m) unlimited
2022-04-06T07:22:06.509439+09:00 localhost <daemon.info> mediaserver[23210]: open files                      (-n) 1024
2022-04-06T07:22:06.510511+09:00 localhost <daemon.info> mediaserver[23210]: pipe size            (512 bytes, -p) 8
2022-04-06T07:22:06.511361+09:00 localhost <daemon.info> mediaserver[23210]: POSIX message queues     (bytes, -q) 819200
2022-04-06T07:22:06.511883+09:00 localhost <daemon.info> mediaserver[23210]: real-time priority              (-r) 0
2022-04-06T07:22:06.512199+09:00 localhost <daemon.info> mediaserver[23210]: stack size              (kbytes, -s) 8192
2022-04-06T07:22:06.512399+09:00 localhost <daemon.info> mediaserver[23210]: cpu time               (seconds, -t) unlimited
2022-04-06T07:22:06.512570+09:00 localhost <daemon.info> mediaserver[23210]: max user processes              (-u) 3791
2022-04-06T07:22:06.512738+09:00 localhost <daemon.info> mediaserver[23210]: virtual memory          (kbytes, -v) unlimited
2022-04-06T07:22:06.512951+09:00 localhost <daemon.info> mediaserver[23210]: file locks                      (-x) unlimited
2022-04-06T07:22:06.513129+09:00 localhost <daemon.info> mediaserver[23210]: --- Starting: /opt/networkoptix/mediaserver/bin/mediaserver-bin -e
2022-04-06T07:22:06.665978+09:00 localhost <user.warning> ecmd[23202]: Environment[CAgComModuleME3630J2A.cpp(675)]: Card not existed or PIN required.
  • Failed Camera request
2022-04-06 07:21:14.726   5269   DEBUG nx::vms::server::WritableStoragesHelper(0x7f17f95a08): Optimal storage selection: candidates number: 1
2022-04-06 07:21:14.726   5269   DEBUG nx::vms::server::WritableStoragesHelper(0x7f17f95a08): Optimal storage selection: candidate: SpaceInfo { totalSpace: 471492067328, freeSpace: 32230633472, spaceLimit: 32212254720, nxOccupied: 414996655487, available: 439279812608, effective: 415015034239, url: /media/ssd/HD Witness Media, isSystem: 0 }
2022-04-06 07:21:14.727   5269   DEBUG nx::vms::server::WritableStoragesHelper(0x7f17f95a08): Optimal storage selection: total effective space: 415015034239, selection point: 0.88151
2022-04-06 07:21:14.728   5269   DEBUG nx::vms::server::WritableStoragesHelper(0x7f17f95a08): Optimal storage selection: running sum: 1
2022-04-06 07:21:14.728   5269   DEBUG nx::vms::server::WritableStoragesHelper(0x7f17f95a08): Optimal storage selection: final result: /media/ssd/HD Witness Media
[FAILED Camera request]
2022-04-06 07:21:16.799   5072   DEBUG nx::network::http::AsyncClient(0x7f700839d8): Failed to establish tcp connection to http://120.75.94.165:7001/api/moduleInformation?showAddresses=false&keepConnectionOpen&updateStream=10. Connection timed out
2022-04-06 07:21:16.800   5072   DEBUG nx::vms::discovery::ModuleConnector::Module(0x7f70078510, 0c5b99ba-1909-69cf-bd29-4d6116a1cc17): Could not connect to 120.75.94.165:7001: HTTP request has failed: [Connection timed out], http code [0]
2022-04-06 07:21:18.969   5072   DEBUG nx::vms::discovery::UdpMulticastFinder(0x7f680195c0): Multicasts are disabled by function
  • NX witness server debug log(VMS killed)
2022-04-06 07:21:39.267   5072   DEBUG nx::network::http::AsyncClient(0x7f700839d8): Failed to establish tcp connection to http://192.168.11.26:7001/api/moduleInformation?showAddresses=false&keepConnectionOpen&updateStream=10. Connection timed out
2022-04-06 07:21:39.268   5072   DEBUG nx::vms::discovery::ModuleConnector::Module(0x7f7005e430, f794ceb1-cc39-71fd-8abe-2b63c0eea84b): Could not connect to 192.168.11.26:7001: HTTP request has failed: [Connection timed out], http code [0]
2022-04-06 07:21:41.553  25009   DEBUG nx::vms::server::WritableStoragesHelper(0x7f38ff7a08): Optimal storage selection: candidates number: 1
2022-04-06 07:21:41.554  25009   DEBUG nx::vms::server::WritableStoragesHelper(0x7f38ff7a08): Optimal storage selection: candidate: SpaceInfo { totalSpace: 471492067328, freeSpace: 32217120768, spaceLimit: 32212254720, nxOccupied: 415001788462, available: 439279812608, effective: 415006654510, url: /media/ssd/HD Witness Media, isSystem: 0 }
2022-04-06 07:21:41.555  25009   DEBUG nx::vms::server::WritableStoragesHelper(0x7f38ff7a08): Optimal storage selection: total effective space: 415006654510, selection point: 0.877943
2022-04-06 07:21:41.556  25009   DEBUG nx::vms::server::WritableStoragesHelper(0x7f38ff7a08): Optimal storage selection: running sum: 1
2022-04-06 07:21:41.557  25009   DEBUG nx::vms::server::WritableStoragesHelper(0x7f38ff7a08): Optimal storage selection: final result: /media/ssd/HD Witness Media
2022-04-06 07:21:43.955   5133   DEBUG QnMServerResourceDiscoveryManager(0x7f6809f150): Network addresses are up to date: { 172.16.0.192, 192.168.0.254 }
2022-04-06 07:21:43.956   5133   DEBUG QnMServerResourceDiscoveryManager(0x7f6809f150): Searcher VIRTUAL_CAMERA took 0 ms to find 0 resources
[VMS kiiled]
2022-04-06 07:22:09.458  23247    INFO START: ================================================================================
2022-04-06 07:22:09.461  23247    INFO START: Network Optix Media Server started, version: 4.2.0.32840, revision: c56b2bc06b90
2022-04-06 07:22:09.461  23247    INFO START: Binary path: /opt/networkoptix/mediaserver/bin/mediaserver-bin
2022-04-06 07:22:09.463  23247    INFO START: Log level: debug
2022-04-06 07:22:09.463  23247    INFO START: Log file size: 10M, backup count: 25, file: /opt/networkoptix/mediaserver/var/log/log_file.log
2022-04-06 07:22:09.464  23247    INFO START: Mutex implementation: qt
2022-04-06 07:22:09.467  23247    INFO MediaServerProcess(0x557cd32740): IniConfig iniFilesDir: /etc/nx_ini/
2022-04-06 07:22:09.478  23248   DEBUG nx::utils::TimerManager(0x7f700146b0): started
2022-04-06 07:22:09.479  23245   DEBUG nx::vms::discovery::UdpMulticastFinder(0x7f700195c0): Listening handler
2022-04-06 07:22:09.510  23247    INFO nx::vms::server::RootFileSystem: Root tool enabled: false
2022-04-06 07:22:09.516  23247   DEBUG nx::vms::server::nvr::hanwha::ServiceProvider(0x7f70037950): Creating the service provider
2022-04-06 07:22:09.517  23247 WARNING nx::vms::server::nvr::hanwha: Unable to get board id, result -1
2022-04-06 07:22:09.517  23247   DEBUG nx::vms::server::nvr::hanwha::ServiceProvider(0x7f70037950): Unable to obtain device info
2022-04-06 07:22:09.554  23247   DEBUG QnSoapServer(0x7f70035ae0): Successfully bound soap server to local port 54249
2022-04-06 07:22:09.558  23247   DEBUG nx::vms::server::ptz::ServerPtzControllerPool(0x7f700583a0): Creating server PTZ controller pool
2022-04-06 07:22:09.565  23252   DEBUG StreamingChunkTranscoderThread(0x7f7000fe90): started
2022-04-06 07:22:09.578  23247    INFO PluginManager(0x7f70061500): Considering to load Server plugin [/opt/networkoptix/mediaserver/bin/plugins/libaxis_analytics_plugin.so]
2022-04-06 07:22:09.580  23253   DEBUG nx::utils::TimerManager(0x7f7005d030): started
2022-04-06 07:22:09.581  23256   DEBUG nx::utils::TimerManager(0x7f7005e860): started
2022-04-06 07:22:09.581  23258   DEBUG nx::utils::TimerManager(0x7f7005f8b0): started
2022-04-06 07:22:09.583  23254   DEBUG nx::utils::TimerManager(0x7f7005d0d8): started
2022-04-06 07:22:09.587  23257   DEBUG nx::utils::TimerManager(0x7f7005e908): started

・gdb-bt

Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
Core was generated by `/opt/networkoptix/mediaserver/bin/mediaserver-bin -e'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  write_back_motion_list (list=0, mb_type=<optimized out>, 
    b8_xy=<optimized out>, b_xy=1368, b_stride=160, sl=0x7f59478a50, 
    h=0x7f583797a0) at libavcodec/h264.h:913
913     libavcodec/h264.h: No such file or directory.
[Current thread is 1 (Thread 0x7f2dfe3180 (LWP 6744))]

Thread 1 (Thread 0x7f2dfe3180 (LWP 6744)):
#0  write_back_motion_list (list=0, mb_type=<optimized out>, 
    b8_xy=<optimized out>, b_xy=1368, b_stride=160, sl=0x7f59478a50, 
    h=0x7f583797a0) at libavcodec/h264.h:913
#1  write_back_motion (mb_type=<optimized out>, sl=0x7f59478a50, 
    h=0x7f583797a0) at libavcodec/h264.h:941

 

[Environment]

Nx Server : 4.2.0.32840 arm64

Camera : AXIS M3105L

 

 

0

Comments

2 comments

Date Votes
  • Ichiro
    • Network Optix team

    Hi Sei-san,

    Thanks for your report. We are going to transfer your question into ticket, please check your email inbox for further information and share the detail with us via the ticket system.

    Thanks.

    0
  • Ichiro
    • Network Optix team

    Hi Yoshi Sei

    Based on the error message from diagnosis, it looks like the camera rejects the requests and returns HTTP 503, which is an issue on the camera side. HTTP 503 means the camera service is unavailable, however, it should be available.

    You could try below : 

    1. Delete the AXIS camera from the Nx Witness.
      Build Number: 34633
      Password: hz4q50
    2. Update the Nx Witness to the latest patch.
    3. Restore the camera back to factory default.
    4. Add the camera back to Nx Witness.
    5. Confirm if the issue persists.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post