Analytics Logs: Monitoring Metadata Flow

Metadata (Events and Objects) produced by an Analytics Plugin finds its way into Nx Witness Server’s internal pipeline, where it is dispatched to its destinations such as the Video Archive or the live video stream going to clients.

Nx Witness Desktop also has its Metadata pipeline — it receives Metadata inside the video stream coming from the Server (either live or from the Video Archive), and passes it to its video rendering engine which can display Objects as colored rectangles on a video frame.

Event Metadata produced by the Plugin is transformed by the Server into the VMS Events of the Event-Rule-Action engine.

There are various situations when Object Metadata (rectangles) produced by the Plugin will not appear on the live camera view in Nx Witness Desktop. The most common reason is a significant delay between the analyzed video frames and the timestamps in the produced object rectangles.

In this case, Nx Witness Desktop receives the rectangles too late — when the appropriate video frames have long been shown — and it will be incorrect to show those late rectangles on the new frames (they would look displaced).

Other reasons include the Plugin generating rectangles with incorrect timestamps (i.e. not taken from the analyzed frames but rather taken from the current time).

To help with investigating these situations, Nx Witness provides a powerful yet simple mechanism called Analytics Logs — Metadata on its way from the Plugin to Nx Witness Server can be captured to text files at various stages of the Server’s and Desktop’s internal pipelines in the parsable format.

Additionally, there is a visualizer for such logs in Nx Witness Desktop, which can draw translucent glasses over the video corresponding to the rectangles written in a log file. 

Recording metadata

There are several points, listed below, in the Nx Witness Server’s and Nx Witness Desktop’s internal metadata pipelines, at which capturing to text files can be activated. Nested points denote further points down the pipeline, and the sibling points denote parallel pipeline branches after a splitting point:

• Nx Witness Server
• outgoing_metadata
  Metadata received directly from the Plugin.
• analytics_event_receptor
  Metadata going to be recorded in the Server database.
• live_stream_provider
  The initial Server’s hub for the stream.
• spush_media_stream_provider
  A further point in the initial Server’s hub for the stream.
• rtsp_consumer_put_data
  Preparing the stream to be fed to the Desktop client.
• rtsp_consumer_process_data
  Feeding the stream to the Desktop client.
• Nx Witness Desktop
• rtp_parser
  Metadata extracted from the stream coming from the Server.
• widget_analytics_controller
  Metadata to be displayed on the video.

To activate metadata logging, in analytics_logging.ini, specify the path to the existing directory in which the log files will be created (see Configuring via .ini files – IniConfig for details about .ini files and their location), and restart Nx Witness Server or Desktop:

analyticsLogPath=analytics_log_dir

Here analytics_log_dir is a path relative to the .ini files directory, or an absolute path.

Note: For a typical Nx Witness installation, the .ini file directories for Nx Witness Desktop and Server will be different, because the Server is run as an OS service (for Linux, under a dedicated user), and the Desktop is run as a user application. So, the Server and the Desktop client each uses its own analytics_logging.ini.

If you want the Object Metadata rectangles (their coordinates) to be included in the logs, specify the following option in the same analytics_logging.ini:

logObjectMetadataDetails=true

The created log file names contain the prefix defining the point of an internal pipeline, id of a Device (Camera) on which the Plugin worked (if applicable to the particular log), and other easily identifiable details like the Primary/Secondary stream choice represented as _high and _low suffixes.

Visualizing recorded metadata

Besides manually investigating the contents of the Analytics Log files, there is a straightforward way to visualize the Object Metadata rectangles on the video in Nx Witness Desktop: the client reads (caches) the entire specified log file into memory and — when it comes to displaying a particular video frame with a certain timestamp — it searches in this cached log file for Object Metadata rectangles with timestamps close to the frame timestamp; if found, it draws these frames as translucent glasses over the video.

To activate Object Metadata rectangle visualization, in desktop_client.ini, specify the absolute path to one of the Analytics Log files, and restart the Desktop client:

debugAnalyticsVideoOverlayFromLogFile=analytics_log_file

To be able to see the translucent glasses, the timestamps in the log file must refer to the same time periods as the timestamps of the video frames. It means that it is impossible to visualize previously written logs on live video from a real camera. Although, there are the following ways:

• Visualize logs while playing back the archive for the period for which the log has been recorded.
• Use a virtual camera (see Testcamera), and instruct it to produce the same timestamps on each run (staring from the same time point): first, run testcamera to record the Analytics Logs, then run testcamera with the same settings to visualize one of these logs.

The visualized logs (visible as translucent glasses) are shown over the video in addition to the regular Object Metadata rectangles (visible as colored thin rectangles). This allows you to visually investigate potential delays (time gaps) between the two.

There are other options in desktop_client.ini which may be helpful while visually debugging the Plugin. See details of their usage in the comments in this .ini file. These options include:

• displayAnalyticsObjectsDebugInfo
  
  Show Object Metadata timestamps and other timing info by appending it to the Object attributes shown next to the rectangles in the Camera Window.

• showDebugTimeInformationInRibbon
  
  In the Right Panel, show Event start and end time, and the precise timestamp of the Object preview video frame.

• showPreciseItemTimestamps
  
  Show precise timestamps in the Camera Window instead of the human-readable date-time.