Metadata (Events and Objects) produced by an Analytics Plugin finds its way into Nx Server’s internal pipeline, where it is dispatched to its destinations such as the Video Archive or the live video stream going to clients.
Nx Desktop also has its Metadata pipeline — it receives Metadata inside the video stream coming from the Server (either live or from the Video Archive), and passes it to its video rendering engine which can display Objects as colored rectangles on a video frame.
Event Metadata produced by the Plugin is transformed by the Server into the VMS Events of the Event-Rule-Action engine.
There are various situations when Object Metadata (rectangles) produced by the Plugin will not appear on the live camera view in Nx Desktop. The most common reason is a significant delay between the analyzed video frames and the timestamps in the produced object rectangles.
In this case, Nx Desktop receives the rectangles too late — when the appropriate video frames have long been shown — and it will be incorrect to show those late rectangles on the new frames (they would look displaced).
Other reasons include the Plugin generating rectangles with incorrect timestamps (i.e. not taken from the analyzed frames but rather taken from the current time).
To help with investigating these situations, Nx Meta provides a powerful yet simple mechanism called Analytics Logs — Metadata on its way from the Plugin to Nx Server can be captured to text files at various stages of the Server’s and Desktop’s internal pipelines in the parsable format.
Additionally, there is a visualizer for such logs in Nx Desktop, which can draw translucent glasses over the video corresponding to the rectangles written in a log file.
There are several points, listed below, in the Nx Server’s and Nx Desktop’s internal metadata pipelines, at which capturing to text files can be activated. Nested points denote further points down the pipeline, and the sibling points denote parallel pipeline branches after a splitting point:
- Nx Server
Metadata received directly from the Plugin.
Metadata going to be recorded in the Server database.
The initial Server’s hub for the stream.
A further point in the initial Server’s hub for the stream.
Preparing the stream to be fed to the Desktop client.
Feeding the stream to the Desktop client.
- Nx Desktop
Metadata extracted from the stream coming from the Server.
Metadata to be displayed on the video.
To activate metadata logging, in analytics_logging.ini, specify the path to the existing directory in which the log files will be created (see Configuring via .ini files – IniConfig for details about .ini files and their location), and restart Nx Server or Desktop:
Here analytics_log_dir is a path relative to the .ini files directory, or an absolute path.
Note: For a typical Nx Meta installation, the .ini file directories for Nx Desktop and Server will be different, because the Server is run as an OS service (for Linux, under a dedicated user), and the Desktop is run as a user application. So, the Server and the Desktop client each uses its own analytics_logging.ini.
If you want the Object Metadata rectangles (their coordinates) to be included in the logs, specify the following option in the same analytics_logging.ini:
The created log file names contain the prefix defining the point of an internal pipeline, id of a Device (Camera) on which the Plugin worked (if applicable to the particular log), and other easily identifiable details like the Primary/Secondary stream choice represented as _high and _low suffixes.
Visualizing recorded metadata
Besides manually investigating the contents of the Analytics Log files, there is a straightforward way to visualize the Object Metadata rectangles on the video in Nx Desktop: the client reads (caches) the entire specified log file into memory and — when it comes to displaying a particular video frame with a certain timestamp — it searches in this cached log file for Object Metadata rectangles with timestamps close to the frame timestamp; if found, it draws these frames as translucent glasses over the video.
To activate Object Metadata rectangle visualization, in desktop_client.ini, specify the absolute path to one of the Analytics Log files, and restart the Desktop client:
To be able to see the translucent glasses, the timestamps in the log file must refer to the same time periods as the timestamps of the video frames. It means that it is impossible to visualize previously written logs on live video from a real camera. Although, there are the following ways:
- Visualize logs while playing back the archive for the period for which the log has been recorded.
- Use a virtual camera (see Testcamera), and instruct it to produce the same timestamps on each run (staring from the same time point): first, run testcamera to record the Analytics Logs, then run testcamera with the same settings to visualize one of these logs.
The visualized logs (visible as translucent glasses) are shown over the video in addition to the regular Object Metadata rectangles (visible as colored thin rectangles). This allows you to visually investigate potential delays (time gaps) between the two.
There are other options in desktop_client.ini which may be helpful while visually debugging the Plugin. See details of their usage in the comments in this .ini file. These options include:
Show Object Metadata timestamps and other timing info by appending it to the Object attributes shown next to the rectangles in the Camera Window.
In the Right Panel, show Event start and end time, and the precise timestamp of the Object preview video frame.
Show precise timestamps in the Camera Window instead of the human-readable date-time.