1. Integration Capabilities – What to Expect

What purpose does AI-driven video analytics serve and how does integrating with Nx Witness help me and my users?

AI-driven computer vision/video analytics solutions serve two basic purposes:

• To augment video to allow operators to accomplish tasks more quickly / accurately
• To automate and remove human operators from a system reaction/process completely
  

For Nx Witness we can distill these two general concepts into something more specific:

Video Augmentation

A Plugin captures metadata from video and display that metadata in the Nx Witness Desktop (object bounding boxes and overlays on to of live or recorded video) to generate an augmented reality view of video feeds. Plugins also generate AI-triggered Events and enable smarter search.

• Example Use Case: Provide an alert to an operator when an unidentified person has entered a secure area, put an overlay on the person on live/recorded video, and allow the operator to search for recorded video from other cameras that include the target individual.
  

System Automation

We can use the Rules Engine in Nx Witness to program the System to react without the need for operator participation.

• Example Use Case: Tony is identified by a Nx Witness system with an integrated face recognition system. The System automatically sends an HTTP Request to a 3rd party Access Control solution authenticating Tony, which opens a secured door to allow Tony entry.

Nx Witness provides two ways to achieve augmentation/automation with A.I. video analytics:

1. Nx Witness captures metadata (detected objects) from integrated computer vision/video analytics solutions and stores the metadata in a dedicated database on the Nx Witness Server application.
2. Nx Witness's Metadata SDK provides the ability to create Custom Events that can be used by operators in the Rules Engine to create rules that automate System reactions.

Objects & Events Methods

The latest version of Nx Witness (v4.0 alpha and later) provides two methods for integrating intelligent video applications:

Objects

• accepts detected objects with bounding boxes and attributes
• stores the entire history of detected objects in the database
• visualizes detected objects and their attributes on live or recorded video
• searches through all detected objects
• shows live feed of objects in the interface

Events

• accepts information about specific event detected on analyzed video
• stores events in the database
• searches through the events log
• takes configurable actions for those events, for example:
  • show notifications
  • bookmark an event
  • send a request to a third-party service (to open a door or to perform any other action)
  • send a command to Zapier (to do a lot of other automation: https://zapier.com/)

The Metadata (Analytics) SDK also provides additional features for deep integrations:

Settings

Allows configuring the analytics solution inside Nx Witness Desktop.

• Configure ‘global’ (VMS System-wide) settings for video analytics such as the license key
• Configure video analytics for a specific camera

Context actions

Allows manipulation of detected objects.

• Add a name to a detected person
• Add a license plate number to a block list
• Mark the detection as incorrect to train AI
  

Plugin events

Send diagnostic notifications to the user.

• Notify the user that analytics service is not available
• Inform the user about incorrect settings configuration

Plugin Use in Nx Witness Desktop

Enabling and Configuring a Plugin

Once a Plugin is installed on Nx Witness Server, Nx Witness Desktops in a System will have a new ‘Analytics’ tab in the Camera Settings dialog where the user can enable or disable the Plugin for the specific camera(s) as well as configure other camera-related settings. Once a Plugin is enabled for a camera Nx Witness Server will feed video frames into the integrated video analytic engine for analysis.

Configuring System-Wide Settings for a Plugin

Users can find and modify the Plugin’s System-wide settings in the Analytics tab on the System Administration dialog.

Viewing detected objects on live / recorded video in the Objects tab of the Notifications panel

Once a Plugin is enabled, users can see bounding boxes and previews for detected objects on live or recorded video by opening a camera in a Layout and selecting the Objects tab in the Notifications Panel.

Bonus: Context Actions

Context actions provided by a Plugin (e.g. enroll face) can be accessed by a user by right-clicking on detected objects in the Object tab of the Notifications Panel.

Accessing Analytic Events

Once a Plugin is enabled, Plugin-defined analytics Events become available in the Event Rules Engine, and users can create and configure Rules using these Plugin Events to trigger System Actions.