Introduction to Creating a Video Analytics Plugin

This tutorial aims to show developers how they can create video analytics plugins for Nx Witness.

The goal is not to teach how to use neural networks and algorithms for computer vision tasks, but rather to show developers how to use the Metadata SDK for writing a video analytics plugin (which might use third party neural networks frameworks for real-world tasks). The plugin is a C++ shared library.

In this tutorial, we will create a plugin to detect and track people, cats, and dogs. For object detection, we will use the OpenCV DNN module in conjunction with the pretrained MobileNet SSD Caffe model. For object tracking, we will use the tbm (tracked by matching) OpenCV module.

We will go over step-by-step how to

• prepare the workspace;
• start the project using a template; 
• load and enable plugin inside VMS Server;
• add an object detector, send object metadata to the Server; and
• add object tracking functionality.