Best Practice for Custom Nx Witness Player (RTSP + GStreamer, Overlay Sync & Performance)

Answered

Liav

May 05, 2026 10:19

Hi,

I'm working on a custom client integration with Nx Witness and would appreciate some guidance on best practices.

Context

• We are building a WPF-based client
• Pulling RTSP streams from Nx Server (not directly from cameras)
• Using GStreamer (D3D11) for decoding and rendering
• Parsing metadata (JSON) from the stream and rendering analytics overlays (bounding boxes)

Main Challenges

1. Accurate overlay synchronization
   Matching overlays to the exact displayed frame (currently using PTS and buffers, but it impacts performance)
2. Performance
   GPU → CPU copies and buffering logic are expensive

Questions

1. Player / SDK
   • Does Nx provide any internal or supported video player component / SDK (similar to other VMS systems)?
   • Or is RTSP + external player (GStreamer/VLC) the recommended approach for custom clients?
2. Overlay synchronization
   • What is the recommended way to sync analytics overlays with video?
   • Is PTS / clock-based synchronization the correct approach?
3. Scalability / Performance
   • Are there recommendations for how many simultaneous analytics video streams can be displayed?
   • Any best practices for multi-stream performance (GPU usage, decoding, etc.)?
4. Metadata
   • Is consuming metadata from the RTSP stream sufficient?
   • Or is there an advantage to using NX Metadata SDK / API for better accuracy or synchronization?

Goal

We aim to achieve behavior similar to Nx Desktop Client:

• Smooth playback
• Accurate overlay timing
• Minimal latency
• Efficient multi-stream handling

Any guidance, documentation, or examples would be greatly appreciated.

Thanks!

0

• Official comment

  

  Adam Pryor

  • Network Optix team

  July 09, 2026 16:06

  Hi Liav thank you for the detailed questions.

  1. Is there a supported player component / SDK?
     • No, we do not offer an embeddable video player component. The supported integration surfaces are the REST API, media streaming endpoints (RTSP in this case), and the Metadata SDK for building analytics integrations on the server side. Nx Desktop and it's player is published at https://github.com/networkoptix/nx_open however this code is provided as-is without documentation or further support. We have a variety of code examples for API support here https://github.com/networkoptix/nx_open_integrations, and developer documentation is here: https://support.networkoptix.com/hc/en-us/categories/32712042906647-Nx-Meta-Developer-Documentation
  2. How should analytics overlays be synchronized with video?
     • The server aligns analytics metadata with media for you. Before forming a media chunk, the server holds media in a reordering buffer (~2 seconds) waiting for the corresponding analytics metadata. It interleaves each metadata packet next to the frame it describes. A more in depth description is found here: https://support.networkoptix.com/hc/en-us/community/posts/360050059114
  3. How many simultaneous analytics streams can be displayed?
     • There is no fixed platform limit. This is bound by your client hardware's decoding capability.
  4. RTSP metadata vs Metadata SDK
     • They both server different purposes. Metadata SDK produces analytics on the server, not a client-consumption API. For historical object search, use the REST API which queries the analytics database.

   

   

   

Please sign in to leave a comment.