camera frame and timestamp exact matching

Answered

Jason Shim

September 26, 2022 02:55

Hi,

I'm developing metadata plug-in for my camera and the plug-in implements a algorithm detecting a type of small and fast objects in the camera device.

It requires exact timestamp synchronization between image and meta because small and fast objects can be easily mismatched. The mismatches cause wrong bound box in object list in the VMS. Ant it's actually happening.

I'm using following code for the metadata's time stamp.

objectMetadataPacket->setFlags(nx::sdk::analytics::IDataPacket::Flags::cameraClockTimestamp);

Is there any other method to improve accuracy of timestamp matching?

Thanks.

0

• 

  Andrey Terentyev

  • Network Optix team

  October 12, 2022 11:50

  Hello Jason,

  At first, let me note that the code

  objectMetadataPacket->setFlags(nx::sdk::analytics::IDataPacket::Flags::cameraClockTimestamp);

  indicates that the metadata packet timestamp is sampled from a camera's own internal clock and should be translated to VMS System clock.
  This option makes sense to be set only in the case when a plugin communicates directly to a camera: receives event metadata, object metadata from a camera, which implies timestamps are set in accordance to the camera's internal clock.
  In this case, the Server will recalculate the timestamp received from a plugin (i.e. from a camera) according to System clock, what it does by default, when the Server communicates with a camera directly.

  
  As per my understanding, in detecting a type of small and fast objects in the camera device the most important thing is jitter control, not timestamp synchronization. In other words, what matters it a difference between timestamps of adjacent frames. That difference should be stable and constant for certain frame rate.
  For example, if frame rate is 30fps, the difference between adjacent frames is supposed to be 1/30 sec. In fact it deviates.

  When the "Trust camera timestamp" option is not set (by default), the Server does some job ("magic")  to smoothen the frame rate and make it stable and to recalculate timestamps to System clock.

  Because of this, we highly recommend letting the Server calculate timestamps and control the jitter.

  Thus, the answer to the question

  Is there any other method to improve accuracy of timestamp matching?

  would be:

  1. Let an analytics plugin to receive frames from the Server.

  2. Assign the timestamp of a frame received to the metadata of an object detected in the frame received.

   objectMetadataPacket->setTimestampUs(frameTimestampUs);

  The mismatches cause wrong bound box in object list in the VMS.

  Could you share some screenshots or screencast, please? I suspect it might be the case of track id duplication.

  0

Please sign in to leave a comment.