Bounding box and attributes in live stream in Nx Meta VMS
Answered
In our analysing video there are multiple objects and we set bounding boxes to them in plugin using pushMetadataPacket. But only one single bounding box is visualized on video in live mode. Remaining objects are only listed in objects list at the right panel. When watching the video as playback (so not in live) then all the bounding boxes are visualized as expected.
Also there is another problem which is maybe related. When setting attributes to object metadata the attributes are not listed in live mode on bounding boxes but again they are visible when watching as playback.
Could you please verify that if there is not some bug in the VMS? I am using NX Meta Server 4.0.0.29991.
-
Additional note: pushMetadataPacket for all of the bounding boxes for a frame is called at the same time
0 -
Hello Tibor Fülöp,
You are using a quite old version of MetaVMS. The latest publicly available build is v4.1.0.31193 R8.
I'd like to encourage to Sign up for Early access program.
Here are details https://support.networkoptix.com/hc/en-us/community/posts/360015851593-Getting-started-with-plugin-development
Then you could download the latest version of MetaVMS and Metadata SDK and keep following the updates.
0 -
Hi,
Thank you for your response.
When trying Nx Meta v4.1.0.31193 R8, all bounding boxes are correctly visualized. But face attributes are visible only for part of the time.
Now we tried Nx Witness 4.0 latest release and Nx Witness 4.0 latest patch version and both of them have same issues like I am describing in the original post. Is there any stable version which could work better in visualizing live bounding boxes and attributes?
0 -
Moved this discussion to the ticket.
The answer will be published upon a solution found.
TICKET
0 -
Here is more detailed problem desctioption.
The expected result - object attributes are continuously visible on the video overlay along with bounding box.
The actual result - object attributes "flickers" on the video overlay with bounding box (attached video)And we send push object metadata packets in this order and with these contents:
Frame Timestamp 1, content: Bounding box, time of pushing 0.000s
Frame Timestamp 2, content: Bounding box, time of pushing 0.100s
Frame Timestamp 3, content: Bounding box, time of pushing 0.200s
Frame Timestamp 1, content: Age and gender attributes, time of pushing 0.250s (these attributes should be visible on video overlay for frames 1, 2 and 3)
Frame Timestamp 4, content: Bounding box, time of pushing 0.300s
Frame Timestamp 5, content: Bounding box, time of pushing 0.400s
Frame Timestamp 6, content: Bounding box, time of pushing 0.500s
Frame Timestamp 4, content: Age and gender attributes, time of pushing 0.550s (these attributes should be visible on video overlay for frames 4, 5 and 6)
Frame Timestamp 7, content: Bounding box, time of pushing 0.600s
Frame Timestamp 8, content: Bounding box, time of pushing 0.700s
Frame Timestamp 9, content: Bounding box, time of pushing 0.800s
Frame Timestamp 7, content: Age and gender attributes, time of pushing 0.850s (these attributes should be visible on video overlay for frames 7, 8, 9 and potentially next ones)
and so on...(The numbers are simplified, e.g. Frame Timestamp 1 is something like 1594990682128000 in reality)
0 -
Here is the resolution.
Frame Timestamp 1, content: Age and gender attributes, time of pushing 0.250s (these attributes should be visible on video overlay for frames 1, 2 and 3)
It is the wrong assumption. They should not. It is not working like this.
Imagine, a moving object. For example, a face in the video you've sent.
Although you are setting "Frame Timestamp 1", there is a physical time gap. Let's assume you start at 12:00
1. 12:00:00.000 > Frame Timestamp 1, content: Bounding box, time of pushing 0.000s
...
4. 12:00:00.200 > Frame Timestamp 1, content: Bounding box, time of pushing 0.000s
During this time gap 12:00:00.000 - 12:00:00.200 the object changed its coordinates so did the bounding boxes and all the previous bounding boxes have already disappeared from the live stream. Which bounding box should this attributes be displayed nearby?
In order to achieve your objective - "object attributes are continuously visible on the video overlay along with bounding box",
you should send attributes every time you send the bounding box either bypushMetadataPacketor bypullMetadataPackets
Have you seen how stub_analytics_plugin is working? Attributes are following bounding boxes all the time.
Please, have a look at stub_analytics_plugin code. It's available in Metadata SDK.
For example,DeviceAgent::cookSomeObjects()in the end callsconst auto objectMetadata = makeObjectMetadata(object.get());
objectMetadataPacket->addItem(objectMetadata.get());and here is
static Ptr<IObjectMetadata> makeObjectMetadata(const AbstractObject* object)
{
auto objectMetadata = makePtr<ObjectMetadata>();
objectMetadata->setTypeId(object->typeId());
objectMetadata->setTrackId(object->id());
const auto position = object->position();
const auto size = object->size();
objectMetadata->setBoundingBox(Rect(position.x, position.y, size.width, size.height));
objectMetadata->addAttributes(object->attributes());
return objectMetadata;
}setBoundingBoxandaddAttributesare invoked in sync.0
Please sign in to leave a comment.
Comments
6 comments