Objects in plugin
Answered
We've implemented a plugin that receives object detections from a third party application and passes it to NX using the metadata SDK. Currently, we get a preview of the objects with the bounding box in the right place but we don't see the bounding box overlayed over the video as I did when compiling the the original example. Object detections are also not persisted between sessions.
Is there anything in particular that may be missing from the metadata to be getting this kind of results?
-
@... - Just a precursor to an answer: There's an option in the Nx Meta Client when you right click on a camera called Object Frames & Info - can you confirm you have that setting set to Show Always?
0 -
Hi Tony, thanks for the fast answer. I didn't see that option before but after enabling it I don't see any difference in behavior. The objects still only shown on the objects tab and no boxes are drawn.
1 -
Can you please check, what is the object detection latency?
Also, do you see the boxes when you try to play video from the past (archive)?
Usually bounding boxes are not showing on live video if they are delivered with a large delay (about a second)
0 -
That would seem to be the case. But when trying to see the archive we get the message that there's no archive. Even if the camera was being recorded. The objects detections also seem to disappear from the interface after any changes to the view are made. Things as changing the filter for the time, changing the tab or closing the interface. None of the objects are saved.
0 -
Hello Ezequiel,
Could you share with us the code of your implementation of DeviceAgent::pullMetadataPackets, please?
Do you use pushMetadataPacket?
0 -
We use pushMetadataPacket. We're receiving the metadata over a REST API and in the handler we use a public method of the device agent to call it. Here's the relevant code.
voidAPIObjectHandler(constRest::Request&request, Http::ResponseWriterresponse){const auto objectMetadataPacket = makePtr<ObjectMetadataPacket>();
rapidjson::Document d;d.Parse<0>(request.body().c_str());
objectMetadataPacket->setTimestampUs(d[0]["timestamp"].GetInt64());objectMetadataPacket->setDurationUs(10000);
for (SizeType i = 0; i < d.Size(); i++) // Uses SizeType instead of size_t{
auto objectMetadata = makePtr<ObjectMetadata>();if(d[i]["name"] == ini().object1JSONName){objectMetadata->setTypeId(ini().object1Id);}elseif(d[i]["name"] == ini().object2JSONName){objectMetadata->setTypeId(ini().object2Id);}elseif(d[i]["name"] == ini().object3JSONName){objectMetadata->setTypeId(ini().object3Id);}objectMetadata->setTrackId(nx::sdk::UuidHelper::fromStdString(d[i]["objectid"].GetString()));objectMetadata->addAttribute(makePtr<Attribute>(IAttribute::Type::string, "TS", std::to_string(d[i]["timestamp"].GetInt64())));objectMetadata->setConfidence(d[i]["score"].GetFloat());float width = d[i]["width"].GetFloat();float height = d[i]["height"].GetFloat();float x = d[i]["x"].GetFloat();float y = d[i]["y"].GetFloat();objectMetadata->setBoundingBox(Rect(x, y, width, height));auto bestShotPacket = makeObjectTrackBestShotPacket(nx::sdk::UuidHelper::fromStdString(d[i]["objectid"].GetString()),d[0]["timestamp"].GetInt64(),Rect(x, y, width, height));// auto objectMetadataPacket = new ObjectMetadataPacket();objectMetadataPacket->addItem(objectMetadata.get());}
objectMetadataPacket->addRef();this->deviceAgent->pubPushMetadataPacket(objectMetadataPacket.get());
response.send(Http::Code::Ok);}voidDeviceAgent::pubPushMetadataPacket(IMetadataPacket*metadataPacket){pushMetadataPacket(metadataPacket);}0 -
Hi Ezequiel,
> But when trying to see the archive we get the message that there's no archive.
What exact message? Could you give the text/screenshot please? Could you send server log_file.log file?
> Even if the camera was being recorded. The objects detections also seem to disappear from the interface after any changes to the view are made. Things as changing the filter for the time, changing the tab or closing the interface. None of the objects are saved.
Could you give us step-by-step instruction for reproducing the behavior described?
0
Please sign in to leave a comment.
Comments
7 comments