Plugin Development support

Ichiro

Network Optix team

October 28, 2025 11:50

Thanks for your questions.

Could you please provide a bit more information about the behavior you observed on the Desktop Client?

The boundary box of the object should follow the object when you call the setBoundingBox method with the timestamp of the frame which the object presented. (in your control)

Is it the same object you’re referring to?

Also, could you help to explain how you’re assigning the trackId for that object?

0

Sandipan Pal

October 28, 2025 16:52

I have a plugin which reads json messages from a websocket. Within the message we have the bounding box coordinates and the object ID.

The bounding box get positioned on object correctly. So in Frame1, the bounding box is correct, however in frame 2 there are 2 bounding boxes, one which was in frame 1 and other in frame2. This gives an illusion that there is a trail of bounding box following the object.

The trackID is set to the objectID that is recieved from the json message.

Here is a reference pic. If you notice, then then is a second box. The person is walking in the video.

0

Ichiro

Network Optix team

October 29, 2025 10:05

Hi Sandipan Pal,

Thanks for your information, but unfortunately this limited information may not be quite helpful for troubleshooting.

I have checked our STUB and sample plugin, it works quite normally and no “duplicated” boundary boxes fro the same object.

As a result, we may kindly request you to provide minimum sample of your implementation of the object metadata generation. specifically in the device_agent.cpp in the following method.

Ptr<IMetadataPacket> DeviceAgent::generateObjectMetadataPacket(int64_t frameTimestampUs)

As you can see in the sample code, the packet introduce the frameTimestampUs, so each frame will have its boundary boxes settings according to the object.

What I can think of for such “duplicated” boundary boxes might be the trackId, different packet, or the timestampUs.

0

Sandipan Pal

October 29, 2025 16:41

Here is the function snippet.

Please could you explain what you mean by ‘such “duplicated” boundary boxes might be the trackId, different packet, or the timestampUs.’?

Thanks.

0

Alex Kehoe

October 31, 2025 09:32

Ichiro Can you urgently let us know about above. We are very behind in delivering this integration to a customers because the duplicated boxes have been haunting us for over a month. Many thanks

0

Ichiro

Network Optix team

October 31, 2025 16:44

Hi Sandipan Pal,

Thank you for your message.

If you’re seeing multiple bounding boxes for the same object, it usually indicates that the server is recognizing them as new or separate objects based on your implementation of device_agent. That’s why we suggested checking whether your trackId values are changing, duplicated, or inconsistent between frames.

From your screen capture, it appears that your implementation may be generating different trackIds for the same object. However, since there are no details provided for trackFor, it’s not possible to confirmc blindly. You might try printing out the trackFor result to verify whether different trackIds are being assigned to the same target.

At this point, we’re confident that the bounding box mechanism itself is working correctly—both for single and multiple objects. We recommend troubleshooting your code by logging additional information to trace the behavior of your implementation.

If you need a reference, please review the sample code in STUB: object_detection / device_agent.cpp, where you can see how multiple moving objects are handled.

0

Sandipan Pal

November 11, 2025 19:28

Thanks for reply. It was indeed the trackID that was getting modified.

However now we have 2 issues -

1. On the client, the AnalyticObjectDetected Event rule does not have any effect. Whether the rule is selected or not selected, the desktop notification for all objects are created as thumbnail on the right hand side panel. Do you know how to stop this?

2. I created an attribute for one of the objects which is derived from the base person object. However the different options of the attribute is not getting exposed on the client. A different attribute is getting exposed. Below is my code and the screenshot of the advanced objects window of the client.

Manifest String -

{
                "id": "com.sample.personWithBri",
                "name": "PersonBRI",
                "base": "nx.base.Person",
                "attributes": [
                {
                    "name": "BRI",
                    "type": "enumeration",
                    "label": "BRI_status",
                    "values": ["high", "medium", "low"]
                }
                ]
            },

The objectTypeID is included as a supported type.
The object type is set with -

 obj->setTypeId("nx.sample.personWithBri");

The Object attribute is set with -

obj->addAttribute(makePtr<Attribute>(Attribute::Type::enumeration, "BRI", o.bri_status.c_str()));

The object attribtue I get on the client side is -

Please could you let me if I am making any mistake with how the attribute is being exposed. All the above code is in the device_agent.

0

Sandipan Pal

November 14, 2025 09:51

Is there an update on this?

0

Ichiro

Network Optix team

December 12, 2025 06:00

Hi Sandipan Pal

Thanks.

First, Object Detection and Event are two different concepts.

In most use cases, object detection continues to run as long as the plugin is enabled. Most users expect detection to always be active while the plugin is running.If object detection stops, the event itself also loses its meaning.

Events are handled separately. Users can choose whether they want to be notified or not, but detection should still continue in the background. Notifications are only triggered when event notifications are enabled.

Because of this, I’m not sure that “stopping object detection while the plugin is enabled” is an appropriate configuration for most scenarios.That said, at the moment, this is the expected behavior.

For the 2nd questions, it seems not very clear, but I assume you may want to have the attribute filter as the clothing color did. Ex:

If that is the case, then it is highly likely that the manifest of your current implementation is not properly configured or declared.

Based on your manifest, the attribute is not defined as an Enum type, and the attribute configuration itself does not seem correct. Please refer to manifest.md for more details, especially regarding the subType definition for attributes.

In your case, the manifest could be :

// Copyright 2018-present Network Optix, Inc. Licensed under MPL 2.0: www.mozilla.org/MPL/2.0/


// DO NOT USE THIS FILE IN ANY PRODUCTION ENVIRONMENT.
// THIS FILE MUST NEVER BE USED IN PRODUCTION.
// IT IS INTENDED FOR TESTING OR REFERENCE PURPOSES ONLY.

#include "device_agent.h"

#include <chrono>

#include <nx/sdk/analytics/helpers/event_metadata.h>
#include <nx/sdk/analytics/helpers/event_metadata_packet.h>
#include <nx/sdk/analytics/helpers/object_metadata.h>
#include <nx/sdk/analytics/helpers/object_metadata_packet.h>

namespace nx {
namespace vms_server_plugins {
namespace analytics {
namespace sample {

using namespace nx::sdk;
using namespace nx::sdk::analytics;

/**
 * @param deviceInfo Various information about the related device, such as its id, vendor, model,
 *     etc.
 */
DeviceAgent::DeviceAgent(const nx::sdk::IDeviceInfo* deviceInfo):
    // Call the DeviceAgent helper class constructor telling it to verbosely report to stderr.
    ConsumingDeviceAgent(deviceInfo, /*enableOutput*/ true)
{
}

DeviceAgent::~DeviceAgent()
{
}

/**
 *  @return JSON with the particular structure. Note that it is possible to fill in the values
 * that are not known at compile time, but should not depend on the DeviceAgent settings.
 */
std::string DeviceAgent::manifestString() const
{
    return 1 + (const char*) R"json(
{
    "supportedTypes":
    [
        { "objectTypeId": ")json" + kHelloWorldObjectType + R"json(" },
        {
            "objectTypeId": ")json" + kPersonWithBriType + R"json(",
            "attributes": ["BRI"]
        },
        { "eventTypeId": ")json" + kNewTrackEventType + R"json(" }
    ],

    "typeLibrary":
    {
        "eventTypes": [
            {
                "id": ")json" + kNewTrackEventType + R"json(",
                "name": "New track started"
            }
        ],

        "objectTypes": [
            {
                "id": ")json" + kHelloWorldObjectType + R"json(",
                "name": "Hello, World!"
            },
            {
                "id": ")json" + kPersonWithBriType + R"json(",
                "name": "My Person With BRI!",
                "base": "nx.base.Person",
                "attributes": [
                    {
                        "name": "BRI",
                        "subtype": "com.sample.personBriType",
                        "type":"Enum"
                    }
                ]
            }
        ],
        "enumTypes": [
            {
                "id": "com.sample.personBriType",
                "name": "BRI",
                "items":
                [
                    "High",
                    "Medium",
                    "Low"
                ]
            }
        ]
    }
}
)json";
}

/**
 * Called when the Server sends a new compressed frame from a camera.
 */
bool DeviceAgent::pushCompressedVideoFrame(Ptr<const ICompressedVideoPacket> videoFrame)
{
    ++m_frameIndex;
    m_lastVideoFrameTimestampUs = videoFrame->timestampUs();

    // Generate an Event on every kTrackFrameCount'th frame.
    if (m_frameIndex % kTrackFrameCount == 0)
    {
        auto eventMetadataPacket = generateEventMetadataPacket();
        // Send generated metadata packet to the Server.
        pushMetadataPacket(eventMetadataPacket);
    }

    return true; //< There were no errors while processing the video frame.
}

/**
 * Serves the similar purpose as pushMetadataPacket(). The differences are:
 * - pushMetadataPacket() is called by the plugin, while pullMetadataPackets() is called by Server.
 * - pushMetadataPacket() expects one metadata packet, while pullMetadataPacket expects the
 *     std::vector of them.
 *
 * There are no strict rules for deciding which method is "better". A rule of thumb is to use
 * pushMetadataPacket() when you generate one metadata packet and do not want to store it in the
 * class field, and use pullMetadataPackets otherwise.
 */
bool DeviceAgent::pullMetadataPackets(std::vector<Ptr<IMetadataPacket>>* metadataPackets)
{
    metadataPackets->push_back(generateObjectMetadataPacket());

    return true; //< There were no errors while filling metadataPackets.
}

//-------------------------------------------------------------------------------------------------
// private

Ptr<IMetadataPacket> DeviceAgent::generateEventMetadataPacket()
{
    // EventMetadataPacket contains arbitrary number of EventMetadata.
    const auto eventMetadataPacket = makePtr<EventMetadataPacket>();
    // Bind event metadata packet to the last video frame using a timestamp.
    eventMetadataPacket->setTimestampUs(m_lastVideoFrameTimestampUs);
    // Zero duration means that the event is not sustained, but momentary.
    eventMetadataPacket->setDurationUs(0);

    // EventMetadata contains an information about event.
    const auto eventMetadata = makePtr<EventMetadata>();
    // Set all required fields.
    eventMetadata->setTypeId(kNewTrackEventType);
    eventMetadata->setIsActive(true);
    eventMetadata->setCaption("New sample plugin track started");
    eventMetadata->setDescription("New track #" + std::to_string(m_trackIndex) + " started");

    eventMetadataPacket->addItem(eventMetadata);

    // Generate index and track id for the next track.
    ++m_trackIndex;
    m_trackId = nx::sdk::UuidHelper::randomUuid();

    return eventMetadataPacket;
}

Ptr<IMetadataPacket> DeviceAgent::generateObjectMetadataPacket()
{
    // ObjectMetadataPacket contains arbitrary number of ObjectMetadata.
    const auto objectMetadataPacket = makePtr<ObjectMetadataPacket>();

    // Bind the object metadata to the last video frame using a timestamp.
    objectMetadataPacket->setTimestampUs(m_lastVideoFrameTimestampUs);
    objectMetadataPacket->setDurationUs(0);

    // ObjectMetadata contains information about an object on the frame.
    const auto objectMetadata = makePtr<ObjectMetadata>();
    // Set all required fields.
    objectMetadata->setTypeId(kPersonWithBriType);
    objectMetadata->setTrackId(m_trackId);
    objectMetadata->addAttribute(makePtr<Attribute>("BRI", "Low")); 

    // Calculate bounding box coordinates each frame so that it moves from the top left corner
    // to the bottom right corner during kTrackFrameCount frames.
    static constexpr float d = 0.5F / kTrackFrameCount;
    static constexpr float width = 0.5F;
    static constexpr float height = 0.5F;
    const int frameIndexInsideTrack = m_frameIndex % kTrackFrameCount;
    const float x = d * frameIndexInsideTrack;
    const float y = d * frameIndexInsideTrack;
    objectMetadata->setBoundingBox(Rect(x, y, width, height));

    objectMetadataPacket->addItem(objectMetadata);

    return objectMetadataPacket;
}

} // namespace sample
} // namespace analytics
} // namespace vms_server_plugins
} // namespace nx

You can edit the device_agent.cpp and device_agent.h in the sample plugin(as show above) in the metaSDK to achieve this as well.

Thanks.

0

Plugin Development support

Comments

Didn't find what you were looking for?