Bounding boxes are not visible/working for custom meta data packets

Answered

Comments

3 comments

  • Avatar
    Andrey Terentyev

    Hello Mohamed,

    1. It order the bounding box to be displayed in the live stream on the scene the metadata has to returned in lest then 2 seconds delay. Are you sure your code (curl query over HTTP, response processing etc.) does not introduce the delay more then 2 seconds?

    2. Does your query return any content to the readBuffer ? Does j has a value?

    Try to dump to log the objectMetadata just before

    objectMetadataPacket->addItem(objectMetadata.get());

    Does it look like what you expect it to be?

    3. Where is m_trackId initialized? Does it have the same value over frames (i.e. pullMetadataPackets calls)?

    0
    Comment actions Permalink
  • Avatar
    Mohamed Thasin

    Hello Andrey, 

     

    After a few modifications on the existing code I am able to view a single bounding box on client Application. Still multiples are not visible even though I am passing multiple coordinates. 

     

    1. It order the bounding box to be displayed in the live stream on the scene the metadata has to returned in lest then 2 seconds delay. Are you sure your code (curl query over HTTP, response processing etc.) does not introduce the delay more then 2 seconds?

         Yes, it returns less than a sec. So, this is not the problem. 

     

    2. Does your query return any content to the readBuffer ? Does j has a value?

         yes, it readBuffer has proper value. Sample Response of J. 

     

    [{"height":0.10374343395233154,"label":"car","width":0.13737773895263672,"x1":0.4747169017791748,"y1":0.27557021379470825},{"height":0.1520291566848755,"label":"car","width":0.23306196182966232,"x1":0.0436464324593544,"y1":0.4737709164619446},{"height":0.05128565430641174,"label":"car","width":0.08260226249694824,"x1":0.5227476954460144,"y1":0.1012464314699173},{"height":0.2633265256881714,"label":"car","width":0.2628181278705597,"x1":0.26807519793510437,"y1":0.6712638735771179},{"height":0.1066703349351883,"label":"car","width":0.1351754069328308,"x1":0.3640907108783722,"y1":0.19261829555034637},{"height":0.13967299461364746,"label":"bike","width":0.09549647569656372,"x1":0.22761660814285278,"y1":0.5885671973228455},{"height":0.06679290533065796,"label":"car","width":0.07677030563354492,"x1":0.7689603567123413,"y1":0.15423545241355896},{"height":0.08153106272220612,"label":"car","width":0.09254050254821777,"x1":0.6379277110099792,"y1":0.19936798512935638},{"height":0.13518506288528442,"label":"car","width":0.11784732341766357,"x1":0.6905719637870789,"y1":0.34653833508491516}]

     

    NX_PRINT<<objectMetadata;
    objectMetadataPacket->addItem(objectMetadata.get());

    >>> The above snippet prints 1. i.e., for each iteration of object in J, it prints 1. (I think it's desired result)

     

    3. Where is m_trackId initialized? Does it have the same value over frames (i.e. pullMetadataPackets calls)?


    m_trackId is initialized in  generateEventMetadataPacket

    m_trackId = nx::sdk::UuidHelper::randomUuid();

    It has the below value and it's same value over frames.

    C47F7161-1D99-457F-B86E-000B09149811

     

    Apart from this, When I print `objectMetadataPacket` at end of each object J iteration completion it has only 1 value. i think this is the issue. objectMetadataPacket is holding single value, instead of appending with previous values.

    `NX_PRINT<<objectMetadataPacket;
    result.push_back(objectMetadataPacket.releasePtr());`

     

    this is a part of log which illustrates the current issue.

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] processMetadataPackets(): Producing 1 metadata packet(s).
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] logMetadataPacketIfNeeded(): Object metadata packet #0 contains 8 item(s).
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] doPushDataPacket() END
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] doPushDataPacket() BEGIN
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] doPushDataPacket() END
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] doPushDataPacket() BEGIN
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] doPushDataPacket() END
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] doPushDataPacket() BEGIN

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1

    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] ----object Metadata---
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1


    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] =====objectMetadataPacket===
    [sample_analytics_plugin_device_{e3e9a385-7fe0-3ba5-5482-a86cde7faf48}] 1


    In the above log it says producing Object metadata packet #0 contains 8 item(s) as I expected. And it's pushing each object meta data into objectMetadataPacket one by one by iterating j. At the end of loop, objectMetadataPacket should contain 8 items. But, it holds only 1. I think this is the reason why I am getting only one rectangle box. If you need any other information please ask me. 

     

    My current CookSomeObjects function looks like below,

     

    std::vector<IMetadataPacket*> DeviceAgent::cookSomeObjects()
    {

     

    std::vector<IMetadataPacket*> result;

     


    auto objectMetadataPacket = makePtr<ObjectMetadataPacket>();
    objectMetadataPacket->setTimestampUs(m_lastVideoFrameTimestampUs);
    objectMetadataPacket->setDurationUs(0);


    CURL *curl;
    CURLcode res;
    std::string readBuffer;
    std::string body = "path=/var/www/html/images/"+std::to_string(m_lastVideoFrameTimestampUs)+".jpg";

    curl = curl_easy_init();
    if(curl) {
    curl_easy_setopt(curl, CURLOPT_URL, "localhost:5000/process");
    curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body.c_str());
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
    res = curl_easy_perform(curl);
    curl_easy_cleanup(curl);
    }

    auto j = json::parse(readBuffer);

    for (auto& context : j) {

    float a = context["x1"];
    float b = context["y1"];
    float c = context["height"];
    float d = context["width"];
    std::string label = context["label"];

     

    auto objectMetadata = makePtr<ObjectMetadata>();

    if(label=="car"){
    objectMetadata->setTypeId(kCarObjectType);
    }
    else if(label=="truck"){
    objectMetadata->setTypeId(kTruckObjectType);
    }
    else if(label=="bike"){
    objectMetadata->setTypeId(kBikeObjectType);
    }


    objectMetadata->setTrackId(m_trackId);
    objectMetadata->setBoundingBox(Rect(a, b, c, d));

    objectMetadataPacket->addItem(objectMetadata.get());

    }


    result.push_back(objectMetadataPacket.releasePtr());
    return result;

    }

    0
    Comment actions Permalink

Please sign in to leave a comment.