issue of Handling bestShotImageData via objectTrackInfo within objectActions (null)

In Progress

Comments

7 comments

  • Avatar
    Sergey Yuldashev

    Hello Vikram,

    We kindly ask to provide more details for the issue you're dealing with.

    my designation of the best shot was successful

    Please specify how the best shot was provided to the server. Best case scenario if you provide a minimal patch to the Sample or Stub plugin code that would replicate the behavior.

    I'm asking that because based on the output you attached there is a chance that no actual image was passed to the Media Server as a best shot which is why you get null for type and 0-size.

    BUT! You did pass the timestamp attribute value which made server automatically decode the video chunk and extract the next closest frame to the requested timestamp value. Which is why the result timestamp is slightly different from the requested. And in fact this is the expected behavior.

    0
    Comment actions Permalink
  • Avatar
    Vikram

    device_agent.cpp

    cv::Mat rgbImage = cv::Mat(
    image_height,            
    image_width,  
    CV_8UC3,                    
    (void*)image_data,    
    (size_t)0);

    std::vector<uchar> buffer;
    cv::imencode(".jpg", rgbImage, buffer);
    std::vector<char> charBuffer(buffer.begin(), buffer.end());

    bestShotPacket->setImageDataFormat("image/jpeg");
    bestShotPacket->setImageData(charBuffer);


    engine.cpp

    const char* imageData = objectTrackInfo->bestShotImageData();
    int imageSize = objectTrackInfo->bestShotImageDataSize();

    if (imageData && imageSize > 0) {
    cv::Mat data(1, imageSize, CV_8UC1, (void*)imageData);
      image = cv::imdecode(data, cv::IMREAD_COLOR);
    cv::imwrite("cv_enrollshot.jpg", image);
    }

    I sent the necessary amount of image data from device_agent.cpp.

    And I intended to receive this in engine.cpp.

    The reason I deemed this successful is that instead of the object being depicted as a videoframe with a background and highlighted with a bbox, only the specified image I designated was displayed. (Do you understand what I mean?)

    0
    Comment actions Permalink
  • Avatar
    Vikram

    Any news? Sergey Yuldashev

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    Hello Vikram,

    I'm afraid I don't understand what you mean. Could you please shed some lint on what is your objective? What is the purpose you need the besthost image data in the Engine class? Could you share a screenshot of the desired result?

    JFYI, the best shot image (if set) is displayed in the right-hand panel when the detected object is being displayed. There is no way to tell the Server what image to display on the scene instead of the frame extracted from the video archive by the timestamp.

    0
    Comment actions Permalink
  • Avatar
    Vikram

    Andrey Terentyev

    I expected that I would be able to retrieve the image sent in the bestshot packet using this function (bestShotImageData) after reading its description. Are you saying that this is not possible?

    0
    Comment actions Permalink
  • Avatar
    Andrey Terentyev

    All these methods are supposed to be invoked by the Server.

    Vikram, I'm trying to help you and to resolve your trouble in aheiving your goal, but you're not answering the questions I'm asking.

    Instead, you're posing additional questions. Answering them is really difficult without knowing your objective.

    Could you please shed some lint on what is your objective? What is your business scenario? What is the purpose you need the best shot image data in the Engine class? Could you share a screenshot of the desired result?

    Having that information from you and having developer team by my hand, I could find and recommend the most suitable solution for your case.

    0
    Comment actions Permalink
  • Avatar
    Vikram

    I appreciate your efforts so far. I think misunderstandings may have grown because English is not my native language. I am working based on the metadata SDK stub example, specifically with the bestshot example and object action example. This is part of the code provided in your examples:

    [object action example: engine.cpp] 

    Result<IAction::Result> Engine::executeActionWithRequirements(
        Uuid trackId,
        Uuid deviceId,
        int64_t timestampUs,
        Ptr<IObjectTrackInfo> objectTrackInfo)
    {
        IAction::Result result;

        std::string message = "Message generated by the Plugin:\n"
            "    Track id: " + UuidHelper::toStdString(trackId) + ",\n"
            "    Device id: " + UuidHelper::toStdString(deviceId) + ",\n"
            "    Timestamp: " + nx::kit::utils::format("%lld us", timestampUs) + ",\n"
            "    Object Track Info:";

        if (objectTrackInfo)
        {
            message += "\n        Best shot frame: "
                + uncompressedVideoFrameToString(objectTrackInfo->bestShotVideoFrame()) + "\n"
                + "        Best shot metadata: "
                + timestampedObjectMetadataToString(objectTrackInfo->bestShotObjectMetadata());
        }
        else
        {
            message += "null";
        }

        result.messageToUser = makePtr<String>(message);

        NX_PRINT << "Executing an Action with requirements, returning a message: "
            << nx::kit::utils::toString(message);
        return result;
    }

    [bestshot example: device_agent.cpp]

    Ptr<IObjectTrackBestShotPacket> DeviceAgent::generateImageBestShot(Uuid trackId)
    {
        auto bestShotPacket = makePtr<ObjectTrackBestShotPacket>(
            trackId,
            m_lastFrameTimestampUs);

        bestShotPacket->setImageDataFormat(m_bestShotGenerationContext.imageDataFormat);
        bestShotPacket->setImageData(m_bestShotGenerationContext.imageData);

        return bestShotPacket;
    }

    Looking at these two, is it really so confusing to think that I could retrieve the data from generateImageBestShot in the engine?

    That was my thinking, but ultimately, the desired result is as follows:

    I want to pass the bestshot image (data) I generated directly to an external API through object action. The videoframe is not sufficient because it doesn't contain the bounding box area I need, so it may not work as intended. Perhaps it would be enough if I could just get the base64 image information from the track info I initially uploaded.

    I hope you understand well now. Thank you.

    0
    Comment actions Permalink

Please sign in to leave a comment.