Why pushUncompressedVideoFrame doesn't work?

In Progress

Follow

Fujiwara Daichi

• November 14, 2024 09:53

I want to use pushUncompressedVideoFrame in DeviceAgent, but never enter inside while test plugin is working. 

Could you please tell me why this happens?

 

Code is following. While working, never enters this method.

Otherwise, when using pushCompressedVideoFrame, program enters method. But it kind of freezes at pushMetadataPacket.

bool DeviceAgent::pushUncompressedVideoFrame(const IUncompressedVideoFrame* videoFrame)

{

    try{

    m_lastVideoFrameTimestampUs = videoFrame->timestampUs();

    // Detecting objects only on every `kDetectionFramePeriod` frame.

    if (m_frameIndex % kTrackFrameCount == 0)

    {

         const auto objectMetadataPacket = makePtr<ObjectMetadataPacket>();

         const auto objectMetadata = makePtr<ObjectMetadata>();

         objectMetadata->setConfidence(1.0);

         objectMetadata->setTrackId(m_trackId);

         static constexpr float d = 0.5F / kTrackFrameCount;

         static constexpr float width = 0.5F;

         static constexpr float height = 0.5F;

         const int frameIndexInsideTrack = m_frameIndex % kTrackFrameCount;

         const float x = d * frameIndexInsideTrack;

         const float y = d * frameIndexInsideTrack;

         objectMetadata->setBoundingBox(Rect(x, y, width, height));

         objectMetadata->setTypeId(kHelloWorldObjectType);

         objectMetadataPacket->addItem(objectMetadata.get());

         MetadataPacketList result;

         if (objectMetadataPacket)

             result.push_back(objectMetadataPacket);

           

         for (const Ptr<IMetadataPacket>& metadataPacket: result)

         {

             metadataPacket->addRef();

             pushMetadataPacket(metadataPacket.get());

         }

     }

    }

    return true;

    }

    catch (std::exception e)

    {

        pushPluginDiagnosticEvent(

            IPluginDiagnosticEvent::Level::error,

            "Object detection error.",

            e.what());

    }

    ++m_frameIndex;

    return true;

}

 

0

• 

  Andrey Terentyev

  • November 15, 2024 08:24
  • Edited

  Hello,

  Most probably, you did not request uncompressed frames from the Server.

  It can be done in the Engine's manifest. See the engine.cpp in the sample analytics plugin of the Metadata SDK

  std::string Engine::manifestString() const
  {
      // Ask the Server to supply uncompressed video frames in YUV420 format (see
      // https://en.wikipedia.org/wiki/YUV).
      //
      // Note that this format is used internally by the Server, therefore requires minimum
      // resources for decoding, thus it is the recommended format.
      return /*suppress newline*/ 1 + (const char*) R"json(
  {
      "capabilities": "needUncompressedVideoFrames_yuv420"
  }
  )json";
  }

  0

  Comment actions Permalink
• 

  Fujiwara Daichi

  • November 18, 2024 02:52

  Hi Andrey,

  Thank you for advise.

   

  By adding capability, successfully done uncompressed frame requests

   

  But then, bounding boxes are not captured. What is wrong with my source code?(same as first post).

  By debugging, below was done without error.

  pushMetadataPacket(metadataPacket.get());

   

   

  0

  Comment actions Permalink

Please sign in to leave a comment.