NX Plugin Frame decode byte array Python endpoint

Completed

Follow

Bacchus Beale

• August 17, 2022 03:04

Hi,

I am sending the frame data to a Python endpoint from metadata SDK.
I get data like this.

Please, how do I decode this to an image? Is this raw bytes? 
In C++ I have used cv::Mat like in the samples but this is not available in Python
would I use np.fromstring and cv.imdecode?
Regards

Bacchus

char* destination = new char[dataSize];
                memcpy(destination, videoFrame->data(0), dataSize);
                std::string dataAsString(destination);

                FrameData frameData;
                frameData.nxCameraID = this->deviceData.deviceID;
                frameData.frameDatasize = dataSize;
                frameData.frameHeight = videoFrame->height();
                frameData.frameWidth = videoFrame->width();
                frameData.frameIndex = this->m_frameIndex;
                frameData.timestampUs = this->m_lastVideoFrameTimestampUs;

                frameData.frameData = dataAsString;

 

0

• 

  Anton Babinov

  • August 23, 2022 11:09

  Hi Bacchus,

  media SDK allows you to request frames in different formats. Decoding will be different based on configuration of your plugin. This behavior is controlled via streamTypeFilter and capabilities fields of the plugin manifest.

  What type of data is expected in python? SDK can provide compressed video frames or the following formats of uncompressed video:

          - `needUncompressedVideoFrames_yuv420`
          - `needUncompressedVideoFrames_argb`
          - `needUncompressedVideoFrames_abgr`
          - `needUncompressedVideoFrames_rgba`
          - `needUncompressedVideoFrames_bgra`
          - `needUncompressedVideoFrames_rgb`
          - `needUncompressedVideoFrames_bgr`

   

   

  0

  Comment actions Permalink
• 

  Bacchus Beale

  • August 25, 2022 05:50
  • Edited

  I made a C++ only version using needUncompressedVideoFrames_bgr like in the samples
  Now, I have Python endpoint doing object detection and tracking
  But when trying to imdecode it as a string or byte array with Open CV it does not work
  
  I also tried PIL Image frombytes and frombuffer, also does not work
  
  Now, I am tring other way around, get thumbnails via API and push metadata
  But I tried running HTTP listener in a new thread and adding to a queue, then use pullMetadataPackets()
  to get from the queue
  
  This causes NX to freeze or start/stop 
  
  Any ideas?
  
  

  Regards

  Bacchus

  0

  Comment actions Permalink
• 

  Andrey Terentyev

  • October 21, 2022 10:43
  • Edited

  Hello Bacchus,

  Sorry for late response.

  Does the issue still persist?

  0

  Comment actions Permalink

Please sign in to leave a comment.