Convert IUncompressedVideoFrame to OpenCV Mat

Answered

Permanently deleted user

November 05, 2019 12:15 Edited

I am unable to convert IUncompressedVideoFrame to openCV Mat. In the example stub_analytics_plugin_ini.h, I changed 

  NX_INI_STRING("bgr", needUncompressedVideoFrames, needUncompressedVideoFramesDescription.c_str());

and in DeviceAgent::checkVideoFrame() did 

    for (int plane = 0; plane < frame->planeCount(); ++plane)
    {
        if (checkVideoFramePlane(frame, pixelFormatDescriptor, plane))
        {
            if (NX_DEBUG_ENABLE_OUTPUT)
                dumpSomeFrameBytes(frame, plane);
            cv::Mat img(frame->height(),frame->width(),
                        CV_8UC1,(char*)frame->data(plane),frame->lineSize(plane)/pixelFormatDescriptor->chromaHeightFactor);
            NX_PRINT<<__func__<<"Got here now - val : "<<img.cols<<" "<<img.rows<<"   linSize = "
                   <<frame->lineSize(plane)<<"   "<<frame->dataSize(plane)<<" plane count= "<<frame->planeCount();
            cv::imwrite("/home/anubhav/Desktop/testVMS.jpg",img);
        }       
  else
        {
            success = false;
        }
    }

 

I am not getting the complete frame. What might be the problem. How can I successfully convert the frame to opencv Mat.

3

• 

  Permanently deleted user

  November 06, 2019 08:30 Edited

  Anubhav,

  Here is how we (the Nx) do this:

  cv::Mat img(
      /*_rows*/ frame->height(),
      /*_cols*/ frame->width(),
      /*_type*/ CV_8UC3, //< BGR color space (default for OpenCV)
      /*_data*/ (void*) frame->data(0),
      /*_step*/ (size_t) frame->lineSize(0))

  0
• 

  Permanently deleted user

  November 20, 2019 06:56

  How about if I choose to use yuv420 only. I believe my camera is not allowing bgr, or is there any alternative.

  0
• 

  Permanently deleted user

  November 20, 2019 07:26 Edited

  The server will transcode video into a requested pixel format, it doesn't matter what specifically camera is streaming.

  Most likely you just need to choose the pixel format required for your neural network or video analytics algorithm.

  0
• 

  Permanently deleted user

  April 16, 2020 08:14

  I am using the above code to store image from the video stream. But, it's coming as black and white image (grey scale). 

  
  
  

   

  

   

   

  Any idea how to get RGB image?

  I have raised  a new post regarding this issue. https://support.networkoptix.com/hc/en-us/community/posts/360043831873-saving-frames-is-returning-gray-scale-image
  it gives more details of my problem. 

  0
• 

  Nx Support

  April 17, 2020 15:52

  That other post explains, how you can choose pixel format.

  The key idea: there are different ways to represent color image: YUV, RGB, etc.

  You need to choose what format do you need specifically: maybe based on how your analytics model was trained, maybe based on other considerations. Then you can request that specific pixel format for the plugin using manifest.

   

  If in this example you are requesting YUV format, so data(0)  is the first component of this representation: Luminance (Y), so by saving this first component you get only brightness component, so you see grayscale image https://en.wikipedia.org/wiki/YUV

  0
• 

  Permanently deleted user

  November 24, 2021 03:38 Edited

  Hi,

  Say I convert IUncompressedVideoFrame into cvMat following the provided method (https://support.networkoptix.com/hc/en-us/community/posts/360037395653/comments/360009552253), do I need to free frame or frame-data(0) ourself, or when will it be freed?

  If I use the converted cvMat in a pipeline consists of 2 or 3 threads, how do I make sure the frame or frame->data(0) lasts until the pipeline is done while not causing memory leak?

  0

Please sign in to leave a comment.