Viewing the saved frame
Answered
I the uncompressed frame to a file in the plugin. But I am not able to view the content.
Configured the camera to send jpeg format.
Below is the simple code to save the frame:
int frameDatasize = videoFrame->dataSize(0) +1;
char* destination = new char[frameDatasize ];
destination[frameDatasize - 1] = '\0';
char* framepath = new char[255];
if (NULL != videoFrame->data(0))
{
const char* a = videoFrame->data(0);
strncpy(destination,videoFrame->data(0), frameDatasize - 1);
ofstream frameFile;
frameFile.open(outFile, ios::out);
frameFile << destination << endl;
frameFile.close();
}
-
Hello Vamsi,
Frame is a set of binary data not textual. To be able to open binary data you have to save it as it is, without any appendings.
By this
destination[frameDatasize - 1] = '\0';
you make destination memory block, supposed to contain original frame binary data, to have an additional "0" (zero) in the end.
And by this
frameFile << destination << endl;
to a file supposed to store original frame binary data you append additional "\n" (new line) symbol, because endl in C++ stands for "\n".
So, the resulting file stores two symbols not supposed to be in the binary file in order it can be interpreted as original frame.
It's pretty predictable you can not view the content, just because the viewer can't recognize a binary frame in this file.0 -
HI Andrey,
Thank you, understood.
So, I have removed the additional allocated byte and 'endl' that's adding the new line.
But still i could not view the saved frame. Am i still missing something?
.
0 -
Vamsi,
Could you please take sample_analytics_plugin from Metadata SDK samples, reproduce your issue with it, and share the whole code here by providing the link to archive file?
0 -
Hi Andrey,
I just modified the earlier code: Below is the code ( _frameCount is a class level variable.)
int frameDatasize = videoFrame->dataSize(0);
int frameMultiplicity = _frameCount % 100;
if (frameDatasize > 0 && frameMultiplicity == 0)
{
const string outFile = "d:/iframes/frame_" + std::to_string(_frameCount) + ".jpeg";
char* destination = new char[frameDatasize ];
if (NULL != videoFrame->data(0))
{
strncpy(destination,videoFrame->data(0), frameDatasize);
ofstream frameFile;
frameFile.open(outFile, ios::out);
frameFile << destination;
frameFile.close();
}
}0 -
Hello Vamsi,
Please take sample_analytics_plugin from Metadata SDK samples, reproduce your issue with it, and share the WHOLE code (i.e. all the files) of sample_analytics_plugin. Just provide the link to the archive file?
0 -
Hi Andrey,
Please find the uploaded plugin code at below location:
https://1drv.ms/u/s!Ag8pGeCnLOAZ4yh20UaMiu-nmVpk?e=J5Mb7z
0 -
Hello Vamsi,
In Engine::manifestString() you request uncompressed video frames from the Server
{
"capabilities": "needUncompressedVideoFrames_yuv420"
}yuv420 is not at all the jpeg format and won't be viewable if you save it to a file with .jpeg extension.
> Configured the camera to send jpeg format.
If camera really sends jpeg, then you need the Server to supply compressed video frames.
Process them in pushCompressedVideoFrame() method in the same way you do now.
Herestrncpy(destination,videoFrame->data(0), frameDatasize);
use memcpy instead of strnspy
Actually, you don't need memory copying at all. See
int frameDatasize = videoFrame->dataSize(0);
int frameMultiplicity = _frameCount % 100;
if (frameDatasize > 0 && frameMultiplicity == 0)
{
const string outFile = "/tmp/frame_" + std::to_string(_frameCount) + ".jpeg";
// char* destination = new char[frameDatasize];
if (NULL != videoFrame->data(0))
{
// memcpy(destination,videoFrame->data(0), frameDatasize);
ofstream frameFile;
frameFile.open(outFile, ios::out);
// frameFile << destination;
frameFile.write(videoFrame->data(0), frameDatasize);
frameFile.close();
}
}
...0 -
Hi Andrey
Should i change the capabilities to
needcompressedVideoFrames_yuv420
Can I use the default implementation in pushCompressedVideoFrame'?My intention is to process the frame in an external service before saving to Nx server. My camera supports Jpeg and H.264.
To start with i have chosen to view in jpeg format.
But if the video frame is compressed how can I process to modify the image.?
0 -
I want to achieve the following so please suggest in line to my goal:
capture the frames from Nx -> send Frame to my program over http -> Process the frame in external program -> push back the frame and metadata to the Nx so that modified frame gets saved.
As I said, I have 2 options H.264 and MPEG from my panasonic camera. Now I have set MPEG.
As you have suggested I will try and let you know the result of capturing the frame.
0 -
Hi,
> Can I use the default implementation in pushCompressedVideoFrame'?
In SDK there is the metadata_sdk/src/nx/sdk/analytics/manifests.md file explaining manifest options.
Here quotation for the "capabilities"
"... No more than one of the following flags can be specified. If none are specified, the Plugin will be
supplied the compressed video frames..."So, in order to have compressed frames you don't specify any frame-format flag in capabilities. In your sabmle_plugin_exampel it should be like this:
return /*suppress newline*/ 1 + (const char*) R"json(
{
"capabilities": ""
}
)json";If your camera is set to supply jpeg, you should have jpeg in videoFrame->data.
> capture the frames from Nx ->Which frame format to choose depends on your "my program". Is your program capable of processing jpeg, MPEG encoded frames, yuv420?
Besides, if you are going to have motion detection enabled on that camera, the Server will decode video stream (either jpeg or MPEG) and produce yuv420 frame for internal purposes."- `needUncompressedVideoFrames_yuv420` - This is the most effective video format, because
it is used internally by the Server, and thus no video frame decoding is needed"> send Frame to my program over http ->
No problem at all. Once you have video frame data, you could send it over a network.> Process the frame in external program -> push back the frame and metadata to the Nx so that modified frame gets saved.
You can't push back the frame. Metadata only.
In this thread there are several posts describing alternative approach to the solution.
https://support.networkoptix.com/hc/en-us/community/posts/360038642973/comments/3600105270940 -
Hi Andrey,
Thanks for the suggestion, sure i shall push back only the metadata from the external application.
However my current issue of capturing and saving the frames in jpeg format is not resolved.
I had followed your suggestions of setting empty string for 'Capabilities' and moved file save logic to 'pushCompressedVideoFrame'.
But the saved frame is not viewable. The file size is around 7KB. I have toggled motion detection also. But did not notice any change..
I have set the compression format on camera to JPEG to output jpeg format.
If i set the format to H.264, in what format will I be able to view the saved image?
0 -
Hello,
Taking into account your objective
> capture the frames from Nx -> send Frame to my program over http -> Process the frame in external program -> push back the frame and metadata to the Nx so that modified frame gets saved.
This goal
> However my current issue of capturing and saving the frames in jpeg format is not resolved.to me looks like "false target".
> If i set the format to H.264, in what format will I be able to view the saved image?
Definitely, no. H.264 uses tricky encoding. Not every frame data contain full frame and might store the delta (difference) from another frame considered as base.
Returning to your objective.
You need to focus on what detection library you are going to use in your external application. What frame type does this library support?
You could clone this repository https://github.com/networkoptix/nx_open_integrations
There are several examples inside, utilizing OpenCV for object detection.Look at step3 example.
The Engine manifests these capabilities"capabilities": "needUncompressedVideoFrames_bgr"
making the Server to supply frames with bgr pixel type.
As you understand these frame will be passed to OpenCV for object detection.
Here is the method declcaration of ObjectDetector class in object_detector.cppDetectionList ObjectDetector::runImpl(const Frame& frame)
As you can see it accepts Frame structure defined in frame.h
struct Frame
{
const int width;
const int height;
const int64_t timestampUs;
const int64_t index;
const cv::Mat cvMat;
public:
Frame(const nx::sdk::analytics::IUncompressedVideoFrame* frame, int64_t index):
width(frame->width()),
height(frame->height()),
timestampUs(frame->timestampUs()),
index(index),
cvMat({
/*_rows*/ height,
/*_cols*/ width,
/*_type*/ CV_8UC3, //< BGR color space (default for OpenCV).
/*_data*/ (void*) frame->data(0),
/*_step*/ (size_t) frame->lineSize(0),
})
{
}
}Pay your attention to the line
/*_type*/ CV_8UC3,
It is how OpenCV knows, frame of which pixel type it is going to deal with.
So, depending on your task, your plugin can be fed with frames of desired pixel format and pass them right away to the detection library without any transcoding.0
Please sign in to leave a comment.
Comments
12 comments