Observing delay when requesting mp4/webm HTTP streams via the API
Answered
I hope everyone is having a great day,
While testing the API mp4/webm streams, we found a noticeable delay (approximately 5 seconds) in the stream. So if a movement occurs in front of the camera, it takes about that long for it to appear on the camera's stream.
Trying different resolutions, or the (rt and sfd) parameters does not seem to have any effect.
Viewing the same camera from the Nx Witness Client does not have this issue. The stream at the client is almost instantaneous.
Note, the way we are currently testing delay is to simply paste the request in a browser and observe the difference in the streams between the Nx Witness Client and the browser.
In order to exclude network conditions, we're testing on a local server with a directly connected camera.
Could there be a way to minimize this delay?
-
Official comment
Hello Yousif,
Let me try to break down the latency you might see between different clients/locations into different sources.
1. Natural Network Latency & Jitter
This is obvious, nothing to add here.
2. Video format factor
Video transmitted over the network might behave as an actual stream(webm, ts, mpegts, etc.) or as chunked file that is downloaded progressively(e.g. mp4). When the format is a stream-like, every video frame will be sent immediately and separately. In case of file-like transmission, the minimal data portion is a video chunk which ultimately can not be less than group of pictures that server receives from the camera.
Typically group of pictures default length for different cameras could be ~0.5-2 seconds.
Bottom line - uttilization of mp4 introduces extra delay in comparison to webm or mpegts formats.
3. Calculation time
Sometimes when you request a stream from the Nx Server, you enforce the transcoding procedure to be applied to the stream. E.g. usage of webm-format will always enforce it since it requires a Server to decode native camera stream and encode using VP8 codec native for webm container. It naturally takes time. Weaker server CPUs and higher requested result resolutions make that delay higher.
4. Video player buffer length
Any modern video player introduces a default live buffer/cache which is used to make video smoother in case of potential network issues(packets retransmission, high network jitter, etc.). With this in mind, no video frame received will be displayed before this extra timeout expires.
As per my check, the default chrome browser buffer is quite high. See the screenshot below.
Nx Desktop live buffer was set to dynamic range of 0-50 ms, ffplay was used with no buffer at all. The url opened in both ffplay and browser was the same. The browser introduces more than 3 seconds of delay in my case.
Please try the same experiment with your stream.
Compare delay between browser video and the same url opened with ffplay using command:ffplay -fflags nobuffer <video_url>
ffplay requires ffmpeg to be installed on your PC.
--------------------------------------------------------------------------------------------------------------------------------------------------
The bottom line - if the CPU usage of Server machine is appropriate(not 100% all the time) there's nothing you could tune on the server side to decrease this delay.
I believe the best option in your case is to play with the web-based videoplayer buffer length.
-
Sergey thank you for the lengthy reply, it was very informative.
I've verified the browser video player is indeed the main factor of latency. In comparison to the Nx client and the ffplay tool, the browser's web player seems to collect several seconds of buffer cache before it displays the first frame.
Do you happen to know if there was an implementation of ffmpeg for web browsers, or anywhere we can find examples of web players that can control the buffer length?0 -
Googling gave me this article:
https://flashphoner.com/embedding-a-webrtc-player-for-live-broadcasts-to-a-website/
I did not go through the source code of the described player but 500 ms latency announced in this article worth checking out.
Aside from that I am unaware of some better implementations.
0 -
- Authorized Reseller
Have you tested latency of the web client on 5.0?
I get the following results:- In Nx Client below 500ms
- Nx web client WebM trasport: 4-5 seconds
- Nx web client HLS trasport: 20-30 seconds
Hardware:
Intel Gen 12 i5 CPU
16GB RAM
RTX3050 GPU
Also from the browser when I connect directly to the camera web interface I also get below 500ms delay.
So ts seems like the Nx web client got a very high latency now.
Do you have any buffer settings spesific for the web client that we can adjust?0 -
Hey, I'm a Product Manager at Nx.
Yousif Iyad
We added WebRTC support in 5.1, which reached Beta. Please, check it out: https://meta.nxvms.com/downloads/betasthere is no solid documentation on it yet, but here is some hint on how you can construct an URL that returns a simple player based on webRTC
https://172.20.1.236:7001/webrtc/?camera_id=084419f6-ec75-6b41-f668-2d8285e9d445&position=1674240507000000So this is our first iteration. We will continue to evolve it in the next versions.
Please, test it and we're waiting for your feedback. Thank you!0
Please sign in to leave a comment.
Comments
5 comments