How to minimise data use when camera is on a different internet connection from server?
Answered
I am setting up some wildlife cameras in a very remote location. The NX Witness Server is installed on a Raspberry Pi 400 with 4 GB RAM and 512 GB WD SD card at "location 1", which has a 4G wireless internet connection with about 30 Mbps down, 10-30 Mbps up, and about 100 ms latency. The test camera connected to it on the same local network is performing as expected.
NX Witness Client is here at home on a medium-spec Win 10 PC with an 11th gen i7 processor and 16 GB RAM. The internet connection is fibre with 300 Mbps down and 100 Mbps up, 1 ms latency. The test camera at location 1 is generally responsive and performs in NX Witness Client as I hoped.
I also have a solar-powered IP camera at "location 2", which along with its box of electronics, solar panel and battery can be moved around the 150-acre study area. The internet connection is the same type and quality as at location 1 but is a separate ISP account. When adding the location 2 camera to the server at location 1, I get OK-to-"Meh" performance given the two cellular routes the data must traverse before I can see it in Client. However, I need to minimise data use as the plans are expensive. Currently the data use is roughly constant, consistent with the substream being provided constantly from location 2 to the Server at location 1. To test this I turned off recording, and data use dropped to negligible levels, except when viewing live of course.
In the user manual at: Working with Nx Witness > Device Management > Setting Up Cameras and Devices > Recording > Setting a Recording Schedule > Setting Up Motion Detection, it states that "For cameras that perform in-device motion detection, Nx Witness does not implement software motion detection". The camera has on-board motion detection enabled. I would have thought that that would limit data transfer to only when a motion event was sent to the server for recording. Do I understand this correctly, and have I missed a setting that can make the excess data use go away? As a workaround it may be possible to install another RPi in the solar set-up and another instance of NX Witness server but I would prefer to avoid that to minimise power consumption. Thanks.
-
Hi Barry Dent,
If you use camera side motion detection, there still will be an ongoing connection between server and camera, which you hardly can minimize.
Adding another RPI to "location 2" is the best method to reduce the communication, since it only will communicate when there is a client/server connection. If the system is cloud connected, there will be communication between server and cloud instance as well, but we're talking about very small packets, unless you're viewing through the Nx Cloud client.
To reduce the bandwidth between server and client, it is helpful to reduce the resolution, quality, and/or framerate.
These levels are determined by the level of detail you need. I have been involved with wildlife camera on remote locations to observe storks, foxes, swifts, terns and badgers where only the presence was required to capture, so the resolution was reduced to 1920x1080, instead of the 5 MP the camera could deliver. 10-15 FPS was more than enough to observe natural behavior, while still having fluent video. Quality was reduced as well, since details weren't important.
I also was involved in the design of a system to detect and identify insects with AI, and bandwidth could be reduced by using lenses with smaller angles, so more detail was captured, instead of using wide-angle lenses with high resolutions.
Another factor with codecs like H.264/H.265 is image noise. The less noise, the less bandwidth will be created and storage retention time increases as a bonus. To reduce noise, it helps to avoid moving objects due to weather conditions and insufficient lighting.
But for each project, except the for the storks, a local server was used, where typically the power consumption of the (NUC) servers was much less than the POE and IR lighting of the cameras.
0 -
Thanks for that. I agree that the power consumption of the NUC or RPi is only a small extra increment compared with the power consumption of the camera and IR lighting. I am still curious though, what is the nature of the data that is flowing between server and client if indeed it is not the substream? It's about 400 MB per hour in my case. When the client-server connection is disestablished by disabling recording, data use for the system drops to 3-4 MB per hour, which is mostly the data being uploaded by the device I am using to monitor battery health (Victron Cerbo GX).
I would like to follow on with another question, which I will post as a separate topic.
Appreciate your help.
0 -
To clarify, that data consumption of 400 MB per hour is when almost no motion is being recorded. The system is set to record on motion-only and not to record the substream continuously. Motion events are only a minute or two per hour in the test set-up.
0 -
Hi Barry Dent,
The 400 MB/hour seems accurate for my system as well. There are ongoing transactions between camera and server to monitor the status of the devices. Each transaction is just a few kB, but there are lots of them, so it will add up.
0 -
Thank you, that sums it up well for me. It's clear that in my situation- an expensive cellular data plan- that the Server needs to be on site on the same network. As technology evolves, we can hope that the telcos will offer ever cheaper and better cellular data plans (sigh).
0 -
I’m looking at doing something similar to the OP. Wisenet wave on a raspberry Pi for a single camera at the entrance of a storage facility. But there are 50 facilities and id like to merge all the servers over cloud to a single system. These will all have a cellular connection. What is the data usage I can expect at idle for these devices when not being viewed. They would just be communicating as being part of a cloud merged system. Would it be more than 200 mb a month?
0 -
-
Hi Calvin Wientjes: some comments that I hope will be helpful. I'm not a technical person so this may require minor correction by others.
I've since moved away from the RPi system with 512 GB Western Digital SD card (set up according to the video on the NXW site). As the card filled up there seemed to be some kind of caching issue, with the video being recorded at only about 1 fps compared with 25 fps the camera was trying to supply. The NXW server logs showed errors several times a second. This could be corrected for a few days each time by restarting the RPi. I increased the unused portion of the SD card to about 80 GB using the instructions on the NXW help site and that made no difference. The sequentially deleted old footage was not being put into Trash but immediately completely deleted. If you want to use RPis at each site I would suggest trying some other type of USB-attached storage or maybe a different operating system than the standard 32 bit RPi flavour of Linux. The RPi system has continued to work perfectly for remote live viewing, just not for recording.
Now I have two remote systems, one solar powered 12 Volt, the other mains power. I presume you are on mains power at the storage facilities. On mains power I have a mid-range NUC and it records several cameras (motion detection only) to an onboard 2TB NVME drive. I think the NUC is overkill. The solar-powered one has an Odroid M1 with 512 GB NVME. Both of these run NXW Server and reasonably current Ubuntu. There is no difference in user experience between the NUC and the Odroid when viewing footage remotely. NXW Client is installed on two PCs in two more locations, and there is the NXW phone app on my Galaxy S23. I chose not to install NXW Cloud but I have all locations and phone connected securely over Tailscale. This has been rock-solid and as far as I can tell there is no communication between the NXW Servers, Clients or mobile app when I am not viewing live or recorded footage. The solar-powered Odroid system has a Teltonika RUT240 router/modem, and a prepay data plan of about 4 GB per month. This has not run out so far for viewing small pieces of footage over perhaps a 20 minute session per day.
0 -
Hi Barry Dent and Calvin Wientjes,
We are pleased to announce the release of a new support article dedicated to assisting you in minimizing data consumption. This resource has been designed to offer practical guidance on conserving data usage within your systems. You can access the article by clicking the following link: Optimizing Nx Witness in Low Data Environments
While it's important to note that achieving zero data consumption is not be possible, we believe that the insights provided in this article will help in significantly reducing data usage, particularly when faced with limited bandwidth resources.
0
Please sign in to leave a comment.
Comments
8 comments