Nx Witness 6.1.2.42921 – Playback Performance Degrades Over Time, Restart Restores Performance
AnsweredHello and thanks in advance for reading this;
We are troubleshooting a recurring playback performance issue in our Nx Witness environment.
Environment:
- Nx Witness Media Server: 6.1.2.42921
- Windows Server 2022
- Two Media Servers
- Approximately 120–150 camera channels per server
- Archive stored on Synology NAS system over SMB 3.1.1
- Several fixed Nx live-view clients / dashboards are connected to the Media Servers
- Windows itself remains responsive when the issue occurs
Main symptom:
Immediately after restarting an Nx Media Server, playback is very fast and smooth. 16x forward playback is fine and 16x reverse playback is also smooth.
After several hours of uptime, reverse playback becomes progressively very slow and starts stuttering badly. Random seeking through recorded footage also becomes noticeably slower.
Restarting the Nx Media Server immediately restores normal playback performance.
We see this behavior on both Media Servers, not just one.
Timeline / observations:
1. Initial state after restart
After restarting the Media Server, archive playback is immediately fast again. 16x reverse playback works smoothly.
2. Degradation after uptime
After several hours, playback degrades again. We have reproduced this multiple times on both Media Servers.
3. Resource usage does not show an obvious leak
We compared the Media Server process directly after restart and later while playback was slow.
We do not see:
- continuously increasing thread count
- continuously increasing handle count
- excessive CPU usage
- excessive memory usage
- growing SMB connection count
Typical slow-state values:
- CPU: approximately 25–35% total server CPU
- Process memory: stable
- Handles: around 6,000
- Threads: around 900
- SMB connection to the NAS remains stable
In some cases the slow state actually had fewer threads and handles than the fast state.
4. Storage
The archives are stored on Synology NAS systems over SMB 3.1.1.
The SMB connections remain established and we do not see a growing number of SMB sessions.
We previously investigated NAS performance, including disk utilization and storage tuning. The storage can become busy during high-speed playback, but the most important observation is that restarting only the Nx Media Server immediately restores playback performance without changing anything on the NAS side.
5. Most important recurring behavior
The pattern is very consistent:
- Nx Media Server restart
- playback immediately becomes fast
- reverse playback at 16x works normally
- server remains up for several hours
- playback is noticeably slow again after several hours
- Windows remains responsive
- CPU, RAM, handles and threads remain normal
- restarting only the Nx Media Server restores performance immediately
6. Changes in hardware/software
The issue started after we made several hardware and software changes at roughly the same time (in an attempt to improve performance):
- Upgraded both Nx Media Servers from the legacy virtual NIC to VMXNET3, providing 10 Gb network connectivity.
- Reconfigured the Synology storage from the previous 8-disk RAID6 configuration to a 12-disk RAID10 array.
- Upgraded Nx Witness Media Server to version 6.1.2.42921.
Questions:
- Is this a known issue in 6.1.2.42921?
- What steps can we take to further test/diagnose these issues?
Thanks!
-
Hi Frank Goossen,
Thanks for bringing this to our attention, and for the detailed write-up.
We don’t currently have a known issue in 6.1.2 that matches this behavior. Because this issue requires a deeper dive into your specific setup, we highly recommend reaching out to your Authorized Reseller. They are best equipped to investigate the root cause of this specific issue and get it resolved or escalated for you.
To speed things up, it helps to have the following ready when you contact them:
- Whether backup, or any analytics plugins are enabled on these Servers
- Roughly how long after a restart performance starts to degrade, and whether it still degrades when the fixed live-view clients and dashboards are disconnected.
- Full Media Server logs at DEBUG level, captured twice: right after a restart (fast state) and once playback is slow, reproducing 16x reverse playback on the same camera both times.
- A process dump of the Media Server taken in the slow state (and, if possible, one in the fast state for comparison).
- The result of testing one camera that records to local storage instead of the NAS, to show whether the slowdown follows the Server or the storage.
Best regards.
0 -
Hi Norman,
Thank you again for your help so far.
I checked the original purchase agreement for our Nx Witness licenses. Our reseller support contract was explicitly declined at the time of purchase, and the quotation states:
“Support direct with Vendor (Network Optix)”
So as far as we understand, technical support for this installation is intended to be handled directly by Network Optix rather than through the reseller.
In the past I was able to create “Requests” but it seems that right now I can only view my old ID’s and not create a new one.
Would it be possible to convert/escalate this forum case into a support ticket?
0 -
Hi Frank Goossen,
Thank you for checking the purchase agreement and sending the details.
I looked into this on our side. Network Optix has no agreement or purchase order with your company.
Network Optix provides technical support through our authorized resellers.
For this issue, please contact the company you purchased the licenses from. If they find that the issue is a bug in the software, they can escalate it to the Nx support team, and we’ll work with them on it.About the change you noticed in the support portal: until earlier this year, a gap in our support system allowed users who aren’t authorized resellers to submit support tickets. We fixed that, which is why you can still see your previous requests but can no longer create new ones. For the same reason, I can’t convert this forum case into a support ticket.
Best regards.
0
Please sign in to leave a comment.
Comments
3 comments