p2p send queue overflow
Answered
Hi
CPU utilization has risen sharply today. logs in attachment
the software client does not open, only the web interface
please tell me what could be the reasonhttps://drive.google.com/file/d/1uVCvYqjeoIRMaDZDRUcb95E0sCKOgFPP/view?usp=sharing
2021-01-26 11:39:39.066 1520 WARNING nx::p2p::Connection(0x7f998c6d9350, {cfdfe1fd-0b65-4d20-aa7e-76f0e2eb3bce}): p2p send queue overflow for peer {cfdfe1fd-0b65-4d20-aa7e-76f0e2eb3bce}, queue size: 52951856. Close connection.
2021-01-26 11:39:39.066 1520 WARNING nx::p2p::Connection(0x7f998c6d9350, {cfdfe1fd-0b65-4d20-aa7e-76f0e2eb3bce}): p2p send queue overflow for peer {cfdfe1fd-0b65-4d20-aa7e-76f0e2eb3bce}, queue size: 52952017. Close connection.
2021-01-26 11:39:39.066 1520 WARNING nx::p2p::Connection(0x7f998c6d9350, {cfdfe1fd-0b65-4d20-aa7e-76f0e2eb3bce}): TOO MANY SIMILAR MESSAGES: p2p send queue overflow for peer {cfdfe1fd-0b65-4d20-aa7e-76f0e2eb3bce}, queue size:
-
Hi @...,
Please check this support article, when you follow these guidelines, it will be easier for us to understand the situation and thus the probable cause.
This message means that local buffer for messages is full. When you update your system to the latest patch, this buffer size will be increased and such message will disappear or at least minimize.
Remember it is only a Warning en not an Error. The number of lines with this message could/should be less, but no worries if it appears ones in a while.
Another cause could be that the network connection between peers is lost or slow. The message send rate is slower than the system generates it.
From the file you provided, it appears that you have connected way too many devices to a single server which is a reasonable explanation for the issue.
The maximum number of devices connected to a single server system should be 128, I counted over 400 devices.
I would reduce the number to the maximum recommend number of devices of 128 per server or even lower if you want to benefit from the failover feature.
This won't clarify the CPU load on the client side. More information, like a reproduction scenario, is required to understand the system and possible effects on the CPU.
0 -
understood you, I will accept it in the future.
the current situation can affect the increased CPU load and lack of connection through the client application?
Thank you
0 -
Hi @...,
I can't tell anything for sure without more system information and the reproduction scenario.
That being said; Lot of requests in combination with a buffer overflow most likely will also affect the CPU load.
0 -
OK. if one of the reasons is insufficient bandwidth. I need to understand which devices exactly correspond to the id from the log. how can i find out if i'm using a web client?
0 -
latest system update 16.01.2021
0 -
Hi @...,
The servers are all equal and all information is shared to all servers, routing depends on response and can be different each cycle.
0 -
I didn't understand your answer. How is this connected with the fact that the management client is not working, with a high CPU load and how can I find out the identifier via the web?
0 -
Hi @...,
As said; This won't clarify the CPU load on the client side. More information, like a reproduction scenario, is required to understand the system and possible effects on the CPU.
I can't tell anything for sure without more system information and the reproduction scenario.
The identifiers (HWID) can be found with the following API call:
http://<server_ip_address>:<port>/ec2/getHardwareIdsOfServers
0 -
maybe I put it wrongly high CPU utilization has grown sharply on the SERVER. and here is a direct connection with the number of devices on the server.
the management client does not start on the remote PC. why, what is needed to find out the reasons and how to fix it?0 -
I don't mind if you repeat your questions, but my answer remains the same.
Please check this support article, when you follow these guidelines, it will be easier for us to understand the situation and thus the probable cause.
I can't tell anything for sure without more system information and the reproduction scenario.
0 -
Nx Witness last ver
Server: ubunta 16.04Topology: single server, 400 cameras
my mistake, the RAM load increased, not the CPUthe answer was not found on the forum. what other information do you need, specifically in this case?
some services do not start on command
2021-01-27 11:17:32.983 240278 INFO START: Log level: info
2021-01-27 11:17:20.666 240238 WARNING QMessageLogContext(0x7ffb433324d8): restart requested!
2021-01-27 11:17:20.667 49594 WARNING QMessageLogContext(0x5573d155d248): Stopping server
2021-01-27 11:17:20.766 49628 INFO MediaServerProcess(0x5573d1181a90): Event loop has returned. Destroying objects...
2021-01-27 11:17:20.810 49628 INFO QnStorageManager(0x7ffb98059b10, Normal): Catalog rebuild operation is canceled
2021-01-27 11:17:20.810 49628 INFO QnStorageManager(0x7ffb9805b410, Backup): Catalog rebuild operation is canceled2021-01-27 11:17:32.985 240278 WARNING QnUnixProcess(0x7f87fc034430): Could not set priority [4] for process [240242]: Permission denied
2021-01-27 11:17:32.986 240278 WARNING nx::vms::server::nvr::hanwha: Unable to get board id, result -10 -
Hi,@...
The maximum number of cameras connected to a single server is 128, for larger systems the limit is set to 100 devices per server.
Please reduce the number of devices per server and check if the issues you encounter still occur.
0 -
OK
the seismic device was spread to two servers and again the administrator client stopped connecting to the server and we cannot continue the configuration.
what could be the reason.
please help.0 -
Hi @...,
Not sure what you mean with the seismic device, but if you divided 400+ cameras over 2 servers, it is still too much. The max. should be 128 cameras per server and issue are likely to happen if you exceed this number.
0 -
yes, I understand you. we want to spread the devices to 4 servers, BUT we cannot.
since the administration client does not open!
how do we split channels without control software?0 -
Hi @...,
Do you connect to the server application with a dedicated client or with a client installed on the same device as the server application is running? In case of the last, please try the first option, in case of the first, I assume the only option is to do a clean uninstall and start over.
0 -
tried both options.
unfortunately the option with a complete reinstallation is not considered.
please decide on the current system.0 -
Hi @...,
Sorry, the only option is to start all over with a clean uninstallation.
0 -
let's try another way. I know that there is a real installation of 1000 kans on one server. And there they used powerful server components.
so that we can temporarily secure the work of the old server and spread all the channels to separate servers, what capacity should the old server have? (considering that this is a virtual server)0 -
If there is someone who added 1000 cameras on the server, that’s fine, but not supported and thus irrelevant. It isn't solely that hardware that has its limitations, it is the system as a whole (hardware, software, and OS) that has its limitations.
See also THIS support article, why we support 128 devices max. per server.
If you want to have more powerful hardware, with 4 VM's, that's fine as well if you know how to set up these VM's, and it's pros and cons.
Don't get me wrong, we encourage people to 'play' with our software to see how it can meet their requirements, and we are more than happy to provide trial licenses to facilitate this, but at the end of the day in case of issues, the specifications and recommended hardware requirements we provide should be met, before we can offer more in-depth support.
0
Please sign in to leave a comment.
Comments
20 comments