Hardware recommendations for 100+ 5mp cameras with 90 days of storage

Answered

Comments

11 comments

  • Official comment
    Avatar
    Norman Graafsma

    Hi Luke,

     

    We don't do any recommendations regarding any brand since Nx can run just fine on nearly all brands (can't think of any specific exception at this moment) as long as it meets our requirements which you can find below: 

    Nx Witness Server Hardware Specs
    Nx Witness Desktop Hardware Specs

    That being said, the Dell r740xd should be more than fine, or actually quite a bit overpowered. 

    I would pick 2 servers with an i3 processor and 8 GB RAM. In this way, you can divide the load on both servers, while they are also capable of running all 100 cameras on a single server with 15 fps at medium quality. In this case, when one server might fail, the cameras will move automatically over to the remaining server and continue to record and you are still able to view all cameras as well. 

    In any case, it is very easy to add NAS storage. the only thing to consider it significant throughput. 100 x 5 MP cameras @15fps at medium quality require a bandwidth of approx. 260 Mbit/s in total. 

    The disclaimer I have to mention: These numbers are meant to act only as a guide to building your system and real-life results could differ significantly, up and down. 

    Comment actions Permalink
  • Avatar
    Luke McFadden

    Thanks for the helpful response.

    I am really liking the idea of using 2 servers.  We have several extra Dell 3060 micros (8th gen i5 w/ 8GB).  I think they would work great, but I’d be limited in not being able to add a 10GbE NIC or internal storage.

    And that’s the main thing I’m trying to understand now, is do I need to record to internal storage?  And then doing a daily “backup archive” to the NAS?  Or can I just write directly to the NAS itself.

    Can you help me understand how NX writes to storage?  From the documentation, it seems to store everything in RAM and then write to storage once every minute?  

    The performance specs for my NAS say:

    Aggregated 1GbE SMB - Sequential Throughput (64KB):  ~450 MB/s read/write
    1GbE SMB - Windows File Transfer (1MB file x 3000): ~90/65 MB/s read/write.

    So with the way that NX writes to storage, could you estimate based on these numbers what I might be looking at?  It’s a 12 bay drive, in RAID 5, SATA 3.5” 10TB 7200/256.  Their test drives have read/write specs at around 230MB/s, and mine around 270MB/s.

    I think that writing to internal drives, that could net us 3-5 days of storage would be the best setup, as it would add another layer of redundancy if we had trouble with the NAS.   But, I don’t know if we can pursue that option yet, so trying to find out if we can write directly to the NAS.

    We likely won’t actively review archived footage very frequently, but I do want to take that into consideration for bandwidth.

    And as I’m typing, that may mean that we have to have 10GbE on the server(s) to support receiving the feeds, but also to support live streaming to a few different clients.

    I know you don’t recommend hardware, so just asking for some clarification to help me pursue options.  :D. THANKS!

    0
    Comment actions Permalink
  • Avatar
    Tanapol Euaungkanakul

    Hi,

    Im using Intel Xeon Silver 4110 CPU

    32GB DDR4 ECC

    8x10TB Enterprise SATA 7200RPM Disk  Non-RAID

    2x10TB for backup Non-RAID

    for 90x5MP cameras all H.265+, 24hr Hi+Low record

    and it works just fine 
    CPU usage is around 30-40%

    I able to keep record around 6 months

    0
    Comment actions Permalink
  • Avatar
    Luke McFadden

    @Tanapol

    Thanks for the helpful information. Do you mind if I ask a few questions?

    I’m excited that you are getting around 6 months. This is 24x7 constant on high and low? Impressive! Are you using VBR?

    What is your memory usage like. I had been planning on just 16G.

    What are you seeing for your incoming network throughout normally?

    And how do you handle your backup since it is smaller than your primary storage?

    0
    Comment actions Permalink
  • Avatar
    Norman Graafsma

    Hi Luke,

    Ignore the 10 Gbit NIC recommendation. Based on the information you provided, I don't see the need for it. 

    For the number of cameras and the calculation I made (100 x 5 MP cameras @15fps at medium quality require a bandwidth of approx. 260 Mbit/s in total), a 1 Gbit NIC is more than enough. 

    Regarding the storage. You can add the NAS as external storage in Nx (Like shown in THIS video) The NAS can be set as 'Main Storage' so in case your server hasn't the capacity to meet the required retention time, you can expand it easily and thus write directly to the NAS.

    Still, I would also equip the server with local storage to keep recording in case the connection between server and NAS fails. Rare in a LAN, but better safe than sorry. 

    Based on the average bandwidth of approx. 260 Mbit/s in total, you would need approx 90 TB of storage for 30 days. 

    But since this strongly depends on your situation. A scene with 24/7 movement gives a lot more storage that a storage room with maybe a few minutes of movement during the day. 

    Also, do you really need 5 MP? Often a camera is used for an overview of a scene and a 1080p (or even a 720p) camera can do this job just fine and will save you a lot of storage. Also, areas with not too much movement could also be set to Motion+Lo-Res recording, which also saves you a lot of motion.

    Regarding RAM, we initially write to RAM, but it remains there as short as possible. The 1-minute delay is meant to add the data to the storage and add the metadata into the database.

    In my home system (i5+8 GM) the buffer is approx. 10-15 sec. but is shown as 1 minute. Also, I can't export the data at this minute yet. I'll have to wait till it is processed in the database. 

    If you like I can bring you into contact with one of our resellers who can advise you what you need, based on your specific situation. Just tell us where you are based. 

    At last, if you have the servers and NAS already, I'm more than happy to provide you some trial licenses, so you can just start using the software, see if you like it and what the storage will be, based on your situation. If you can record the data for at least a week, you can use the built-in storage analytics to see how much storage you would need for your specific situation. 

    @Tanapol made a valuable addition. The use of H.265 might reduce the storage significantly, especially in a situation where the movement is limited. 

    0
    Comment actions Permalink
  • Avatar
    Tanapol Euaungkanakul

    Luke McFadden,

    I using that server for something else too

    but memory usage of NX mediaserver alone is about 3.5GB I think your 16GB of RAM maybe OK

    and my record is 7x24, All of them are VBR

    (Auto configure by NX; 70% are set to Hi or Best 15fps/ 30% are set to medium 15fps)

    most of my cameras are indoor cams(only people moving on working hours)
    AVG bitrate around 0.2-3.0 Mbps

    and about 10 of them are outdoor cams
    AVG bitrate around 3.0-5.5 Mbps

    Total bitrate of 115 Mbps

    I use Dual 10 Gbit NIC (one for standard LAN another for CCTV Network)

    Incoming network throughput for CCTV NIC report by Windows's taskmanager is about 70-100 Mbps
    I think 1 Gbps is fine for you plan to use this exclusively for NX only 

    For backup I use scheduled backup(realtime backup will not work because backup disk is much more smaller than Main storage) so it will backup only few newest days which is OK for me.

     

    PS. I use DW Spectrum 3.2.20833 (one of NX OEMs) 
    I have some issue for version higher than is so I stick with this until v.4.0 is available

     

    0
    Comment actions Permalink
  • Avatar
    Luke McFadden

    @Tanapol, thanks for the detailed response!

    @Norman, Thanks for the info as well.  I do not yet have the server/NAS, still doing a bit of research before moving forward with a decision.

    I really like the Hive ability for failover, but I'm wondering for simplicity about just running the VMS from my NAS in a VM?  I've seen that there a Docker package, but seems that a VM would be better for support?

    My available NAS options include the follow processors:

    It's about a $1000 price jump to the higher model.  Would the Atom be able to run the VMS with 110 cameras well?  I could put that cost difference into two i5 Dell's with a 8TB drive for intermediate storage, which would be better from several perspectives, but just curious about the capabilities with these two processors.  I'd bump the RAM to 16GB for anything.

    0
    Comment actions Permalink
  • Avatar
    Luke McFadden

    @tanapol Any particular reason you are not using RAID with your drives? Just curious

    0
    Comment actions Permalink
  • Avatar
    Tanapol Euaungkanakul

    I didn't use RAID in CCTV
    Because it waste of space
    and older footages is didn't really importance
    (but it nice to have for quick overview of movement)
    Only a newest month in backup should be enough for me

    For using RAID5 or 6 in 10TB drives it will take too long to rebuild the array once it fails and when rebuild, all of disk will take 100% of load and may causes another disk failure which mean totally lost of all data. I experienced that before. 

    So no RAID5 for me but I still do RAID6 if it impotence. 


    I do a quick test on single lost of HDD without RAID only some segments of each camera will lost  (eg. 1min segment of VDO for evey 5 mins in each camera ) so it acceptable for me

    For NAS running VM I already try it with some 3000$ NAS but compared this price to proper Supermicro or any other server brand it way underpowered and I think it VM is too unreliable (I didn't use it with VMS) but VM sometime crashed or stop responding for no reason.

    1
    Comment actions Permalink
  • Avatar
    Tanapol Euaungkanakul

    Check this out, look at Launch Date of the CPU that synology offer i think it not worth it if you going to run VM on it.
    https://ark.intel.com/content/www/us/en/ark/products/91202/intel-xeon-processor-d-1521-6m-cache-2-40-ghz.html

    I use Synology before and I love it in term of home use(share photos and files) because it have nice GUI and offer some app like DSPhoto which easy to use for user.
    But when it come to business it way too underpowered and overpriced.

    1
    Comment actions Permalink
  • Avatar
    Luke McFadden

    Thanks for that feedback. Yes I wish I could easily order a custom server. But it is a bit difficult here.

    We only have one r740 configuration for new servers, and a few options for used servers. New r740 is 8 bay, r630xd is 12 bay. But most all are over configured for what we need, and just have to buy as is.

    Dell PowerEdge R740 Server, Intel Xeon Silver 4114 2.2G, 10C/20T, 9.6GT/s 2UPI, 14M Cache, Turbo, HT (85W) DDR4-2400, iDRAC Group Manager, Enabled, PowerEdge 2U LCD Bezel,Chassis with up to 8 x 3.5" SAS/SATA Hard Drives for 1CPU Configuration, Riser Config 1, 4 x8 slots, 14G OpenManage placemat, Quick Sync 2 (At-the-box mgmt),32GB(2x 16GB) RDIMM, 2666MT/s, Dual Rank, Blank for 1CPU Configuration, No Additional Processor, iDRAC9,Enterprise, (4x2TB) 7.2K RPM NLSAS 12Gbps 512n 3.5in Hot-Plug Hard Drive , PERC H730P RAID Controller, 2GB NV Cache, Mini card, 6 Performance Fans forR740/740XD, Standard 1U Heatsink, DVD+/-RW,SATA,Int, Dual, Hot-plug, Redundant Power Supply (1+1), 750W,2x Deskside Power Cord, 220V, 2M (Cambodia), Trusted Platform Module 1.2,2x Long Jumper Cord, C13-C14,4m,12a (APCC except ANZ), Broadcom 57416 2 Port 10Gb Base-T + 5720 2 Port 1Gb Base-T, rNDC, No Operating System, 3years warranty on hardware.

    This one is around $4500, but comes with expensive drives that would be hard to resell here I think.

    0
    Comment actions Permalink

Please sign in to leave a comment.