What is the suggested amount of space needed in the Splunk syslog server to collect logs from 700-1000 network devices?

[cebo_myeza]

I am working on a Splunk project for collecting syslogs from the company network devices, so now I want to implement Splunk to collect logs to 700-1000 network devices, but I am not sure about the amount of space that will be needed in the Splunk syslog server for collecting logs from 700-1000 network devices.

[ltrand]

Depends more on the amount of logs per day vs the number of devices. But in your senerio, if each device did 1GB/day you would need 1TB to hold 1 day. If you wanted to rotate and hold two days on disk, that's 2TB with no compression, 1.5TB with (suggesting 2:1 ratio). Add in overhead to allow for logrotate to function properly and usage spikes, and 3TB becomes reasonable.

It really depends on if it's 1000 firewalls, or 1000 switches, or what the mix is. Get a sample, do some basic math, and you'll get to a better number. Off the top with no more info I'd call for 2 servers with 3TB of usable disk and hope for the best.

[cebo_myeza]

thanks

how can i save the disc space on my syslog sever lets say i just want to check and reset my logs in every 3 moths period in a sever...is that possible?

[ltrand]

Suggesting your in linux, you'll want to put log-rotate on a cron schedule, then use the find function to remove any files older than 90 days.

http://www.cyberciti.biz/faq/how-do-i-rotate-log-files/
http://linuxconfig.org/setting-up-logrotate-on-redhat-linux

sample find & remove old archives:
find /syslog/* -name '*.gz' -mtime +90 -exec rm {} \;

[MuS]

Hi cebo_myeza,

Syslog can be fairly good compressed by Splunk and the resulting disc space usage is pretty low.
There is one things to remember when using syslog, see the best practice http://answers.splunk.com/answers/77688/best-practices-for-logging.html

Regarding the disc space; It is hard to tell what it will be in your case. One easy thing you can do, is send let's say 10 syslog streams into Splunk for some days and see what it will use on the server. This will give you some idea and a base to calculate on.

Hope this helps ...

cheers, MuS

[cebo_myeza]

hi Mus

i need your help.
i did the calculations as you said and it worked perfectly i now know the disk space for my server but now since the company wants to assign me a syslog server for the splunk to use..they also want to know the Core CPU and RAM required for my server...is this important? if so please help how can i approach it to find the RAM and CPU required for my sever?

thanks

[MuS]

Are you talking about the syslog server or the Splunk server? For the Splunk server take a look at the docs about the system requirements http://docs.splunk.com/Documentation/Splunk/6.2.3/Installation/Systemrequirements

[cebo_myeza]

i am talking about syslog server

[MuS]

Best to ask this in the forum of the syslog software provider, since this is only Splunk related. But usually hardware requirements for a syslog server are not to high