Getting Data In

What is the suggested amount of space needed in the Splunk syslog server to collect logs from 700-1000 network devices?

cebo_myeza
Path Finder

I am working on a Splunk project for collecting syslogs from the company network devices, so now I want to implement Splunk to collect logs to 700-1000 network devices, but I am not sure about the amount of space that will be needed in the Splunk syslog server for collecting logs from 700-1000 network devices.

0 Karma

ltrand
Contributor

Depends more on the amount of logs per day vs the number of devices. But in your senerio, if each device did 1GB/day you would need 1TB to hold 1 day. If you wanted to rotate and hold two days on disk, that's 2TB with no compression, 1.5TB with (suggesting 2:1 ratio). Add in overhead to allow for logrotate to function properly and usage spikes, and 3TB becomes reasonable.

It really depends on if it's 1000 firewalls, or 1000 switches, or what the mix is. Get a sample, do some basic math, and you'll get to a better number. Off the top with no more info I'd call for 2 servers with 3TB of usable disk and hope for the best.

cebo_myeza
Path Finder

thanks

how can i save the disc space on my syslog sever lets say i just want to check and reset my logs in every 3 moths period in a sever...is that possible?

0 Karma

ltrand
Contributor

Suggesting your in linux, you'll want to put log-rotate on a cron schedule, then use the find function to remove any files older than 90 days.

http://www.cyberciti.biz/faq/how-do-i-rotate-log-files/
http://linuxconfig.org/setting-up-logrotate-on-redhat-linux

sample find & remove old archives:
find /syslog/* -name '*.gz' -mtime +90 -exec rm {} \;

MuS
SplunkTrust
SplunkTrust

Hi cebo_myeza,

Syslog can be fairly good compressed by Splunk and the resulting disc space usage is pretty low.
There is one things to remember when using syslog, see the best practice http://answers.splunk.com/answers/77688/best-practices-for-logging.html

Regarding the disc space; It is hard to tell what it will be in your case. One easy thing you can do, is send let's say 10 syslog streams into Splunk for some days and see what it will use on the server. This will give you some idea and a base to calculate on.

Hope this helps ...

cheers, MuS

cebo_myeza
Path Finder

hi Mus

i need your help.
i did the calculations as you said and it worked perfectly i now know the disk space for my server but now since the company wants to assign me a syslog server for the splunk to use..they also want to know the Core CPU and RAM required for my server...is this important? if so please help how can i approach it to find the RAM and CPU required for my sever?

thanks

0 Karma

MuS
SplunkTrust
SplunkTrust

Are you talking about the syslog server or the Splunk server? For the Splunk server take a look at the docs about the system requirements http://docs.splunk.com/Documentation/Splunk/6.2.3/Installation/Systemrequirements

0 Karma

cebo_myeza
Path Finder

i am talking about syslog server

0 Karma

MuS
SplunkTrust
SplunkTrust

Best to ask this in the forum of the syslog software provider, since this is only Splunk related. But usually hardware requirements for a syslog server are not to high

Get Updates on the Splunk Community!

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...