Is there any way to the total capacity of syslog server?

[debjit_k]

Hi All, 

I'm very new in Splunk kindly guide.

We have one syslog server integrated on UF my question is how to know the capacity of syslog server.

Is there any way I could know the total capacity of syslog server.

Thank you in advance!!

[PickleRick]

Depends on what you mean by "capacity". If you mean the performance limits - it higly depends on what syslog server we're talking about. Splunk's built in plain tcp and udp inputs are not very well suited for production use (due to performance reasons as well as no sane way of keeping some network-level metadata) so I wouldn't use them in real life.

If we're talking about rsyslog or syslog-ng based solutions (including sc4s) possible performance can be influenced heavily by what you do with the events you receive, whether you use TLS and so on. But in general, syslog is a relatively lightweight messaging system so unless you're doing something strange, relatively small machine should suffice. I have a 16-core server as a central syslog-collector and most of the time it's using at most 4-5 cores with traffic at almost constant 30k EPS. But on the next layer, where I receive the events, do some fancy rewriting, filtering and other stuff and forward it to HEC inputs I see a constant use of about 14-16 cores. So you see that for the same traffic the load can be significantly different depending on what you do with it.

[debjit_k]

Hi 

Thank your for this wonderful info.

But my asking is is there any path on server where I can see the specification of syslog server

For example 

6 core processor 

Rom is 1 GB 

And also I want to know is there any way I can know for 1 day Syslog server can inject 4GB of data just like the Splunk license utility. 

Thank you 

[PickleRick]

As I wrote before - the specs for the syslog server depend on how loaded it will be. You can have a syslog server on a machine as small as 1CPU and 128MB of RAM. Hell, you can even set up a syslog server on a small ARM-based machine. Syslog is a veeeeeery broad term.

[isoutamo]

If your daily ingest amount with syslog is only 4GB then you can use almost anything as it's hw. That 6(v)CPU + 1GB should work excellent with it. And with 4GB/day I suppose that 1vCPU + 1GB memory works also. 

As @PickleRick said, the amount of CPUs and mem is totally dependent what you are doing with syslog and how many and which kind of inputs you have for it. But I propose that start with small virtual machine and increase it's size if/when needed. 

r. Ismo

[gcusello]

Hi @debjit_k,

are you speaking of using Splunk as syslog server or using a sylog-ng to write syslog in a file and then index that file using Splunk?

In the first case, I hint to see the Splunk Connect for Syslogs App https://splunkbase.splunk.com/app/4740/ that helps you in connection and data parsing.

Otherwise, you could also use the normal network inputs present in each Splunk, but you have to manually configure your input.

If the second case, you have to configure your syslog-ng server and  your Universal Forwarder to read those files.

I don't see limits in ingestion except that, speaking of syslogs, if there's too traffic or something in the network or maintenance, you loose your syslogs; for this reason it's a best practice to have two Heavy Forwarders configured to ingest syslogs with a Load Balancer as front end, to be sure that all syslogs are taken even if there's a problem or maintenance on one HF.

Ciao.

Giuseppe

[debjit_k]

Hi @gcusello 

Thank you for the response.

But my asking is is there any path on server where I can see the specification of syslog server

For example 

6 core processor 

Rom is 1 GB 

And also I want to know is there any way I can know for 1 day Syslog server can inject 4GB of data just like the Splunk license 

[gcusello]

Hi @debjit_k,

if you're speaking of the hardware reference of a Splunk Server to use as Syslog server, you are speaking of an Heavy Forwarder and you can take the values for a Stand Alore server at https://docs.splunk.com/Documentation/Splunk/8.2.6/Capacity/Referencehardware 

In few words:

• 12 CPUs,
• 12 GB RAM,
• Disk 50 GB,
• virtual (also physical but it isn't mandatory).

In my experience this is the standard Splunk requirements, but as HF I usually use less resources:

• 4/8 CPUs,
• 8 GB RAM,
• Disk 50 GB,
• virtual .

Monitoring it the load to understand if it is sufficient to manage the peaks periods.

Remember the duplication of HFs and the Load Balancer, this i really important!

If you haven't a Physical Load Balancer, you can use DNS to balance the traffic.

Ciao.

Giuseppe