Should I use a heavy forwarder or indexer for this scenario?

[Exeterengineeri]

Greetings,

I'm trying to figure out if there is an advantage to having a heavy forwarder over just an indexer in the following scenario:

• All of the infrastructure is virtual and is on the same hypervisor. Resources are dedicated.
• Firewall logs are the primary reason for possibly using a heavy forwarder.
• No pre or post processing of the logs is required--we just want them indexed.
• Search factor and replication factor are both set to one. We don't need replicated data or indexes.

On one hand, I understand that if I use a heavy forwarder, I can span the output across multiple indexers. On the other hand, why not just make this machine an indexer, itself, and if I want it to be more of a dedicated resource, just don't let any other UFs or HFs know about it. Yet it would still be part of the cluster. What are there pros and cons of each approach? Thanks in advance.

[sjohnson_splunk]

What kind of firewall? Checkpoint, Palo Alto, Cisco ASA?

There is a specific app for Checkpoint and depending on the volume and number of devices you need to collect from you might need to scale up. If you are just going to forward syslog data from the device then I would recommend that you send that to a syslog server first and then monitor the log files with a universal forwarder.

[MuS]

Firewall logs are the primary reason for possibly using a heavy forwarder.

If those logs are coming in over syslog I suggest to read this excellent post http://www.georgestarcher.com/splunk-success-with-syslog/

cheers, MuS

[mhassan_splunk]

I wrote an extensive blog on syslog-ng with splunk. You may find it useful:
http://blogs.splunk.com/2016/05/05/high-performance-syslogging-for-splunk-using-syslog-ng-part-1/

[skoelpin · Accepted solution]

Excellent question @Exeterengineering

The only disadvantage of installing a heavy forwarder vs. an indexer would be the inability to do distributed searches, so the question is, do you need to scale horizontally sometime down the road or do you believe you have enough beef to do the searching?

Another factor to consider is how much data do you index per day?

[Exeterengineeri]

We very well could have a need to scale beyond what we have now. How about something like a syslog-ng server with a universal forwarder spreading the output to the indexers?

[skoelpin]

This is a great idea. If you want the ability to "scale horizontally" then you need to add more indexers and set up UF's on your syslog server. Adding heavy forwarders would prevent you from scaling since they do not have the ability to distribute your searches to other instances.

[Exeterengineeri]

Thank you, and after some more reflection, I agree. Syslog-ng it is.

[skoelpin]

Great to hear! If this answered your question, can you please accept the answer?