Is it possible to forward events from search head (not installing addon on indexers)?

[SonofLiberty]

Good morning!
We have a complex architecture with multiple indexers and couple search heads one of which we have full control of.

When trying to configure CEF Addon to send events to our CEF receiver, we have realized that we need to install the configured addon to all indexers we have which is kind complicated thing because we have many and they are distributed across different data centers and regions.

Is it possible to configure Search Head to be a main CEF events forwarder without involving indexers at all?
Thank you.

[SonofLiberty]

@risgupta, thank you for your answer.

I’ve played with many possible options in inputs.conf/outputs.conf:

1) Set indexAndForward to true/false in outputs app folder and cef app folder
2) Made adjustments to cef index configuration
3) Moved /bin folder from apps/splunk_app_cef to apps/Splunk_TA_cefout

Unfortunately, I was not able to make it work.

From what I can see in the logs there is some activity going on, for example:

Splunkd.log
01-22-2018 09:12:38.784 +0900 INFO TailReader - Batch input finished reading file='/opt/splunk/var/spool/splunk/1516579955_57250.stash_cef_Production_RSyslog'
01-22-2018 09:12:39.820 +0900 INFO TailReader - Batch input finished reading file='/opt/splunk/var/spool/splunk/1516579956_35645.stash_cef_Production_RSyslog'
01-22-2018 09:12:40.835 +0900 INFO TailReader - Batch input finished reading file='/opt/splunk/var/spool/splunk/1516579957_75123.stash_cef_Production_RSyslog'

CEF Index also valid and exists according to what I see during splunk initialization process:

Checking indexes...
Validated: _audit _internal _introspection _telemetry _thefishbucket **cef* cim_summary firedalerts history idx_common_6y idx_ss main os perfmon summary unix_summary windows wineventlog
Done*

However, when I search for index=cef, or sourcetype=stash_cef, it still returns absolutely nothing.
I believe this is the reason for events not being sent: there's nothing to send basically.

However, when I configure CEF App and click on "Show preview of CEF events", it shows correct results and events do exist.

Was anybody able to configure search head as a forwarder?

[micahkemp]

Are your indexers part of a cluster? If so, you would add the app to etc/master-apps on your cluster master and apply the bundle.

You will often find apps that require installation on the indexer (or heavy forwarder). You definitely need to have a means to apply apps to your indexers en masse.

[risgupta]

Yes you can. What you need to do is to use
indexAndForward=true in the outputs.conf for your searchhead.

[harsmarvania57]

Hi,

Based on documentation http://docs.splunk.com/Documentation/CEFapp/2.0.1/DeployCEFapp/Howtheappworks, Add-on is require on Indexers because Add-on which is present on Indexers is passing data to Syslog receivers.