Is it possible to see _audit and _internal events from all search heads on each of them?

[m_efremov]

I have a Splunk 6.2 search head cluster integrated with indexer cluster. If I search in index=_audit, I see _audit events from all members of indexer cluster, but only from one member of search head cluster (from search head on which this query is performed). Same situation with _internal index (but in _internal I also see events from all my forwarders). I want to see _audit and _internal data from all my splunk search heads (not only from all indexers and forwarders). Is it possible? How to configure?

[dveuve_splunk]

Once you have the right config in place to get the data in, also think about how to make sure that you are able to make sense of that data. I have an app designed to analyze _audit data and provide analytics for user behavior and adoption, in case that is helpful. https://splunkbase.splunk.com/app/2632/

[m_efremov]

Interestig, but there is the notification "Do not install on a Search Head Cluster" at app's documentation: https://splunkbase.splunk.com/app/2632/#/documentation . I have a search head cluster and can't test this app on it.

[dveuve_splunk]

You should generally have at least one non-clustered search head in your environment to allow for management applications such as the DMC and this app. This can live on administrative functions such as the deployer. You may want to consider setting one up in the future.

[Steve_G_ · Accepted solution]

For instructions on how to forward internal data from the search heads to the indexers, see http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Forwardsearchheaddata

[m_efremov]

I have been tested it and it works. Solutions from previous comments are also correct, main idea is to point search heads to indexers in outputs.conf

[jimodonald]

forward those events to your indexers just like you'd configure any forwarder.

[m_efremov]

Thank you for answer, it may be a good solution, but is there any native method? I have no forwarders on indexer cluster members, but _internal and _audit log from all of them accessible on all other indexers and from search head cluster members (that integrated with the indexer cluster).

[dveuve_splunk]

M_efremov, just configure an outputs.conf on your search heads to point to your indexers. There is no requirement for an external forwarder, and this is the native, best practice approach for Splunk. This will ensure that any logs, and also any summary indexed data, will be available everywhere you need it.

[yannK]

You can use the same outputs.conf you deployed on forwarders, or use the sh UI (receiving and forwarding)

Remember that you need to make sure that the destination indexes exists on the indexers (by example if you had custom summary indexes on the SH)

[m_efremov]

I will test this solution and report here about results, thanks again