I am a recent hire and am in a predicament. Our Splunk environment is pretty typical, there are clustered indexers/search heads. We have deployed SplunkESS and I am now in the phase where I want to start making our data useable and actionable. The issue I am having is that I am not entirely sure the best approach for adding new or missing data which our policy dictates we should have. For example, when I look at the cisco or palo alto source types, I see that they currently show that the data lives on the Search Head or rather a mounted NFS share of the ESS search head and NOT the indexers. Splunk ESS seems to come shipped with these sort of settings by default, all the apps look locally for data rather than the Indexers. What could be going on here and how do I fix this? Any help would be appreciated.
You don't want any search head (especially not the Enterprise Security one) to do the inputs (threat feeds aside, that's another story). If there's any way, let the NFS files/directories be monitored by any other Splunk instance.
A way to go would be: identify your use cases your management wants to have inside your SIEM. After that, you can start identifying your sources you will need and which data you need to normalize (CIM). Then you have done the (possibly) hardest work and "only" the correlation of your data is left.
not used that app, but i have feeling you should install it on the indexer AND the search head, but remove the inputs.conf from the search head. That way you get the data on your indexer and the UI stuff on the search head