Splunk Enterprise Security

Exclude index from ES dataset

chris
Motivator

Hi,

we are currently adding data sources to our Splunk environment. We try our best to make it CIM compliant. We have a dedicated ES search head and we do not want ES to look at this data. How can we make sure that it is excluded from ES. I'd rather not set up new dedicated indexers just for the new data since we would probably loose performance and the setup (and therefore maintenance) will become more complicated.

Thanks,
Chris

0 Karma
1 Solution

maciep
Champion

So you have a search head dedicated for just the ES app? And you have other search heads to use outside of ES? If that's the case, I'd say just don't put the config for your new sources on the ES search head. I think most CIM-compliance happens at search time, so if ES doesn't have the search time config for those new sources, then those fields shouldn't be available for the dm acceleration.

If that's not an option, because maybe the log data is cim compliant with just k/v extractions, then is it in a different index? I believe the latest version of the CIM app allows you to choose which indexes apply to a given data model. So if those new sources are in their own index, just uncheck that index for the datamodel config on the ES box.

Also, in ES you have the option to import apps. By default, if an app is named something like TA* or Splunk* or DA* (etc), it's automatically imported into ES. If you have apps not named that, you can tell ES the name of your app and will import it. Likewise, I believe you can also tell ES not to import an app if it meets the default naming convention. So if your config is in a separate, just tell ES to ignore it.

Hopefully I understood the problem correctly. And there may be better solutions, but that's what comes to mind for me.

View solution in original post

maciep
Champion

So you have a search head dedicated for just the ES app? And you have other search heads to use outside of ES? If that's the case, I'd say just don't put the config for your new sources on the ES search head. I think most CIM-compliance happens at search time, so if ES doesn't have the search time config for those new sources, then those fields shouldn't be available for the dm acceleration.

If that's not an option, because maybe the log data is cim compliant with just k/v extractions, then is it in a different index? I believe the latest version of the CIM app allows you to choose which indexes apply to a given data model. So if those new sources are in their own index, just uncheck that index for the datamodel config on the ES box.

Also, in ES you have the option to import apps. By default, if an app is named something like TA* or Splunk* or DA* (etc), it's automatically imported into ES. If you have apps not named that, you can tell ES the name of your app and will import it. Likewise, I believe you can also tell ES not to import an app if it meets the default naming convention. So if your config is in a separate, just tell ES to ignore it.

Hopefully I understood the problem correctly. And there may be better solutions, but that's what comes to mind for me.

chris
Motivator

The data has the same format/sourcetype as existing data that is relevant for ES but resides in a different index. I configured the data models in CIM to only include specific Indexes. Thanks a lot.

Regards
Chris

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...