How Splunk Searches tsidx files in getting results

[aashiqwork]

Hi All,

I am a newbie to Splunk Enterprise Security and currently I am trying my hands on Splunk ES to explore more on the SIEM areas.

I have couple of questions on basic working principles on CIM and correlation searches.

I understand once we normalize the incoming data according to CIM compatible. Splunk automatically links with the particular datamodel based on tags for example Malware_Attacks datamodel links the incoming data(Indexed and normalized data which is available in index named test) with tags malware and attack. and also the datamodel acceleration is enabled.

In this scenario for correlation searches the tstats command looks into the tsidx file to get the search results. My question here is how Splunk scans multiple indexes in my case the data is available in test index and there may be indexes called test1, test2 and all of these indexes has CIM compatible data for Malware.

Whether of all these data in each of the indexes compiled into one tsidx files for each datamodel or it uses different techniques to scan each of the indexes for getting the result.

Please correct if any of my above understanding is incorrect.

Your help is appreciated !!!

Thanks

Aashiq

[shivanshu1593 · Accepted solution]

I believe if you're getting data from a endpoint protection systen or an EDR, you should look into storing them in the same index, with different sourcetypes, unless you're ingesting sysmon, regmon, profmon logs etc, in which case different indexes for them would be fine.

To answer your question, if you want Splunk to search in multiple indexes for Malware related data to collect for your datamodel, you can always edit the CIM configuration, and under Malware, add the name of those indexes and click save.

After this change, Splunk will bag and tag the required data from all of your Indexes for you. Your tstats command should start giving you the accurate results. Hope this helps.

Thanks,

S

Disclamier: If it helps you, please accept it as a solution.

[richgalloway]

When you use tstats to search a data model, the DM knows which indexes to use and will open the appropriate tsidx files.