Getting Data In

Would a summary index be the best way to keep an index of all the outputs of a savedsearch?

_jgpm_
Communicator

I have a saved search that is being run through my dashboard with a text input using the "$token$" operator. I would like to keep the output of the saved search for a historical record. Would the best way to store this data be passing the output to a summary index? I would eventually like to search and report of this new index of data.

The output of the saved search is a N x {5,} matrix of mixed data types.

Thank you!

0 Karma
1 Solution

esix_splunk
Splunk Employee
Splunk Employee

It depends on your specific use case and requirements. The issue with summary indexes is that if you miss / skip a saved populating search, you will have gaps in your data. If you are aware of these limitations, it should work fine.

Another potential method might be to create a data model and use that. There is no need to backfill on these as they harvest and populate every 5 minutes. However, depending on your base populating search, you may not be able to use data models.

View solution in original post

esix_splunk
Splunk Employee
Splunk Employee

It depends on your specific use case and requirements. The issue with summary indexes is that if you miss / skip a saved populating search, you will have gaps in your data. If you are aware of these limitations, it should work fine.

Another potential method might be to create a data model and use that. There is no need to backfill on these as they harvest and populate every 5 minutes. However, depending on your base populating search, you may not be able to use data models.

_jgpm_
Communicator

Thanks for the answer. At this point, I don't fully know what my use cases are going to be. With respect to your issue, my concern is more around how my saved searches will evolve over time. I know I am very far away from a mature saved search and if I start using version 1.0 to populate the SI, I fear that the SI content will differ greatly over time. I guess I could always embed version data into the SI events and use that to regenerate fresh SI content from the recent version saved search if it is necessary. The requirement would be to retain the raw data prior to the saved search and overwrite the old version content in the SI. Is that possible? Is it like the current index behavior where all I have to do is

search OldVersionID | delete | append [ search NewVersionID ]

Thank you.

0 Karma
Get Updates on the Splunk Community!

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...

From Alert to Resolution: How Splunk Observability Helps SREs Navigate Critical ...

It's 3:17 AM, and your phone buzzes with an urgent alert. Wire transfer processing times have spiked, and ...