Knowledge Management

is it right to use summary index to save non-statistical data?

RiccardoV
Communicator

Hi,
I'm using summary index, but I am not sure if I'm doing it right.
I have several searches that save data into my summary index. Some of them are saving statistical data, ie. how many events for each category I found.
But I need to speed up some different queries, where I need to display a table with many fields, ie. md5 - process - timestamp - category - etc.

Is it correct to run a scheduled query that saves data in summary index in that way?

Thanks!

edit: here is my search

sourcetype="tcp-raw" type=WSAEXEC OR type=WSAPROCESS
| eval ...
| search ...
| stats count ...
| stats dc(det) AS ...
| eval Category = ...
| table field1, field2, field3....

I did some changes, of couse 🙂

I retrieve the data in my dashboard in this way:

index="summary" 
| dedup field1, field2
| stats count by field1, field2
| sort -count
| head 10
| fields field1, field2, ....

thanks, again 🙂

edit #2:

here is my new search, only with streamable commands:

sourcetype="tcp-raw" type=WSAEXEC OR type=WSAPROCESS
| eval ...
| search ...
| stats count ...
| stats dc(det) AS ...
| eval Category = ...
| table field1, field2, field3....

I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?

How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?

thanks!

0 Karma
1 Solution

lguinn2
Legend

You said "I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?

How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?"

Answers:

First, you do not need to leave the start time and finish times empty in the time range. Splunk figures out how to accelerate date in the 1 month range automatically, regardless of the start and finish times.

Second, whenever you run this search, it will be accelerated. You don't need to do anything more; there are no special steps. In fact, if you run a similar search and Splunk can leverage the underlying acceleration summary, it will!

Finally, you shouldn't need to do any maintenance on the acceleration summary; Splunk will keep it valid and up to date.

View solution in original post

lguinn2
Legend

You said "I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?

How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?"

Answers:

First, you do not need to leave the start time and finish times empty in the time range. Splunk figures out how to accelerate date in the 1 month range automatically, regardless of the start and finish times.

Second, whenever you run this search, it will be accelerated. You don't need to do anything more; there are no special steps. In fact, if you run a similar search and Splunk can leverage the underlying acceleration summary, it will!

Finally, you shouldn't need to do any maintenance on the acceleration summary; Splunk will keep it valid and up to date.

RiccardoV
Communicator

thanks again for you answers @lguinn!

0 Karma

lguinn2
Legend

A search that contains a transaction command can't be accelerated. But I still think that report acceleration might be a better way to do this. Why are you using the transaction command? I think this might be optimized - quite a lot, actually.

RiccardoV
Communicator

Edit #2: I updated again my inizial search with only streamable commands and I accelerated my search..please have a look!

0 Karma

RiccardoV
Communicator

thanks again for your reply @lguinn!
I modified my search and now I'm 100% transaction-free 🙂

I updated the search in first post!

0 Karma

lguinn2
Legend

If you want to speed up searches, but not save statistical data, use report acceleration instead of summary indexing. That said, there are rules about which searches can be accelerated.

Can you show us the actual search that you want to run?

RiccardoV
Communicator

thanks @lguinn for your answer. I've just updated my question with the searches!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...