Hi,
I'm using summary index, but I am not sure if I'm doing it right.
I have several searches that save data into my summary index. Some of them are saving statistical data, ie. how many events for each category I found.
But I need to speed up some different queries, where I need to display a table with many fields, ie. md5 - process - timestamp - category - etc.
Is it correct to run a scheduled query that saves data in summary index in that way?
Thanks!
edit: here is my search
sourcetype="tcp-raw" type=WSAEXEC OR type=WSAPROCESS
| eval ...
| search ...
| stats count ...
| stats dc(det) AS ...
| eval Category = ...
| table field1, field2, field3....
I did some changes, of couse 🙂
I retrieve the data in my dashboard in this way:
index="summary"
| dedup field1, field2
| stats count by field1, field2
| sort -count
| head 10
| fields field1, field2, ....
thanks, again 🙂
edit #2:
here is my new search, only with streamable commands:
sourcetype="tcp-raw" type=WSAEXEC OR type=WSAPROCESS
| eval ...
| search ...
| stats count ...
| stats dc(det) AS ...
| eval Category = ...
| table field1, field2, field3....
I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?
How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?
thanks!
You said "I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?
How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?"
Answers:
First, you do not need to leave the start time and finish times empty in the time range. Splunk figures out how to accelerate date in the 1 month range automatically, regardless of the start and finish times.
Second, whenever you run this search, it will be accelerated. You don't need to do anything more; there are no special steps. In fact, if you run a similar search and Splunk can leverage the underlying acceleration summary, it will!
Finally, you shouldn't need to do any maintenance on the acceleration summary; Splunk will keep it valid and up to date.
You said "I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?
How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?"
Answers:
First, you do not need to leave the start time and finish times empty in the time range. Splunk figures out how to accelerate date in the 1 month range automatically, regardless of the start and finish times.
Second, whenever you run this search, it will be accelerated. You don't need to do anything more; there are no special steps. In fact, if you run a similar search and Splunk can leverage the underlying acceleration summary, it will!
Finally, you shouldn't need to do any maintenance on the acceleration summary; Splunk will keep it valid and up to date.
thanks again for you answers @lguinn!
A search that contains a transaction
command can't be accelerated. But I still think that report acceleration might be a better way to do this. Why are you using the transaction command? I think this might be optimized - quite a lot, actually.
Edit #2: I updated again my inizial search with only streamable commands and I accelerated my search..please have a look!
thanks again for your reply @lguinn!
I modified my search and now I'm 100% transaction-free 🙂
I updated the search in first post!
If you want to speed up searches, but not save statistical data, use report acceleration instead of summary indexing. That said, there are rules about which searches can be accelerated.
Can you show us the actual search that you want to run?
thanks @lguinn for your answer. I've just updated my question with the searches!