topic Re: Need help in Summary Indexing in Knowledge Management

Need help in Summary Indexing

theouhuios — Mon, 07 Jan 2013 21:21:12 GMT

Hello

I am using a summary indexing to get a chart but it doesn't look like its helping much. I need to have a chart of counts of incidents in last 31days and the search I had before was probably not helping much as I did it on time.

It doesn't help much as there will be incidents on different time. Would it help if I summarize it for one day and run at midnight to collect the results of the previous day and just use that?

Any ideas?

Re: Need help in Summary Indexing

MarioM — Tue, 08 Jan 2013 06:13:47 GMT

what you could try is to use bucket :

sourcetype=xxx| dedup record.incidentId | bucket _time span=1d |sistats count by _time record.priority record.groupArea record.assignmentGroup record.groupDepartment record.affectedCI

Re: Need help in Summary Indexing

Drainy — Tue, 08 Jan 2013 08:19:20 GMT

To offer an alternative to the right Honourable MarioM 🙂

Just run it after midnight for the previous day, every day.
To create your first set of data just run the backfill script to populate your index with the last X days of data and from then on it will update each night. Usually I run SI searches for the previous day at around 1am just so I'm happy all events have arrived and because its a nice round number 🙂

E.G. Each night run this for the previous day;

sourcetype=xxx earliest=-1d@d latest=-0d@d | dedup record.incidentId|sistats count by record.priority record.groupArea record.assignmentGroup record.groupDepartment record.affectedCI

Backfill with;

./splunk cmd python fill_summary_index.py -app APPNAME -name "SAVEDSEARCHNAME" -et EARLIESTTIMEMODIFIER -lt LATESTTIMEMODIFER -j 2 -owner SEARCHOWNER

It will prompt for user details when run and the timemodifers are the usual splunk ones, to backfill for a month you can use -et -1mon@mon -lt -0d@d <-- I say one day as your new scheduled search will pick up the current days events when it executes.

The -j flag sets how many concurrent searches it runs, I have put 2 but you can increase this if you have the spec and its not a busy box.

Re: Need help in Summary Indexing

theouhuios — Tue, 08 Jan 2013 15:07:52 GMT

I did think of doing this, but it will kill the box with the cpu usage 🙂

Re: Need help in Summary Indexing

Drainy — Tue, 08 Jan 2013 18:10:56 GMT

How will it kill the box? this is how you do summary indexing 🙂 If you are talking about the backfill then you don't have to backfill at all, if you did want to you could set it to one search and leave it overnight