Splunk Search
Highlighted

How do we get our summary index search to produce our expected result?

Path Finder

Hi,

We are planning to implement summary indexing in our dashboards. As part of it, I have created a scheduled search below which would give us time by hosts, and have enabled summary indexing for this report.

Provided: Summaryindex name to : testhadoop
Added fields as: index=index1

index=index1 sourcetype="st:*:s1:s2" 
 | rex field=Real "(?<min>\d+)m" | rex field=Real "\d+m(?<sec>\d+)\.\d+" | rex field=Real "\d+m\d+\.(?<ms>\d+)" 
 | eval rt=min.":".sec.".".ms | convert mstime(rt) as seconds timeformat=%m:%s.%N 
 | eval seconds=round(seconds,2) | timechart avg(seconds) as secs by Host | eval Baseline=3|

and then we have created a dashboard with the search below:

index=index1 sourcetype="st:*:s1:s2" | rex field=Real "(?<min>\d+)m" 
 | rex field=Real "\d+m(?<sec>\d+)\.\d+" | rex field=Real "\d+m\d+\.(?<ms>\d+)" | eval rt=min.":".sec.".".ms | convert mstime(rt) as seconds timeformat=%m:%s.%N
 | eval seconds=round(seconds,2) | sitimechart avg(seconds) as secs by Host | eval Baseline=3|collect index=testhadoop testmode=true

But this is not retrieving us the result we want. Instead, it is inserting new fields where these fields are not available in our raw data, but when we use above search, these additional fields are being displayed:

psrsvd_ct_seconds   psrsvd_gc   psrsvd_nc_seconds   

Can someone please assist on it? We are actually new to Summary Indexing and planning to implement it.

Thanks in advance.

0 Karma
Highlighted

Re: How do we get our summary index search to produce our expected result?

SplunkTrust
SplunkTrust

This should be your summary index search (say saved search name is ssavgtimebyhost)

index=index1 sourcetype="st:*:s1:s2" 
  | rex field=Real "(?<min>\d+)m" | rex field=Real "\d+m(?<sec>\d+)\.\d+" | rex field=Real "\d+m\d+\.(?<ms>\d+)" 
  | eval rt=min.":".sec.".".ms | convert mstime(rt) as seconds timeformat=%m:%s.%N 
  | eval seconds=round(seconds,2) | sitimechart avg(seconds) as secs by Host 

This should be your dashboard search (which should use the data stored in summary index testhadoop)

index= testhadoop source=ss_avg_time_by_host | timechart avg(seconds) as secs by Host | eval Baseline=3

The collect command is for summary indexing from the search itself, don't run this command unless you want to add data from the search to specified index.

Highlighted

Re: How do we get our summary index search to produce our expected result?

SplunkTrust
SplunkTrust

Have a look at this video from Splunk for more info http://www.splunk.com/view/SP-CAAACZW

0 Karma
Highlighted

Re: How do we get our summary index search to produce our expected result?

Path Finder

Thanks Somesh,Sure will go through it.

Normally this should look like this when you run with out summary indexing commands, this would list time and each host with the values, so that we can easily create a tme line by host.

_time   host1   host2   host3
time          0.1sec    0.9            0.1

But when I have run the saved search with summaryindex, I was getting below fields(may be in appropriate)

host    _time   psrsvd_ct_seconds   psrsvd_gc   psrsvd_nc_seconds   psrsvd_sm_seconds   psrsvd_v

Since we have these many fields and no appropriate field to opt for seconds value to get in to a dashboard.

Thanks

0 Karma
Highlighted

Re: How do we get our summary index search to produce our expected result?

SplunkTrust
SplunkTrust

These are fields created by summary index specific commads (sistats/sichart/sitimechart). The benefit of using si* commands is that it will allow your run the same command that you used for summary index. Try the command I gave you.

| timechart avg(seconds) as secs by Host

Alternatively, you can update your summary index search to use regular commands. And then it will saved the fields as it is found in results.

This should be your summary index search (say saved search name is ssavgtimebyhost)

 index=index1 sourcetype="st:*:s1:s2" 
   | rex field=Real "(?<min>\d+)m" | rex field=Real "\d+m(?<sec>\d+)\.\d+" | rex field=Real "\d+m\d+\.(?<ms>\d+)" 
   | eval rt=min.":".sec.".".ms | convert mstime(rt) as seconds timeformat=%m:%s.%N 
   | eval seconds=round(seconds,2) | timechart avg(seconds) as secs by Host 

This should be your dashboard search (which should use the data stored in summary index testhadoop) (check the field name is changed in timechart)

 index= testhadoop source=ss_avg_time_by_host | timechart avg(secs) as secs by Host | eval Baseline=3
0 Karma
Highlighted

Re: How do we get our summary index search to produce our expected result?

Path Finder

Got you, So I can be able to view results in dashboards only after the summary is created?(If I schduled search to run for every one hour) ,after one hour the summary would created in testhadoop summary index and then only I can view results in dashboard(as of now I could see no results found for my dashboard query).

Thanks

0 Karma
Highlighted

Re: How do we get our summary index search to produce our expected result?

Path Finder

Hi Somesh,

Now I understood the concept, we got results for summary index.

But there is an search results issue here,
When we saving my search like below ,now results are separated by Hosts.
...| timechart avg(seconds) as secs by Host

In Summary Index dashboard all the hosts are reporting as individual fields instead to be Host.
and since dashboard is not displaying results by Hosts?

Is there any way can handle this scenario?

here is how event looks in Summary Index, now Host (in scheduled search ) has become field.

   07/29/2016 12:55:00 -0700, search_name=hadoop_usage_Testing, search_now=1469822400.000, info_min_time=1469818800.000, info_max_time=1469822400.000, info_search_time=1469822417.931, 
       **lgasvdv="0.130000"**

Thanks

0 Karma