Summary indexing

Nicholas_Key — Sun, 22 Aug 2010 06:41:03 GMT

Hi all,

Quick question about summary indexing:

I have this configuration in the savedsearches.conf

[esxtop_Group_Cpu_VM_SI]
action.summary_index = 1
action.summary_index._name = summary_vmware
cron_schedule = */5 * * * *
description = Summary index for esxtop_Group_Cpu (VM)
dispatch.earliest_time = -8m@m
dispatch.latest_time = -3m@m
displayview = flashtimeline
enableSched = 1
realtime_schedule = 0
request.ui_dispatch_view = flashtimeline
search = index="vmware" sourcetype="esxtop_Group_Cpu" | stats count by VM

After waiting for 5 minutes, I tried to search

index=”summary_vmware” source=”esxtop_Group_Cpu_VM_SI” | stats count by VM

But I’m seeing

VM      count
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM="net-cdp.4238", count=55     1
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM="net-cdp.4239", count=55     1
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM="vmware-vmkauthd.4241", count=110    1
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM="vobd.4230", count=55    1
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM="vobd.4231", count=55    1
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM=FT, count=110    1
00, search_name=esxtop_Group_Cpu_VM_SI, search_now=1282431600.000, info_min_time=1282431120.000, info_max_time=1282431420.000, info_search_time=1282431632.311, VM=LinuxTaskMemPool, count=110  1
...
...
...

How does the search string looks like to get the values of fieldname VM, eg VM="net-cdp.4238", VM="net-cdp.4239", VM="vmware-vmkauthd.4241", VM="vobd.4230", VM=FT

since

index=”summary_vmware” source=”esxtop_Group_Cpu_VM_SI” | stats count by VM

does not get the values of fieldname VM

Re: Summary indexing

gkanapathy — Sun, 22 Aug 2010 07:36:05 GMT

It would be helpful if you showed up the original data you are summarizing over.

It seems to me that your quotes are wrong in your reporting query, but maybe that's from some copy/paste to here.
Also, either (the former is probably preferable):
- Your summarizing search should use sistats instead of stats; or
- Your reporting search should use sum(count) instead of count

Nevertheless, you should be getting a correct field extraction. Sometimes, the field extraction rules for summary data have been overridden, possibly because the sourcetype has been changed from stash, or some host-based or source-based rule takes precedence and prevents it from running right.

I will also note that your schedule will generate summaries over time ranges from

01:02:00 - 01:06:59
01:07:00 - 01:11:59
01:12:00 - 01:16:59
01:17:00 - 01:21:59
...
etc.

which while correct, are not particularly great if you wanted to aggregate or timechart up to 15min or 1h intervals typically on a quarter-hour or hour. Better to line them up on 5 minute markers by changing the cron to 3,8,13,18,23,28,33,38,43,48,53,58 * * * *

topic Summary indexing in Getting Data In

Summary indexing

Re: Summary indexing