How to create a summary index?

[mm12]

Hi ,

I want to create summary index for the below OS metrics process . How to achieve this.

 1.Avg CPU per week*
 2.Avg memory per week*
 3. Avg /var/log/ % used, per week*
 4. # processes running, per week*

Thanks

[gcusello]

Hi @mm12,

at first you have to index metrics from your servers using dedicated Add-Ons like Splunk_TA_nix (https://splunkbase.splunk.com/app/833/) for Linux systems or Splunk_TA_Windows (https://splunkbase.splunk.com/app/742/) for Windows Servers.

Then you you could schedule every week one or (better) more searches to record in a summary index (called e.g. summary_statistics) the average calculation for each metric, like the following:

index=os sourcetype=cpu
| stats earliest(_time) AS _time avg(cpu) AS value BY host
| eval metric="cpu_utilization"
| table _time host metric value 
| collect index=summary_statistics

or 

index=os sourcetype=top
| stats earliest(_time) AS _time avg(memory) AS value BY host
| eval metric="memory"
| table _time host metric value 
| collect index=summary_statistics

etc...

In this way you'll have in the summary index four informations:

• date (only one for each week),
• host,
• metric, the metric you want to analyze,
• value, the value for the metric.

Then you can run a search on the summary index:

index=summary_statistics
| bin _time span=1w
| eval TimeStamp=strftime(_time),"%Y-%m-%d")
| table Timestamp host metric value

About the searches to schedule, I hint to se in dedicated apps like Splunk App for Linux and Unix (https://splunkbase.splunk.com/app/273/) to have all the parametrs to use in the searches.

 Ciao.

Giuseppe

[mm12]

@gcusello 

| mstats avg(cpu_metric.pctIdle) AS "cpu_idle" WHERE index=os_metric bY host
| search
[| inputlookup lookup_mapping
| search service="os_metric" host= * | table host]
| eval "cpu_usage"=round(100-cpu_idle, 2)
| sitimechart avg(cpu_usage) as "cpu_usage" by host

I used the above query to check the CPU usage and schedule it as report and enable the summary Metric index
I got three fields like psrsvd_ct_cpu_usage, psrsvd_nc_cpu_usage and
psrsvd_sm_cpu_usage. Now I need to query the summary index to use it in dashboard. How to achieve that.

Can anyone help me on this please.

[mm12]

@ITWhisperer  I can able to write separate queries for example 

For Avg CPU

| mstats avg(cpu_metric.pctIdle) AS "cpu_idle" WHERE index=linux_os_metrics by host CPU span=1m
| eval "cpu%_Utilized"=round(100-cpu_idle, 2)
|search CPU=all |fields - CPU cpu_idle | timechart span=1m avg(cpu%_Utilized) by host

But how to combine that in summary index

Thanks

Manju

[ITWhisperer]

It depends what it is you want in your summary index

You could have a report for each metric and add them all to the same summary index as events for each minute. You could distinguish which metric came from which report using the search_name added to the event by the summary index process

If you want a single event per minute with all the stats for all the hosts, you need to construct a report that gives you that information, then schedule it and enable summary indexing to the summary index of your choice. Note that summary indexes have to be created before they can be used.

[ITWhisperer]

Start by creating a report that provides that information from your events