Splunk Search

Multiple queries.

biec1
Explorer

index=index_name earliest=-30m@m latest=now
| stats latest(_time) as _time avg(cpu_usage) as cpu_usage by host
| eval var1=if(cpu_usage > 30,0,1)
| table host var1 cpu_usage _time

This query gets the average cpu utilization for the past 30 minutes.
I would like to create the list of high cpu utilization hosts for the past 12 hours, based on the var1 result of above 30 minutes average.

Tags (1)
0 Karma
1 Solution

DalJeanis
Legend

We've modified your code to run for each half hour period across 12 hours, and to output var1=1 when High...

index=index_name earliest=-12h@h latest=@30m
| bin _time span=30m
| stats avg(cpu_usage) as cpu_usage by host _time 
| eval var1=if(cpu_usage > 30,1,0) 
| table host var1 cpu_usage _time

Then, if you want ONLY the periods of high utilization, use...

| where var1==1

Then, if you want ONLY a list of the CPUs that were High at some point in the last 12 hours, and the record for the latest _time they were High, use...

| where var1==1
| sort 0 -_time
| dedup host 

Or, if you want to see the entire timeline for the last 12 hours for every CPU that reached "High" at any point in that time, use...

| eventstats sum(var1) as HighPeriodCount
| where HighPeriodCount>0

View solution in original post

0 Karma

Richfez
SplunkTrust
SplunkTrust

You have most of this answer already. You just need to extend your time period to 12 hours, bin the results into 30 minute buckets before your stats, then do your stats by both host AND time.

Reformatting for better readability (since it's now longer).

index=index_name earliest=-12h@h latest=now 
| bin span=30m _time 
| stats latest(_time) as _time avg(cpu_usage) as cpu_usage by host _time 
| eval var1=if(cpu_usage > 30,0,1) 
| table host var1 cpu_usage _time

Give that a shot, let us know how well it works!

Happy Splunking,
Rich

0 Karma

DalJeanis
Legend

@rich7177

| stats latest(_time) as _time ... by host _time

???

0 Karma

DalJeanis
Legend

We've modified your code to run for each half hour period across 12 hours, and to output var1=1 when High...

index=index_name earliest=-12h@h latest=@30m
| bin _time span=30m
| stats avg(cpu_usage) as cpu_usage by host _time 
| eval var1=if(cpu_usage > 30,1,0) 
| table host var1 cpu_usage _time

Then, if you want ONLY the periods of high utilization, use...

| where var1==1

Then, if you want ONLY a list of the CPUs that were High at some point in the last 12 hours, and the record for the latest _time they were High, use...

| where var1==1
| sort 0 -_time
| dedup host 

Or, if you want to see the entire timeline for the last 12 hours for every CPU that reached "High" at any point in that time, use...

| eventstats sum(var1) as HighPeriodCount
| where HighPeriodCount>0
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Telemetry Pipeline Management

Balancing Scale and Spend: Gaining Control Over High-Volume Metrics in Splunk Observability Cloud As ...

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...