Splunk Search

Multiple queries.

biec1
Explorer

index=index_name earliest=-30m@m latest=now
| stats latest(_time) as _time avg(cpu_usage) as cpu_usage by host
| eval var1=if(cpu_usage > 30,0,1)
| table host var1 cpu_usage _time

This query gets the average cpu utilization for the past 30 minutes.
I would like to create the list of high cpu utilization hosts for the past 12 hours, based on the var1 result of above 30 minutes average.

Tags (1)
0 Karma
1 Solution

DalJeanis
Legend

We've modified your code to run for each half hour period across 12 hours, and to output var1=1 when High...

index=index_name earliest=-12h@h latest=@30m
| bin _time span=30m
| stats avg(cpu_usage) as cpu_usage by host _time 
| eval var1=if(cpu_usage > 30,1,0) 
| table host var1 cpu_usage _time

Then, if you want ONLY the periods of high utilization, use...

| where var1==1

Then, if you want ONLY a list of the CPUs that were High at some point in the last 12 hours, and the record for the latest _time they were High, use...

| where var1==1
| sort 0 -_time
| dedup host 

Or, if you want to see the entire timeline for the last 12 hours for every CPU that reached "High" at any point in that time, use...

| eventstats sum(var1) as HighPeriodCount
| where HighPeriodCount>0

View solution in original post

0 Karma

Richfez
SplunkTrust
SplunkTrust

You have most of this answer already. You just need to extend your time period to 12 hours, bin the results into 30 minute buckets before your stats, then do your stats by both host AND time.

Reformatting for better readability (since it's now longer).

index=index_name earliest=-12h@h latest=now 
| bin span=30m _time 
| stats latest(_time) as _time avg(cpu_usage) as cpu_usage by host _time 
| eval var1=if(cpu_usage > 30,0,1) 
| table host var1 cpu_usage _time

Give that a shot, let us know how well it works!

Happy Splunking,
Rich

0 Karma

DalJeanis
Legend

@rich7177

| stats latest(_time) as _time ... by host _time

???

0 Karma

DalJeanis
Legend

We've modified your code to run for each half hour period across 12 hours, and to output var1=1 when High...

index=index_name earliest=-12h@h latest=@30m
| bin _time span=30m
| stats avg(cpu_usage) as cpu_usage by host _time 
| eval var1=if(cpu_usage > 30,1,0) 
| table host var1 cpu_usage _time

Then, if you want ONLY the periods of high utilization, use...

| where var1==1

Then, if you want ONLY a list of the CPUs that were High at some point in the last 12 hours, and the record for the latest _time they were High, use...

| where var1==1
| sort 0 -_time
| dedup host 

Or, if you want to see the entire timeline for the last 12 hours for every CPU that reached "High" at any point in that time, use...

| eventstats sum(var1) as HighPeriodCount
| where HighPeriodCount>0
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...