Splunk Search

timechart and trendline command

VijaySrrie
Builder

Hi All,

Need help with Timechart and trendline command for below query
Both timechart and trendline command are not working

index=_introspection sourcetype=splunk_resource_usage component=Hostwide
| eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct')
| stats Perc90(total_cpu_usage) AS cpu_usage latest(_time) as _time by Env Tenant
| timechart span=12h values(cpu_usage) as CPU
| trendline sma2(CPU) AS trend
Labels (1)
0 Karma
1 Solution

VijaySrrie
Builder
index=_introspection sourcetype=splunk_resource_usage host IN ("hostname" ) component=Hostwide
| eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct')
| eval Tenant=case(match(host,"name"),"Core",match(host,"name"),"Enterprise Security",match(host,"name"),"Critical Reports",match(host,"hostname"),"Mgmt",match(host,"hostname"),"IDX",match(host,"hostname"),"AWE",match(host,"hostname"),"ABC",1==1,host)
| eval Env=case(match(host,"hostname"),"Prod",match(host,"hostname"),"E2E",match(host,"hostname"),"ABC",1==1,splunk_server)
| fields host_zone Tenant _time total_cpu_usage
| table host_zone Tenant _time total_cpu_usage | search host_zone="pr" Tenant="Core"
| bin span=24h aligntime=@d
_time
| stats Perc90(total_cpu_usage) AS cpu_usage BY _time
| trendline sma2(cpu_usage) AS trend | fields * trend

View solution in original post

0 Karma

VijaySrrie
Builder
index=_introspection sourcetype=splunk_resource_usage host IN ("hostname" ) component=Hostwide
| eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct')
| eval Tenant=case(match(host,"name"),"Core",match(host,"name"),"Enterprise Security",match(host,"name"),"Critical Reports",match(host,"hostname"),"Mgmt",match(host,"hostname"),"IDX",match(host,"hostname"),"AWE",match(host,"hostname"),"ABC",1==1,host)
| eval Env=case(match(host,"hostname"),"Prod",match(host,"hostname"),"E2E",match(host,"hostname"),"ABC",1==1,splunk_server)
| fields host_zone Tenant _time total_cpu_usage
| table host_zone Tenant _time total_cpu_usage | search host_zone="pr" Tenant="Core"
| bin span=24h aligntime=@d
_time
| stats Perc90(total_cpu_usage) AS cpu_usage BY _time
| trendline sma2(cpu_usage) AS trend | fields * trend
0 Karma

VijaySrrie
Builder

I want to know how much CPU is utilized in our environment along with the trendline @ITWhisperer 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Please share some representative anonymised sample events in a code block

How often do you want to sample the cpu used?

Are Env and Tenant already extracted?

Do you want the stat broken down by Env and Tenant as well as time or some other dimensions?

0 Karma

VijaySrrie
Builder

@ITWhisperer Yes, env and tenant are already extracted, yes, we need stat broken by env and tenant as well as time

 

{"datetime":"08-19-2024 10:40:30.196 +1000","log_level":"INFO","component":"Hostwide","data":{"cpu_arch":"x86_64","os_name":"Linux","os_name_ext":"Linux","os_build":"#1 ABC Thu Apr 4 03:33:23 EDT 2024","os_version":"3.10.0-1160.118","instance_guid":"ABCDEFGH","splunk_version":"9.2.1","mem":"382641.051","mem_used":"41983.578","swap":"511.996","swap_used":"511.996","pg_paged_out":"50842005897","pg_swapped_out":"164124","forks":"00000600","cpu_count":"24","virtual_cpu_count":"48","runnable_process_count":"19","normalized_load_avg_1min":"1.14","cpu_user_pct":"45.35","cpu_system_pct":"10.68","cpu_idle_pct":"43.98"}}

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You can't timechart by more than 2 dimensions and _time is one of those, try combining Env and Tenant

index=_introspection sourcetype=splunk_resource_usage component=Hostwide
| eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct')
| eval EnvTenant=Env.":".Tenant
| timechart Perc90(total_cpu_usage) AS cpu_usage span=12h useother=f by EnvTenant

ITWhisperer
SplunkTrust
SplunkTrust

What is it you are trying to achieve?

At the moment, you are getting one stats result for each Env Tenant combination with the latest time stamp for that Env Tenant. This doesn't sound like something useful to timechart or trend.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...