topic Re: timechart and trendline command in Splunk Search

timechart and trendline command

VijaySrrie — Tue, 13 Aug 2024 12:08:40 GMT

Hi All,

Need help with Timechart and trendline command for below query
Both timechart and trendline command are not working

index=_introspection sourcetype=splunk_resource_usage component=Hostwide | eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct') | stats Perc90(total_cpu_usage) AS cpu_usage latest(_time) as _time by Env Tenant | timechart span=12h values(cpu_usage) as CPU | trendline sma2(CPU) AS trend

Re: timechart and trendline command

ITWhisperer — Tue, 13 Aug 2024 13:23:22 GMT

What is it you are trying to achieve?

At the moment, you are getting one stats result for each Env Tenant combination with the latest time stamp for that Env Tenant. This doesn't sound like something useful to timechart or trend.

Re: timechart and trendline command

VijaySrrie — Tue, 13 Aug 2024 23:27:48 GMT

I want to know how much CPU is utilized in our environment along with the trendline @ITWhisperer

Re: timechart and trendline command

ITWhisperer — Wed, 14 Aug 2024 07:24:07 GMT

Please share some representative anonymised sample events in a code block

How often do you want to sample the cpu used?

Are Env and Tenant already extracted?

Do you want the stat broken down by Env and Tenant as well as time or some other dimensions?

Re: timechart and trendline command

VijaySrrie — Mon, 19 Aug 2024 00:46:00 GMT

@ITWhisperer Yes, env and tenant are already extracted, yes, we need stat broken by env and tenant as well as time

{"datetime":"08-19-2024 10:40:30.196 +1000","log_level":"INFO","component":"Hostwide","data":{"cpu_arch":"x86_64","os_name":"Linux","os_name_ext":"Linux","os_build":"#1 ABC Thu Apr 4 03:33:23 EDT 2024","os_version":"3.10.0-1160.118","instance_guid":"ABCDEFGH","splunk_version":"9.2.1","mem":"382641.051","mem_used":"41983.578","swap":"511.996","swap_used":"511.996","pg_paged_out":"50842005897","pg_swapped_out":"164124","forks":"00000600","cpu_count":"24","virtual_cpu_count":"48","runnable_process_count":"19","normalized_load_avg_1min":"1.14","cpu_user_pct":"45.35","cpu_system_pct":"10.68","cpu_idle_pct":"43.98"}}

Re: timechart and trendline command

ITWhisperer — Mon, 19 Aug 2024 09:06:24 GMT

You can't timechart by more than 2 dimensions and _time is one of those, try combining Env and Tenant

index=_introspection sourcetype=splunk_resource_usage component=Hostwide | eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct') | eval EnvTenant=Env.":".Tenant | timechart Perc90(total_cpu_usage) AS cpu_usage span=12h useother=f by EnvTenant

Re: timechart and trendline command

VijaySrrie — Fri, 23 Aug 2024 10:52:37 GMT

index=_introspection sourcetype=splunk_resource_usage host IN ("hostname" ) component=Hostwide | eval total_cpu_usage=('data.cpu_system_pct' + 'data.cpu_user_pct') | eval Tenant=case(match(host,"name"),"Core",match(host,"name"),"Enterprise Security",match(host,"name"),"Critical Reports",match(host,"hostname"),"Mgmt",match(host,"hostname"),"IDX",match(host,"hostname"),"AWE",match(host,"hostname"),"ABC",1==1,host) | eval Env=case(match(host,"hostname"),"Prod",match(host,"hostname"),"E2E",match(host,"hostname"),"ABC",1==1,splunk_server) | fields host_zone Tenant _time total_cpu_usage | table host_zone Tenant _time total_cpu_usage | search host_zone="pr" Tenant="Core" | bin span=24h aligntime=@d _time | stats Perc90(total_cpu_usage) AS cpu_usage BY _time | trendline sma2(cpu_usage) AS trend | fields * trend