Splunk Search

Need help in writing a query in Splunk to display the CPU Utilization of the Splunk forwarder (Universal) resources/process running on a Windows Machine

dnvs007
New Member

Logs i am receiving from the Windows machine are like below: (Getting just the "instance=_Total" logs)

02/09/2018 23:07:04.086 +1000
collection=CPUTime
object=Processor
counter="% User Time"
instance=_Total
Value=1.8358905012031694

Please help me write a query to address the issue ASAP.

0 Karma
1 Solution

manish_singh_77
Builder

Hi dnvs007,

Please try the below query and let me know if it works or not. Since we have the same index so we can use this search query..

index=os| search COMMAND="splunk*" OR process_name="splunk*" | eval CPU_Percent=coalesce(PercentProcessorTime,pctCPU) | timechart span=15m max(CPU_Percent) by host useother=f | sort _time

View solution in original post

0 Karma

manish_singh_77
Builder

Hi dnvs007,

Please try the below query and let me know if it works or not. Since we have the same index so we can use this search query..

index=os| search COMMAND="splunk*" OR process_name="splunk*" | eval CPU_Percent=coalesce(PercentProcessorTime,pctCPU) | timechart span=15m max(CPU_Percent) by host useother=f | sort _time

0 Karma

manish_singh_77
Builder

index=os sourcetype=foo OR sourcetype=coo host=$hostname$ | multikv fields pctCPU COMMAND PercentProcessorTime Name |eval pctCPU="PercentProcessorTime" | bucket span=1d _time | stats avg(Value) as pctCPU by _time host | where pctCPU>75.0| timechart span=1d avg(avgcpu) as "CPU Processor Time" by host limit=0

try this and let me know...

0 Karma

dnvs007
New Member

Thanks Manish.. Will definitely try and get back... Also just to clarify, will the multisearch usage not work?

0 Karma

manish_singh_77
Builder

It is not required..

0 Karma

dnvs007
New Member

Thanks again Manish! All i want from the query is for it to display on my dashboard the usage of Splunk resource utilization of the CPU(splunkd) on each server(Windows/Linux) in just one query...

0 Karma

dnvs007
New Member

Hi Manish,

That did not work... maybe i should have explained better..

I have multiple Linux hosts and Multiple Windows hosts..

Linux hosts come with index=os and fields pctCPU and COMMAND

However, windows hosts come with fields PercentProcessorTime and Name(which has splunkd info)

i have tokens created for all these hosts as you already know by "$hostname$

I now need a query which will show how much CPU Splunk is utilizing on these servers and display a graph on my dashboard.

Please help. Quick help much appreciated...

0 Karma

dnvs007
New Member

To Add i am using the query:
index=os sourcetype=foo OR sourcetype=coo host=$hostname$ | multikv fields pctCPU COMMAND PercentProcessorTime Name | multisearch [ search host=$hostname$ COMMAND="splunk*"] [search host=$hostname$ Name="splunk*" | eval pctCPU="PercentProcessorTime"] | timechart span=15m max(pctCPU) by host | sort _time

As posted earlier, gives me results for Linux hosts but not for Windows hosts.

Please help

0 Karma

manish_singh_77
Builder

what information you have in the below fields, could you please share..

pctCPU
COMMAND
PercentProcessorTime
Name

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What have you tried so far and what results did you get?

---
If this reply helps you, Karma would be appreciated.
0 Karma

dnvs007
New Member

Hello there!

Tried below and variations of below:

earliest=-15m sourcetype="Perfmon:CPU" counter="% Processor Time" | where (instance!="Idle" AND instance!="System") | stats avg(Value) by host,instance

even though does not exactly meet my requirement.

Nothing works because, i am getting logs only of "instance=_Total"

Please let me know if you need more details.

Thanks a ton! Greatly appreciate a quick response.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

No SPL will help you find what is not indexed. If you need something other than instance=_Total then you should take steps to get it.

Based on what you have, you may get results with this query.

index=foo sourcetype="Perfmon:CPU" instance=_Total | stats avg(Value) by host
---
If this reply helps you, Karma would be appreciated.
0 Karma

dnvs007
New Member

Thank you Rich Galloway! Sorry for the late response...

I got the processes for the Windows servers indexed. Could you please help me merge a query for Linux with Windows ?

The query i use is as below:
index=os sourcetype=foo OR sourcetype=coo host=$hostname$ | multikv fields pctCPU COMMAND | search host=$hostname$ COMMAND="splunk*" | timechart span=15m max(pctCPU) by host | sort _time

The indexed fields for windows is: PercentTime for pctCPU and Name for Command.

0 Karma

dnvs007
New Member

To add on i will need the graph to either show max values for Linux or for windows at a time in a consolidated query

0 Karma

dnvs007
New Member

Came up with this query:

index=os sourcetype=foo OR sourcetype=coo host=$hostname$ | multikv fields pctCPU COMMAND PercentProcessorTime Name | multisearch [ search host=$hostname$ COMMAND="splunk*"] [search host=$hostname$ Name="splunk*" | eval pctCPU="PercentProcessorTime"] | timechart span=15m max(pctCPU) by host | sort _time

Could someone please help in letting me know if this is the right query to get results as i need it to be?

0 Karma
Get Updates on the Splunk Community!

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...