Getting Data In

Monitor CPU cores of a Linux machine

hishamjan
Explorer

Hi, 

 

I have a Linux machine running on Centos 6.10 with a quad-core processor  (16 threads)

 

hishamjan_0-1617256707290.png

On Splunk, is there a way to monitor each of its threads? Because sometimes, the threads reach 100% and my system stops working until the threads are down to the current percentage values as in the above screenshot.

Any degree of help will be appreciated.

 

Regards.

0 Karma

jokertothequinn
Path Finder

This will do the trick:

| mstats avg(cpu_metric.*) as cpu_* WHERE index=<your_metrics_index> by CPU, host
| table CPU, host
| eventstats max(CPU) as cpu_count by host
| table cpu_count, host
| eval cpu_count=cpu_count+

 

the data being used is from the add on Link to the splunk add on for Splunk Add-on for Unix and Linux docs 

0 Karma

jokertothequinn
Path Finder

This is a query which can get you the cpu core for both nix and win servers:

 

| mstats avg(Processor.*) as * WHERE (index=win-metrics) instance!="_Total" host="***"  by host instance span=5m
| table _time, host, instance, "%_Processor_Time"
| stats max(instance) as "cpu_core" by host
| eval cpu_core=cpu_core + 1
| append
[| mstats avg(cpu_metric.*) as cpu_* WHERE (index=nix-metrics) host="***" by CPU, host
| table CPU, host
| eventstats max(CPU) as cpu_core by host
| stats max(cpu_core) as cpu_core by host
| eval cpu_core=cpu_core + 1 ]
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hishamjan,

yes, you can monitor everything,

You can use the Splunk AddOn for Unix and Linux (https://splunkbase.splunk.com/app/833/) to take the logs and the Splunk App for Unix and Linux (https://splunkbase.splunk.com/app/273/) to display information.

If instead you want a custom control, you can create a custom scripted input using the command you used for the screenshot (https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptedInputsIntro) to index the values and you can crete your own dashboard.

Anyway I hint to try with the splunkbase apps before to start to develop a custom one.

Ciao.

Giuseppe

0 Karma

hishamjan
Explorer

hi @gcusello 

sorry for the late response, but my Linux machine already has Splunk AddOn for Unix installed.

 

Is it possible to monitor its cores (threads)?

 

Thanks

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hishamjan,

in the Splunk AddOn for Unix there are some scripts that run Linux commands as  cpu or iostat or others: they, when enabled, take stats on the cpu utilization.

I don't know the command you used in your screenshot, but if it's one of the already presente in app scripts you can take results only enabling it.

If instead the command you used is another one, you can see how scripts are done in that app and create your own scripted input using your command.

Anyway, in few words:

  • create in the $SPLUNK_HOME/etc/apps/Splunk_TA_nix/bin the file your_command.sh;
  • in the your_command.sh file put the command with all the options you want;
  • enable it to be executed;
  • create a stanza in inputs.conf (the one in local folder not in default) defining the frequency execution;
  • restart splunk on the machine or deploy it using the Deployment Server.

the script is very easy, it contains only the command as you run it in SSH.

the inputs.conf stanza is something like this:

 

[script://./bin/your_command.sh]
sourcetype = cpu_usage
source = your_command
interval = 300
disabled = 0

 

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

&#x1f342; Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...