All Apps and Add-ons

SAI + Nmon (Unix hosts) - entities are not showing up

alexeyglukhov
Path Finder

Hello all !
I did read all the missing entities cases here, but mine is slightly different.
I am trying to feed metrics from Unix Nmon into SAI and those metrics are coming into metrics index.

I configured needed parameters:
_meta = entity_type::nix_host in inputs.conf
nmon-metrics index added into SAI indexes macro
sourcetype = em_metrics is host assignment (MetaData:Host) - it is done my Nmon's TRANSFORMS-hostfield=nmon_metrics_csv_hostoverride

Windows entities (perfmon config for metrics) appeared in SAI, but not Unix ones.

I am attaching a few screenshots to compare what metrics coming from Windows and Unix - my suspicion is:
SAI expects certain metric naming convention
SAI expects additional mandatory dimensions (other than entity_type::nix_host)
or something else

Any help will be appreciated.
alt text

alt text

0 Karma
1 Solution

alexeyglukhov
Path Finder

Ok, everyone !

Research showed that SAI has two saved searches:
for Windows hosts - "Entity Class - perfmon", which searches for metrics with "Processor." or "processor."

| mcatalog values("host") as "dimension.identifier.host" [ mcatalog values(_dims) as info where **metric_name=Processor.* OR metric_name=processor.*** AND (`sai_metrics_indexes`)...

for Unix hosts - "Entity Class - os", which searches for metrics with "cpu.*"

| mcatalog values("host") as "dimension.identifier.host" [ mcatalog values(_dims) as info where **metric_name=cpu.*** AND (`sai_metrics_indexes`)...

So, add cpu. prefix to your custom metrics (in my case I modified Nmon parser scripts)
It can be only CPU related metrics or all - up to you, but I assume if CPU metrics stopped coming, after some time that entity might get status either disabled or disappear even though other metrics will still be coming.

Or modify saved search to include your metrics' prefix.

Happy splunking 🙂

View solution in original post

0 Karma

alexeyglukhov
Path Finder

Ok, everyone !

Research showed that SAI has two saved searches:
for Windows hosts - "Entity Class - perfmon", which searches for metrics with "Processor." or "processor."

| mcatalog values("host") as "dimension.identifier.host" [ mcatalog values(_dims) as info where **metric_name=Processor.* OR metric_name=processor.*** AND (`sai_metrics_indexes`)...

for Unix hosts - "Entity Class - os", which searches for metrics with "cpu.*"

| mcatalog values("host") as "dimension.identifier.host" [ mcatalog values(_dims) as info where **metric_name=cpu.*** AND (`sai_metrics_indexes`)...

So, add cpu. prefix to your custom metrics (in my case I modified Nmon parser scripts)
It can be only CPU related metrics or all - up to you, but I assume if CPU metrics stopped coming, after some time that entity might get status either disabled or disappear even though other metrics will still be coming.

Or modify saved search to include your metrics' prefix.

Happy splunking 🙂

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...