Splunk ITSI

Best Approach of adding KPI based on different entities

AKG11
Path Finder

Hi,

we are using ITSI Service map/Service Analyzer to monitor services.  we have an use case where for same service we need to add multiple KPI and those KPI depends on different entities.
For Example: We have Infrastructure related KPI which uses host as entity, another KPI is "service Up" which basically check service is up and in this case entity is "process name".  Also have KPI for Garbage collection which also has different entity.

Question: I am trying to understand which is the best way to handle such scenario. where we can add all these KPI without making service map too complex.

AKG11_0-1709737985253.png

 

Labels (2)
0 Karma

srauhala_splunk
Splunk Employee
Splunk Employee

Hi @AKG11 

* KPI which uses host as entity,

* KPI is "service Up" which basically check service is up and in this case entity is "process name". 

* KPI for Garbage collection which also has different entity.

You could have multiple different strategies for this. All does however sound like they host specific.  "process name" will be related to a host, so does the Garbage collection right? 

I would make the KPI searches for "service Up" and "Garbage collection" exposes the host for every result. Use the filter to entities in service by host, and use the split by on a pseudo-entites for example process_name or garbage_collection_name. 

Then the result will be that all KPIs are filtered to the entity (host) of the service and split by different entites per KPI. Note this will be a bit wonky if you have multiple hosts by service, for example host1, host2, host3. 

In that case, to be able to distinguish a process from one host to another, you would need to create a combination of host and process, etc to split the data by. I.e. | eval my_process_entity = process_name."-".host

Filter kpi by hosy 

Split kpi by my_process_entity

If this widely used use-case also consider creating real entities and entity types for this use-cases to be able to create entity dashboards. 

Hope this gives some ideas! 

Kind Regards, 

Seb 

0 Karma

AKG11
Path Finder

 @srauhala_splunk  Thanks for response

Q. You could have multiple different strategies for this. All does however sound like they host specific.  "process name" will be related to a host, so does the Garbage collection right?

A. That's not the scenario. One host could have multiple process running on it. Some times even same process running on multiple host. In that case we have to use combination of process and host as entity. As multiple process on one host hence multiple GC on same host.

We wanted to have process based service because one service could be dependent on another Service.

0 Karma

srauhala_splunk
Splunk Employee
Splunk Employee

@AKG11  

 

"One host could have multiple process running on it. Some times even same process running on multiple host"

 - Yes would expect that, and that would work fine with the example I provided you. 

You can make the service process centric if you want and scale the service tree by process. I.e. filter by process and split by process_name_host_name, and similar a separate garbage collector service. You can even have both i.e. host and process, gc entities in one service and filter by entity of either in the KPI.  

But to be honest that sounds a but wonky to me. but you know your services best 😉 

/Seb 

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...