Splunk IT Service Intelligence

How to automatically create services in ITSI with correct entity filtering ?



Let me describe what i´m trying to do.
I have a base search with multiple KPIs to monitor K8S pods and clusters.
I have also 2 CSV import searches :
- One to create the pods entities with additional info field with format cluster_name#metadata.namespace.

index=payment_k8s sourcetype="kube:objects:pods" | rename metadata.name AS pod, metadata.namespace AS namespace | eval entity = cluster_name."#".namespace | fields pod entity
  • One for the list of services to be created.

    index=payment_k8s sourcetype="kube:objects:namespaces" status.phase="Active" | rename metadata.name as namespace | eval entity=cluster_name."#".namespace, ServiceTemplate="PAY:K8S", env=if(cluster_name="payment_bma","bg","ge"), ServiceTitle="PAY.".upper(env).".K8S.".namespace | stats latest(metadata.requestResourceVersion) by ServiceTemplate,ServiceTitle,entity,env

I have manually created a service and based on this service i created a service template named PAY:K8S. Each service should represent a K8S namespace. The entities in these services will be the pods belonging to the respective namespace.
In each service, i should have an Entity Rule like |Info| entity matches cluster_name#namespace (which is the "entity" field in the base search and in the first CSV Import search above).

I do not want to edit the match manually because i want to create the services automatically without having to go into every single service to update this Entity Rule.

How can i do that ?

0 Karma

Splunk Employee
Splunk Employee

When you're creating services via service template from a csv file, you can define an entity_rule column in the csv file and let ITSI populates the different entity rules per service.

But this would require you to configure the entity rule in the service template in such a way that the entity rules from the csv file are used during the service creation. You can refer to the following page for service template configuration:

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!