Splunk ITSI

How to manipulate metric values so that Service Health Score can be changed on Ad-Hoc basis for demonstrations?

Chiranjit-Mitra
Loves-to-Learn

I've got the service monitoring in place and now I want to run ad-hoc tests. How do we spoof fake alarms and see how the impact relationship within the service model change?

I figured the following two options:

  1. Used the stressors (Linux Stressor-ng https://www.mankier.com/1/stress-ng#Examples) on the servers to increase or decrease the resource consumption.
    1. This option is not so fast for customer demonstrations when it comes to break and fix things quickly.

  2. Ingest fake alarms using HEC towards the itsi_tracked_alerts index.
    1. I managed to get this working using curl, however, the event doesn’t seem to associate with the metrics.

Do you have any tips for achieving this use case? Are there any other options available?

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Fueling your curiosity with new Splunk ILT and eLearning courses

At Splunk Education, we’re driven by curiosity—both ours and yours! That’s why we’re committed to delivering ...

Splunk AI Assistant for SPL 1.1.0 | Now Personalized to Your Environment for Greater ...

Splunk AI Assistant for SPL has transformed how users interact with Splunk, making it easier than ever to ...

Unleash Unified Security and Observability with Splunk Cloud Platform

     Now Available on Microsoft AzureOn Demand Now Step boldly into the AI revolution with enhanced security ...