Splunk ITSI

How to manipulate metric values so that Service Health Score can be changed on Ad-Hoc basis for demonstrations?

Chiranjit-Mitra
Loves-to-Learn

I've got the service monitoring in place and now I want to run ad-hoc tests. How do we spoof fake alarms and see how the impact relationship within the service model change?

I figured the following two options:

  1. Used the stressors (Linux Stressor-ng https://www.mankier.com/1/stress-ng#Examples) on the servers to increase or decrease the resource consumption.
    1. This option is not so fast for customer demonstrations when it comes to break and fix things quickly.

  2. Ingest fake alarms using HEC towards the itsi_tracked_alerts index.
    1. I managed to get this working using curl, however, the event doesn’t seem to associate with the metrics.

Do you have any tips for achieving this use case? Are there any other options available?

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...