Splunk Search

how to create a script command in a ksh

riotto
Path Finder

Hello,

I have some alerts that send an email with the events to me if triggered. I need to create a custom script for the alerts that check if that splunk indexer is the indexer that should be is receiving events from our application logs. 

Our application can run in two cities but only one city is active. We actually failover if necessary to the other city, so the splunk indexer in that city  only  will receive events from the application logs.  I cannot keep alerts from both indexers active, so the standby city needs to have all its alerts disabled.  We failover manually which causes us to manually disable alerts on indexer in the city we are switching from and enable the alerts in the city we are switching to. I need to all the alerts enabled at both cities all the timeI need the alerts to do an nslookup of the IP that is active for the application and if that IP is on that indexer's side then the alert is valid and should email the events to me. If the IP is not on that indexer's side, the alert is a false positive and nothing needs to get emailed.  Any help is appreciated

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @riotto,

You can lookup ip address to filter alert result in your alert search like below;

**As an example activeidx.acme.com resolves 10.1.1.1 which is current indexer. You should put its own resolved ip address in each city indexer.

| your search for alert
append [| makeresults | eval idx_fqdn="activeidx.acme.com" | lookup dnslookup clienthost as idx_fqdn OUTPUT clientip | fields clientip ]
| eventstats latest(clientip) as my_ip
| where my_ip="10.1.1.1"

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...