Reporting

Name of an Alert (Saved Search) available in a search query?

da029jo
Explorer

Is it possisble to reference the Name of a Report/Alert (Saved Search) in a search query?

I have a search query that i have to clone and use lots of times as backend server farm scales. The search text for the Alert is identical each time, with the exception of the filter value for the host. This needs to be changed each time. I have included this in the title of the Alert so if i could reference the name of the alert as a string value i could use the host value contained in the name to make the search different without having to edit it every time.

0 Karma
1 Solution

sloshburch
Splunk Employee
Splunk Employee

I was thinking a macro as well. So you'd have something like:

index=data log_level=ERROR `ignore_maintenance_hosts`

Then other users can have access to edit and add to the macro in the ticks without editing the main search.

View solution in original post

0 Karma

sloshburch
Splunk Employee
Splunk Employee

I was thinking a macro as well. So you'd have something like:

index=data log_level=ERROR `ignore_maintenance_hosts`

Then other users can have access to edit and add to the macro in the ticks without editing the main search.

0 Karma

da029jo
Explorer

Thank you to both renjith.nair and SloshBurch.

Macro works great combined with REST api to modify it as/when the server is in placed into Maintenance mode.

0 Karma

da029jo
Explorer

Is there a REST api endpoint that would allow me to alter the stored text in the Macro via a REST call when i place servers into and out of maintenance mode?

0 Karma

sloshburch
Splunk Employee
Splunk Employee

yea, probably. I'm not sure which endpoint but it's likely to be in the Knowledge or Configuration collection @ http://docs.splunk.com/Documentation/Splunk/latest/RESTREF

If you don't see it there, maybe spin this endpoint question into a new answers post to get more visibility. Since everything in Splunk UI uses REST endpoints, it's just a matter of finding the right one and playing with it to get it right.

0 Karma

da029jo
Explorer

Got it, for reference for anyone readin this later the REST api to use is the Configuration Endpoint witht properties usage:
http://docs.splunk.com/Documentation/Splunk/6.4.2/RESTREF/RESTconf#properties.2F.7Bfile.7D.2F.7Bstan...

Where the {file} is macros and {stanza} is the name of the macro.

Thank you

0 Karma

somesoni2
Revered Legend

In that case I would go by creating a lookup table with list of hosts under maintenance currently. You can then add a subsearch to your base search to remove the hosts under maintenance (from lookup table). So that the alert will work for other hosts.

Something on this line

your base search NOT [| inputlookup host_under_maintenance.csv | table host ] | rest of the search
0 Karma

jkat54
SplunkTrust
SplunkTrust

Have you seen this savedsearch command?

http://docs.splunk.com/Documentation/Splunk/6.2.5/SearchReference/Savedsearch

You can save a search like this:

index=index host=$hostname$

Save it as "testsearch"

Then you use |savedsearch testsearch host="someValue" which will make a search like this:

index=index host=someValue

So now you can do some fun stuff.

Also if you created a search macro like this:
(host=serverBeingUpgraded host=otherServerBeingUpgraded)

Name it "upgrading"

Then use the macro in your search
index=index NOT `upgrading` | ...

This way you're just maintaining the search macro.

0 Karma

somesoni2
Revered Legend

What type of alert is this (what is search query and alert condition)? If it's acceptable to have one single alert (email OR script) for all those hosts, they you could just create a lookup table with host name and then use map command to run your search/saved search with the hosts from lookup. This way it will be scalable (for new hosts, you just need to update the lookup).

0 Karma

da029jo
Explorer

Hi, thanks. Basically I want an alert on any 'ERROR' in s particular log file. However I want to ensure we do not get alerts during known server maintenance. So I imagined creating multiple alerts where the host is filtered so that I had different alerts that I could disable whilst that particular server was being 'maintained' but ensure alerts still fire for other servers.

Hope that all makes sense?

0 Karma

renjith_nair
Legend
0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...