Splunk Search

Control Alerting Based on Host Groups

rlaan
Path Finder

I want to be able to create searches that will only look at hosts from different levels of our SDLC environment so for example a DEVELOPMENT group, TESTING and PRODUCTION. Also including a group to managed decommissioned servers until they return to use.

DEV = "host d, d1, d2, d3... dn"
TEST = "host t, t1, t2, t3... tn"
PRD = "host p, p1, p2, p3... pn"
NO_ALERT = "host d1, t1, p1"

I would like to be able to create searches and reports using "search DEV NOT NO_ALERT | foo "

Is this possible to avoid having to manually update each report whenever the scope of an environment changes?

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

You could tag your hosts and search for tag=DEV NOT tag=NO_ALERT.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

You could tag your hosts and search for tag=DEV NOT tag=NO_ALERT.

View solution in original post

rlaan
Path Finder

How do you allow all users to see tags that area created?

May they be quickly created using the deployment server for easier management? For a large amount of hosts the gui seems bulky

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

As with most objects in Splunk, you can control a tag's sharing between private (default), app (requires write permissions for the app), and global (requires admin) as well as per-role read and write permissions - just go to Settings -> Tags -> All unique tags -> Permissions for the tag you want to share.

For adding a large list I'd drop down to tags.conf and insert them there in bulk - whether you do that on your search head, or through a deployment server managed app doesn't really matter. Just do it like you manage all your Splunk configuration.
Alternatively, you could maintain a lookup file that resolves hosts to a stage field, and have users search using that field. Should be easier to maintain for large lists, but can be slower to search for.

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.