Splunk Search
Highlighted

how to use a lookup table to identify missing tags from a search?

Splunk Employee
Splunk Employee

Heres the ask...

I want to run a spl to see what tags are MISSING from a potential host by looking at a lookup file that contains all of the required tags. So for example if everything is tagged right it should look like:

ID ----- TAG 1 (env) ----- TAG 2 (location) ------- TAG 3 (owner)
Alpha ---------- Dev ---------- OnPrem ---------- Bob
Bravo ---------- QA ---------- OnPrem ---------- Sally
Charlie ---------- Prod ---------- Cloud ---------- Fred

In the real world where tags are missing it looks like this:

ID ----- TAG 1 (env) ----- TAG 2 (location) ------- TAG 3 (owner)
Alpha ---------- Dev ---------- OnPrem ---------- NULL
Bravo ---------- NULL ---------- NULL ---------- Sally
Charlie ---------- NULL ---------- Cloud ---------- Fred

The lookup file I have is called tags.csv and the name of the column is "Name" which contains a list of tags that should be present on all servers:
tag.env
tag.location
tag.owner

I want to be able to run an SPL that shows me the SERVER NAME (field is called "id") and a list of the missing tags based on the lookup file (tags.csv) so it should show something like:

Server Name / Missing tags:
Alpha / owner
Bravo / env, location
Charlie / env

I have been struggling with this and can get it to show me which tags match, but I can't get it to do the reverse and so far I've spent several hours looking at splunkanswers with no success. I would love some input if anyone has an idea of how to accomplish this.

0 Karma
Highlighted

Re: how to use a lookup table to identify missing tags from a search?

SplunkTrust
SplunkTrust

How does your data looks like? What all fields are present apart from field id (server name) in your raw data?

0 Karma
Highlighted

Re: how to use a lookup table to identify missing tags from a search?

Splunk Employee
Splunk Employee

Lots of data, from ip to hostname to asset tag, serial number, vendor, location, etc ..

I'm not at the office right now so I can't provide any of it at this time.

0 Karma
Highlighted

Re: how to use a lookup table to identify missing tags from a search?

Motivator

Hi jhaggard,

you can use something like this:

your search... 
| table id tag.env tag.location tag.owner

| eval missing_fields = ""
| foreach tag* [ eval missing_<<FIELD>> = if(isnull(<<FIELD>>),1,0)]
| foreach missing_* [ eval missing_fields = if(<<FIELD>>=1,missing_fields+"<<FIELD>>"+",",missing_fields) ]

| stats values(missing_fields) by id

This will just check if the your fields (in this example my case tag*) is null or not without the need to use a lookup. I would just be wary of using . in field names. try to replace it with _

------------
Hope I was able to help you. If so, an upvote would be appreciated.
0 Karma
Highlighted

Re: how to use a lookup table to identify missing tags from a search?

Splunk Employee
Splunk Employee

Thanks @diogofgm , thats an interesting approach , however the lookup file is required. One reason is some tags are for a dev env, some are for prod and they differ, the lookup file allows us to indicate which tags we care about.

0 Karma
Highlighted

Re: how to use a lookup table to identify missing tags from a search?

Motivator

Ok, that raises a few more questions.
How do you distinguish dev from prod tags?
Do you have other fields in the lookup (other than "Name")?
Also, how does your lookup related to your raw data? What are you using as an input field?

------------
Hope I was able to help you. If so, an upvote would be appreciated.
0 Karma