Splunk Search

how to use a lookup table to identify missing tags from a search?

jhaggard_splunk
Splunk Employee
Splunk Employee

Heres the ask...

I want to run a spl to see what tags are MISSING from a potential host by looking at a lookup file that contains all of the required tags. So for example if everything is tagged right it should look like:

ID ----- TAG 1 (env) ----- TAG 2 (location) ------- TAG 3 (owner)
Alpha ---------- Dev ---------- OnPrem ---------- Bob
Bravo ---------- QA ---------- OnPrem ---------- Sally
Charlie ---------- Prod ---------- Cloud ---------- Fred

In the real world where tags are missing it looks like this:

ID ----- TAG 1 (env) ----- TAG 2 (location) ------- TAG 3 (owner)
Alpha ---------- Dev ---------- OnPrem ---------- NULL
Bravo ---------- NULL ---------- NULL ---------- Sally
Charlie ---------- NULL ---------- Cloud ---------- Fred

The lookup file I have is called tags.csv and the name of the column is "Name" which contains a list of tags that should be present on all servers:
tag.env
tag.location
tag.owner

I want to be able to run an SPL that shows me the SERVER NAME (field is called "id") and a list of the missing tags based on the lookup file (tags.csv) so it should show something like:

Server Name / Missing tags:
Alpha / owner
Bravo / env, location
Charlie / env

I have been struggling with this and can get it to show me which tags match, but I can't get it to do the reverse and so far I've spent several hours looking at splunkanswers with no success. I would love some input if anyone has an idea of how to accomplish this.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Hi jhaggard,

you can use something like this:

your search... 
| table id tag.env tag.location tag.owner

| eval missing_fields = ""
| foreach tag* [ eval missing_<<FIELD>> = if(isnull(<<FIELD>>),1,0)]
| foreach missing_* [ eval missing_fields = if(<<FIELD>>=1,missing_fields+"<<FIELD>>"+",",missing_fields) ]

| stats values(missing_fields) by id

This will just check if the your fields (in this example my case tag*) is null or not without the need to use a lookup. I would just be wary of using . in field names. try to replace it with _

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

jhaggard_splunk
Splunk Employee
Splunk Employee

Thanks @diogofgm , thats an interesting approach , however the lookup file is required. One reason is some tags are for a dev env, some are for prod and they differ, the lookup file allows us to indicate which tags we care about.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Ok, that raises a few more questions.
How do you distinguish dev from prod tags?
Do you have other fields in the lookup (other than "Name")?
Also, how does your lookup related to your raw data? What are you using as an input field?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

somesoni2
Revered Legend

How does your data looks like? What all fields are present apart from field id (server name) in your raw data?

0 Karma

jhaggard_splunk
Splunk Employee
Splunk Employee

Lots of data, from ip to hostname to asset tag, serial number, vendor, location, etc ..

I'm not at the office right now so I can't provide any of it at this time.

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...