Splunk Search

how to use a lookup table to identify missing tags from a search?

jhaggard_splunk
Splunk Employee
Splunk Employee

Heres the ask...

I want to run a spl to see what tags are MISSING from a potential host by looking at a lookup file that contains all of the required tags. So for example if everything is tagged right it should look like:

ID ----- TAG 1 (env) ----- TAG 2 (location) ------- TAG 3 (owner)
Alpha ---------- Dev ---------- OnPrem ---------- Bob
Bravo ---------- QA ---------- OnPrem ---------- Sally
Charlie ---------- Prod ---------- Cloud ---------- Fred

In the real world where tags are missing it looks like this:

ID ----- TAG 1 (env) ----- TAG 2 (location) ------- TAG 3 (owner)
Alpha ---------- Dev ---------- OnPrem ---------- NULL
Bravo ---------- NULL ---------- NULL ---------- Sally
Charlie ---------- NULL ---------- Cloud ---------- Fred

The lookup file I have is called tags.csv and the name of the column is "Name" which contains a list of tags that should be present on all servers:
tag.env
tag.location
tag.owner

I want to be able to run an SPL that shows me the SERVER NAME (field is called "id") and a list of the missing tags based on the lookup file (tags.csv) so it should show something like:

Server Name / Missing tags:
Alpha / owner
Bravo / env, location
Charlie / env

I have been struggling with this and can get it to show me which tags match, but I can't get it to do the reverse and so far I've spent several hours looking at splunkanswers with no success. I would love some input if anyone has an idea of how to accomplish this.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Hi jhaggard,

you can use something like this:

your search... 
| table id tag.env tag.location tag.owner

| eval missing_fields = ""
| foreach tag* [ eval missing_<<FIELD>> = if(isnull(<<FIELD>>),1,0)]
| foreach missing_* [ eval missing_fields = if(<<FIELD>>=1,missing_fields+"<<FIELD>>"+",",missing_fields) ]

| stats values(missing_fields) by id

This will just check if the your fields (in this example my case tag*) is null or not without the need to use a lookup. I would just be wary of using . in field names. try to replace it with _

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

jhaggard_splunk
Splunk Employee
Splunk Employee

Thanks @diogofgm , thats an interesting approach , however the lookup file is required. One reason is some tags are for a dev env, some are for prod and they differ, the lookup file allows us to indicate which tags we care about.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Ok, that raises a few more questions.
How do you distinguish dev from prod tags?
Do you have other fields in the lookup (other than "Name")?
Also, how does your lookup related to your raw data? What are you using as an input field?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

somesoni2
Revered Legend

How does your data looks like? What all fields are present apart from field id (server name) in your raw data?

0 Karma

jhaggard_splunk
Splunk Employee
Splunk Employee

Lots of data, from ip to hostname to asset tag, serial number, vendor, location, etc ..

I'm not at the office right now so I can't provide any of it at this time.

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...