Splunk Search
Highlighted

Is it possible to use lookups to validate data coming in, using metadata?

Champion

HI,

Is it possible to create get entries in a serverclass (or a lookup), and then validate that data has been received from each host by comparing it againsts metadata?

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Legend

You could try something like this, for a list of host NOT in metadata

|inputlookup lookupfilewithhosts.csv | search NOT [| metadata type=hosts | fields host ]
0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Legend

I usually verify that all the hosts in my perimeter are sending data using a lookup and a search like this:

|inputlookup perimeter.csv | eval count=0 | append [ search _internal | stats count by host ] | stats sum(count) AS Total by host | where Total=0

in this way the hosts with Total!=0 are sending logs and hosts with Total=0 don't do it.
Bye.
Giuseppe

View solution in original post

Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Champion

Thanks. I try this, and get an error: [subsearch]: Could not find an index named "_blocksignature". My recollection is that index is no longer part of 6.4.

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Legend

where I used index=_internal, you have to use your index in which you want to verify the host list,
could you send the search to verify with the host list in perimeter?
Bye.
Giuseppe

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Champion

here's my search, which returns counts for servers that are collecting, but doesn't show for servers that aren't....

|inputlookup testagain.csv | eval count=0 | append [ search index=dh_clocksync | stats count by host ] | stats sum(count) AS Total by host | where Total=0

My lookup has 195 entries. When I remove the where column at the end, I get counts for 161 servers. I need to see which of those 195 aren't showing.

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Legend

what is the name of the lookup's field in which there are the host's names?
if it isn't "host" you have to modify the search in this way:
|inputlookup testagain.csv | rename lookupfield AS host | eval count=0 | append [ search index=dhclocksync | stats count by host ] | stats sum(count) AS Total by host | where Total=0

Bye.
Giuseppe

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Champion

weird. i updated my lookup, and the total count is greater than the number of entries in my lookup file.

|inputlookup clocksynclookup | eval count=0 | append [ search index=dhclocksync | stats count by host ] | stats sum(count) AS Total by host

IF I do |inputlookup clocksync_lookup, i get a total of 195 entries. When I use this complete search, I get 366 entries.

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Legend

this means that you have different host names between lookup and index.
Verify the case, or insert transformation in uppercase in both the searches like this

|inputlookup clocksynclookup | eval count=0 | eval host=upper(host) | append [ search index=dhclocksync | eval host=upper(host) | stats count by host ] | stats sum(count) AS Total by host

In this way you'll have 195 items.
Bye.
Giuseppe

0 Karma
Highlighted

Re: Is it possible to use lookups to validate data coming in, using metadata?

Champion

Fantastic. Didn't realize Splunk was case sensitive when comparing fields...

This works great and is a good prototype for future work. Thanks!

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.