topic Re: Comparing two field values for != in Splunk Search

Comparing two field values for !=

himynamesdave — Thu, 06 Feb 2014 11:50:42 GMT

I'm trying to create a search comparing then validating two fields in Splunk... but struggling.

My first search uses a lookup to add a field called sha256 to an event (what I'm doing is adding a hash to an event):

source="syslog.log" | lookup sha256 raw as _raw | fields sha256

I also have a set of the same events indexed in Splunk with the hashes already writtern under a field called: event_hash.

source="syslog-prehash.log" | fields event_hash

Note: Although we have two sets of the same events (lookup hash [syslog.log] / pre-hashed [syslog-prehas.log]) the event timestamps are slightly different because of the ways the pre-hashed events are written and indexed.

Now I want to compare the field sha256 [syslog.log] and event_hash [syslog-prehash.log] to see if there are any hashes that do not match.

As the fields (hashes) are alphanumeric I am struggling to find a way to do this. I'm thinking I could join the two searches and pipe a where command looking for hashes that only appear once (to show hashes that have no matches).

Does anyone have any suggestions on a better way to write this search?

Thanks!

Re: Comparing two field values for !=

gfuente — Thu, 06 Feb 2014 12:02:56 GMT

Hello

As you said, it will work with a join. But I think it will perform better using something like:

source="syslog.log" OR source="syslog-prehash.log" | stats dc(source) as DCS by hash

Previously you need to create an Alias of the original fields to "hash", so in both sources you have the same field name, and therefore the stats count by that common field works.

Then you will get the results, if you get a "2" value, then the hash matches for that particular hash, if you a "1" value then you only have the hash from one of the sources, so no match.

After that you could filter the ones that doesn´t match, using: search DCS=1

Regards

Re: Comparing two field values for !=

MuS — Thu, 06 Feb 2014 14:46:42 GMT

may I ask why not use a simple

source="syslog.log" OR source="syslog-prehash.log" | where sha256!=event_hash

this should work as well