topic Re: How to search for a field name that a value exists for? in Splunk Search

How to search for a field name that a value exists for?

ltrand — Wed, 09 Dec 2015 15:07:31 GMT

I've been noodling on a problem that I can't seem to easily solve. We are bringing in JSON documents that describe files such as documents, executables, and container files. Because of the nature of container files specifically, (that the nesting is not predictable) we know that there is going to be a field that will contain an MD5 for all of the objects in the container, but we don't know what they will be named (the field names are directly related to the container file structure, so will constantly change). The field is generally in the following format:

root.sub1.sub2.objectname.MD5=value

With that said, what is a good way to search for an MD5 string against the whole dataset & then return the field name? If I can figure that out, then I can probably string together the rest to grab other objects from the event.

Thanks!

Re: How to search for a field name that a value exists for?

hortonew — Wed, 09 Dec 2015 17:28:26 GMT

If you don't know what field contains an md5 hash, something like the following will loop over all fields in an event, and create a new field for md5 that you can parse later on

index=_internal | eval field1="098f6bcd4621d373cade4e832627b4f6" | head 1 | foreach * [rex field=<<FIELD>> "^(?<md5>[a-f0-9]{32}$)"] | table _time, md5

Re: How to search for a field name that a value exists for?

ltrand — Wed, 09 Dec 2015 21:34:36 GMT

Would there be a way to eval the field name to discover all that end in .MD5?

Re: How to search for a field name that a value exists for?

hortonew — Wed, 09 Dec 2015 21:41:49 GMT

You'd have to build a delimited list of fields, which is doable. You want to output a list of fields that contain a value that ends in the literal string ".MD5" ?

Re: How to search for a field name that a value exists for?

woodcock — Thu, 10 Dec 2015 00:20:20 GMT

Like this:

index=_internal | eval TestField="This has matching MD5=098f6bcd4621d373cade4e832627b4f6 string" | head 1 | foreach * [eval firstFieldWithString=if((isnull(firstFieldWithString) AND like(<<FIELD>>, "%098f6bcd4621d373cade4e832627b4f6%")), "<<FIELD>>", firstFieldWithString) ] | table _time, firstFieldWithString

Re: How to search for a field name that a value exists for?

MuS — Thu, 10 Dec 2015 00:35:10 GMT

Hi ltrand,

this basic run everywhere search will match all fields that end with .MD5 and get the value into a new field called md5

| gentimes start=-1 
| eval myRaw="root.sub1.sub2.objectname.MD5=foo root.sub2.objectname.MD5=boo root.objectname.MD5=baz" 
| rex max_match=0 field=myRaw "\.MD5=(?<md5>[^\s]+)" 
| table md5

The first and second line is only needed to create some dummy events, so you won't need it. Tweak it as needed to match your events.

Hope this helps ...

cheers, MuS