topic Re: Create one field from many, even when some fields don't exist in Splunk Search

Create one field from many, even when some fields don't exist

thisissplunk — Fri, 23 May 2014 14:45:59 GMT

Ok, so I understand how to eval something like this:

index=mystuff | eval
NewBigField=firstname + "-" + lastname
+ "-" + phonenumber

But there is a problem. Not all events will have "phonenumber" in them. This breaks the eval, so NewBigField gets populated with nothing instead of just firstname and lastname.

Going one step further, there are around 50 fields like phone number that may or may not exist in an event which I need to put into one big field with an eval.

What is the easiest way to create this new big field in a working fashion with dynamic fields? The only current option I see right now is 50 coalesce statements which is unacceptable.

Re: Create one field from many, even when some fields don't exist

MuS — Fri, 23 May 2014 14:58:24 GMT

Hi thisissplunk,

use fillnull like this

 index=mystuff | fillnull value="no number" phonenumber | eval NewBigField=firstname + "-" + lastname + "-" + phonenumber

hope this helps ...

cheers, MuS

Re: Create one field from many, even when some fields don't exist

thisissplunk — Fri, 23 May 2014 15:05:29 GMT

Seems like this will work, but is there any way to accomplish my goal without adding 50 extra statements (one for every possible field)? Maybe something like

fillnull value="no number" *

Re: Create one field from many, even when some fields don't exist

MuS — Fri, 23 May 2014 15:28:25 GMT

Yes just use it likt this

fillnull value="whateveryouneed"

If you don't provide any field as fillnull option, fillnull will use all available fileds

Re: Create one field from many, even when some fields don't exist

thisissplunk — Fri, 23 May 2014 15:33:23 GMT

This is great and I'll be testing soon. Thanks!

Re: Create one field from many, even when some fields don't exist

thisissplunk — Fri, 23 May 2014 19:40:31 GMT

So to complicate this further, these fields are not actually fields. They are being extracted by Splunk and displayed as fields on the left hand column. In reality they are bits and peices taken out of a huge field in the event.

Fillnull can't work because when a field like phonenumber is empty at the source, it is never brought into Splunk in the first place. Arg. Sounds like I'll need to get with our Splunk guys.

Re: Create one field from many, even when some fields don't exist

MuS — Fri, 23 May 2014 20:14:06 GMT

I have to post this as new answer, since it is too big for a comment 🙂

Take this run everywhere example based on two field extractions using regex and it works perfectly:

index=_internal | head 1 | eval myFoo="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas lorem eros, vehicula sed justo at, feugiat imperdiet arcu. Phasellus auctor mauris sapien, eget eleifend ante mollis eget. Aliquam id leo aliquam, auctor quam et, sollicitudin mauris. Pellentesque sagittis molestie quam. Aenean eu aliquet nisl. Donec ornare nisl ipsum, sed ultricies sapien varius nec. Mauris arcu quam, interdum in tincidunt et, ultrices in nisl. Sed et quam justo. Vestibulum imperdiet, neque ut consequat sodales, ipsum neque euismod massa, vel dignissim quam felis sed massa. Sed lacinia velit et lorem interdum venenatis. Pellentesque a blandit ipsum, in interdum lorem. Etiam risus lorem, facilisis eu mauris eget, porta viverra nulla." | rex field=myFoo "\w+\s(?<myRex>\w+)\s" | rex field=myFoo "\d+\s(?<myNotRex>\w+)\s" | fillnull value="no hit" | table myRex myNotRex

this will return a table like this image below, where the field myNotRex with the not matching regex will be filled by fillnull using the value "no hit"

cheers, MuS