Splunk Search
Highlighted

Rewriting field values in search time

Path Finder

Hello,

I've got some events like this extracting fields using kv_mode=auto:

key1="value1", key2="value2", null1="NULL", key3="value3", null2="NULL", key4="value4"

This config generates these fields:

1. key1="value1
 2. key2="value2"
 3. null1="NULL"
 4. key3="value3"
 5. null2="value2"
 6. key4="value4"

At search time, and without modifying my SPL queries (so I guess it should be done with some config at props.conf and transforms.conf), I need to rewrite null fields (null1 and null2 in the example) to contain the value "" (empty string), null value (null(), which I think it's the same for Splunk) or even removing these kv-null-pairs. I have done this before, but at index-time, using a sedcmd, but it's not so clear for me to get this working at search time, which is what I need.

Ideally, fields should look like this:

 1. key1="value1
 2. key2="value2"
 3. null1=""
 4. key3="value3"
 5. null2=""
 6. key4="value4"

or

 1. key1="value1
 2. key2="value2"
 3. key3="value3"
 4. key4="value4"

Thanks for your help!

0 Karma
Highlighted

Re: Rewriting field values in search time

Esteemed Legend

Like this:

foreach * [eval <<FIELD>>=if(like($<<FIELD>>$, "NULL"), null(), $<<FIELD>>$)]
0 Karma
Highlighted

Re: Rewriting field values in search time

Path Finder

Thanks woodcook!! But, is this SPL? I cannot modify my queries. I need to do it with any config if it even exists

0 Karma
Highlighted

Re: Rewriting field values in search time

Esteemed Legend

You need SEDCMD:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anonymizedata

Like this:

 SEDCMD-NULL2null = 's/="NULL"/=""/'
0 Karma
Highlighted

Re: Rewriting field values in search time

Path Finder

But this is at index time... I need at search time and not in the query.

0 Karma
Highlighted

Re: Rewriting field values in search time

Contributor

You could use separate eval statement in props.conf. Similar to woodcocks foreach example - although you would have to do it separately for each field like this ...

[your_sourcetype]
EVAL-null1 = if(null1="NULL","",null1)
EVAL-null2 = if(null2="NULL","",null2)

This would be applied at search time unlike the SED command.

0 Karma
Highlighted

Re: Rewriting field values in search time

Path Finder

Yes, I've tried that, but it generates a performance problem. Each single event have up to 50 fields and any off them can (or cannot) be NULL, and this index has 50.000.000 events, so it's a problem evaluating each field of each event when making a search.

I think I could rephrase the question like this:
Is it possible to modify _raw before extracting fields at search time??

0 Karma