Rewriting field values in search time

[jdanij]

Hello,

I've got some events like this extracting fields using kv_mode=auto:

key1="value1", key2="value2", null1="NULL", key3="value3", null2="NULL", key4="value4"

This config generates these fields:

1. key1="value1
 2. key2="value2"
 3. null1="NULL"
 4. key3="value3"
 5. null2="value2"
 6. key4="value4"

At search time, and without modifying my SPL queries (so I guess it should be done with some config at props.conf and transforms.conf), I need to rewrite null fields (null1 and null2 in the example) to contain the value "" (empty string), null value (null(), which I think it's the same for Splunk) or even removing these kv-null-pairs. I have done this before, but at index-time, using a sedcmd, but it's not so clear for me to get this working at search time, which is what I need.

Ideally, fields should look like this:

 1. key1="value1
 2. key2="value2"
 3. null1=""
 4. key3="value3"
 5. null2=""
 6. key4="value4"

or

 1. key1="value1
 2. key2="value2"
 3. key3="value3"
 4. key4="value4"

Thanks for your help!

[lquinn]

You could use separate eval statement in props.conf. Similar to woodcocks foreach example - although you would have to do it separately for each field like this ...

[your_sourcetype]
EVAL-null1 = if(null1="NULL","",null1)
EVAL-null2 = if(null2="NULL","",null2)

This would be applied at search time unlike the SED command.

[jdanij]

Yes, I've tried that, but it generates a performance problem. Each single event have up to 50 fields and any off them can (or cannot) be NULL, and this index has 50.000.000 events, so it's a problem evaluating each field of each event when making a search.

I think I could rephrase the question like this:
Is it possible to modify _raw before extracting fields at search time??

[woodcock]

You need SEDCMD:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anonymizedata

Like this:

 SEDCMD-NULL2null = 's/="NULL"/=""/'

[jdanij]

But this is at index time... I need at search time and not in the query.

[woodcock]

Like this:

foreach * [eval <<FIELD>>=if(like($<<FIELD>>$, "NULL"), null(), $<<FIELD>>$)]

[jdanij]

Thanks woodcook!! But, is this SPL? I cannot modify my queries. I need to do it with any config if it even exists