Splunk Search

Rewriting field values in search time

jdanij
Path Finder

Hello,

I've got some events like this extracting fields using kv_mode=auto:

key1="value1", key2="value2", null1="NULL", key3="value3", null2="NULL", key4="value4"

This config generates these fields:

1. key1="value1
 2. key2="value2"
 3. null1="NULL"
 4. key3="value3"
 5. null2="value2"
 6. key4="value4"

At search time, and without modifying my SPL queries (so I guess it should be done with some config at props.conf and transforms.conf), I need to rewrite null fields (null1 and null2 in the example) to contain the value "" (empty string), null value (null(), which I think it's the same for Splunk) or even removing these kv-null-pairs. I have done this before, but at index-time, using a sedcmd, but it's not so clear for me to get this working at search time, which is what I need.

Ideally, fields should look like this:

 1. key1="value1
 2. key2="value2"
 3. null1=""
 4. key3="value3"
 5. null2=""
 6. key4="value4"

or

 1. key1="value1
 2. key2="value2"
 3. key3="value3"
 4. key4="value4"

Thanks for your help!

0 Karma

lquinn
Contributor

You could use separate eval statement in props.conf. Similar to woodcocks foreach example - although you would have to do it separately for each field like this ...

[your_sourcetype]
EVAL-null1 = if(null1="NULL","",null1)
EVAL-null2 = if(null2="NULL","",null2)

This would be applied at search time unlike the SED command.

0 Karma

jdanij
Path Finder

Yes, I've tried that, but it generates a performance problem. Each single event have up to 50 fields and any off them can (or cannot) be NULL, and this index has 50.000.000 events, so it's a problem evaluating each field of each event when making a search.

I think I could rephrase the question like this:
Is it possible to modify _raw before extracting fields at search time??

0 Karma

woodcock
Esteemed Legend

You need SEDCMD:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anonymizedata

Like this:

 SEDCMD-NULL2null = 's/="NULL"/=""/'
0 Karma

jdanij
Path Finder

But this is at index time... I need at search time and not in the query.

0 Karma

woodcock
Esteemed Legend

Like this:

foreach * [eval <<FIELD>>=if(like($<<FIELD>>$, "NULL"), null(), $<<FIELD>>$)]
0 Karma

jdanij
Path Finder

Thanks woodcook!! But, is this SPL? I cannot modify my queries. I need to do it with any config if it even exists

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...