Splunk Search

Rewriting field values in search time

jdanij
Path Finder

Hello,

I've got some events like this extracting fields using kv_mode=auto:

key1="value1", key2="value2", null1="NULL", key3="value3", null2="NULL", key4="value4"

This config generates these fields:

1. key1="value1
 2. key2="value2"
 3. null1="NULL"
 4. key3="value3"
 5. null2="value2"
 6. key4="value4"

At search time, and without modifying my SPL queries (so I guess it should be done with some config at props.conf and transforms.conf), I need to rewrite null fields (null1 and null2 in the example) to contain the value "" (empty string), null value (null(), which I think it's the same for Splunk) or even removing these kv-null-pairs. I have done this before, but at index-time, using a sedcmd, but it's not so clear for me to get this working at search time, which is what I need.

Ideally, fields should look like this:

 1. key1="value1
 2. key2="value2"
 3. null1=""
 4. key3="value3"
 5. null2=""
 6. key4="value4"

or

 1. key1="value1
 2. key2="value2"
 3. key3="value3"
 4. key4="value4"

Thanks for your help!

0 Karma

lquinn
Contributor

You could use separate eval statement in props.conf. Similar to woodcocks foreach example - although you would have to do it separately for each field like this ...

[your_sourcetype]
EVAL-null1 = if(null1="NULL","",null1)
EVAL-null2 = if(null2="NULL","",null2)

This would be applied at search time unlike the SED command.

0 Karma

jdanij
Path Finder

Yes, I've tried that, but it generates a performance problem. Each single event have up to 50 fields and any off them can (or cannot) be NULL, and this index has 50.000.000 events, so it's a problem evaluating each field of each event when making a search.

I think I could rephrase the question like this:
Is it possible to modify _raw before extracting fields at search time??

0 Karma

woodcock
Esteemed Legend

You need SEDCMD:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Anonymizedata

Like this:

 SEDCMD-NULL2null = 's/="NULL"/=""/'
0 Karma

jdanij
Path Finder

But this is at index time... I need at search time and not in the query.

0 Karma

woodcock
Esteemed Legend

Like this:

foreach * [eval <<FIELD>>=if(like($<<FIELD>>$, "NULL"), null(), $<<FIELD>>$)]
0 Karma

jdanij
Path Finder

Thanks woodcook!! But, is this SPL? I cannot modify my queries. I need to do it with any config if it even exists

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...