Splunk Search

Can you help me use multiple regex for a single field?

MattibergB
Path Finder

Hi all,

We are trying to do the following:

At index time we want to use 4 regex TRANSFORMS to store values in two fields.

The issue I am running into is that I only want to keep the results from the regex that was not empty and not write the matches from the regex that matched before.

Here is our current set-up:

props.conf

TRANSFORMS-test= test1,test2,test3,test4

outputs.conf

[test1]
    REGEX = \S{3}.\d{1,2}.\d{1,2}\:\d{1,2}\:\d{1,2}.(([a-zA-Z0-9]{1,5}-){1,5}\S{1,3}).([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::empty message::$2
    [test2]
    REGEX = \S{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\s(([a-zA-Z0-9]{1,5}-){1,5}\S{1,3})\s([\S]+):\s([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::$1 message::$2
    [test3]
    REGEX = \S{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\.\d{1,3}\s([\S]+):\s([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::$1 message::$2
    [test4]
    REGEX = (%|-)([\S]+):\s([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::$1 message::$2

Is there a way to combine the regex that runs so that only the last one with actual values is written?

I tried using these in a an SPL, and as long as I did not use the same field names, I got the results. I was not able to combine it, the reason for index time is being able to use tstats.

Any help is appreciated.

Tags (2)
0 Karma
1 Solution

lakshman239
SplunkTrust
SplunkTrust

You can use coalesce https://www.splunk.com/blog/2014/03/21/search-command-coalesce.html

One way would be extract them into 4 different values and use coalesc to get the first non-null values to message_type and message

in your props.conf

EVAL-message_type = coalesce(message_type1,message_type2,message_type3,message_type4)

View solution in original post

0 Karma

arkadyz1
Builder

When more than one regex matches, values from which are you getting? The last one (alphabetically)? Or multivalue?

If it's just a single value, I'd suggest naming and ordering the stanzas in such a way that the preferred one goes last. And listing them in the same (reverse in terms of priority) order in props.conf. If it becomes a multivalue, try adding MV_ADD=false (though I doubt it will help).

0 Karma

MattibergB
Path Finder

I am getting multivalue's back, i did try MV_ADD=false but it is still multivalue.
Might be time to start go back to the drawing board and figure out if there is a better way to do this.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

You can use coalesce https://www.splunk.com/blog/2014/03/21/search-command-coalesce.html

One way would be extract them into 4 different values and use coalesc to get the first non-null values to message_type and message

in your props.conf

EVAL-message_type = coalesce(message_type1,message_type2,message_type3,message_type4)
0 Karma

MattibergB
Path Finder

Thank you for the suggestion, were are going to split up the regex and then use coalesce to get the right values.

0 Karma