Splunk Search

Can you help me use multiple regex for a single field?

MattibergB
Path Finder

Hi all,

We are trying to do the following:

At index time we want to use 4 regex TRANSFORMS to store values in two fields.

The issue I am running into is that I only want to keep the results from the regex that was not empty and not write the matches from the regex that matched before.

Here is our current set-up:

props.conf

TRANSFORMS-test= test1,test2,test3,test4

outputs.conf

[test1]
    REGEX = \S{3}.\d{1,2}.\d{1,2}\:\d{1,2}\:\d{1,2}.(([a-zA-Z0-9]{1,5}-){1,5}\S{1,3}).([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::empty message::$2
    [test2]
    REGEX = \S{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\s(([a-zA-Z0-9]{1,5}-){1,5}\S{1,3})\s([\S]+):\s([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::$1 message::$2
    [test3]
    REGEX = \S{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\.\d{1,3}\s([\S]+):\s([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::$1 message::$2
    [test4]
    REGEX = (%|-)([\S]+):\s([\S\s]+)
    WRITE_META = true
    FORMAT = message_type::$1 message::$2

Is there a way to combine the regex that runs so that only the last one with actual values is written?

I tried using these in a an SPL, and as long as I did not use the same field names, I got the results. I was not able to combine it, the reason for index time is being able to use tstats.

Any help is appreciated.

Tags (2)
0 Karma
1 Solution

lakshman239
SplunkTrust
SplunkTrust

You can use coalesce https://www.splunk.com/blog/2014/03/21/search-command-coalesce.html

One way would be extract them into 4 different values and use coalesc to get the first non-null values to message_type and message

in your props.conf

EVAL-message_type = coalesce(message_type1,message_type2,message_type3,message_type4)

View solution in original post

0 Karma

arkadyz1
Builder

When more than one regex matches, values from which are you getting? The last one (alphabetically)? Or multivalue?

If it's just a single value, I'd suggest naming and ordering the stanzas in such a way that the preferred one goes last. And listing them in the same (reverse in terms of priority) order in props.conf. If it becomes a multivalue, try adding MV_ADD=false (though I doubt it will help).

0 Karma

MattibergB
Path Finder

I am getting multivalue's back, i did try MV_ADD=false but it is still multivalue.
Might be time to start go back to the drawing board and figure out if there is a better way to do this.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

You can use coalesce https://www.splunk.com/blog/2014/03/21/search-command-coalesce.html

One way would be extract them into 4 different values and use coalesc to get the first non-null values to message_type and message

in your props.conf

EVAL-message_type = coalesce(message_type1,message_type2,message_type3,message_type4)
0 Karma

MattibergB
Path Finder

Thank you for the suggestion, were are going to split up the regex and then use coalesce to get the right values.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...