Hi all,
We are trying to do the following:
At index time we want to use 4 regex TRANSFORMS to store values in two fields.
The issue I am running into is that I only want to keep the results from the regex that was not empty and not write the matches from the regex that matched before.
Here is our current set-up:
props.conf
TRANSFORMS-test= test1,test2,test3,test4
outputs.conf
[test1]
REGEX = \S{3}.\d{1,2}.\d{1,2}\:\d{1,2}\:\d{1,2}.(([a-zA-Z0-9]{1,5}-){1,5}\S{1,3}).([\S\s]+)
WRITE_META = true
FORMAT = message_type::empty message::$2
[test2]
REGEX = \S{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\s(([a-zA-Z0-9]{1,5}-){1,5}\S{1,3})\s([\S]+):\s([\S\s]+)
WRITE_META = true
FORMAT = message_type::$1 message::$2
[test3]
REGEX = \S{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\.\d{1,3}\s([\S]+):\s([\S\s]+)
WRITE_META = true
FORMAT = message_type::$1 message::$2
[test4]
REGEX = (%|-)([\S]+):\s([\S\s]+)
WRITE_META = true
FORMAT = message_type::$1 message::$2
Is there a way to combine the regex that runs so that only the last one with actual values is written?
I tried using these in a an SPL, and as long as I did not use the same field names, I got the results. I was not able to combine it, the reason for index time is being able to use tstats.
Any help is appreciated.
You can use coalesce https://www.splunk.com/blog/2014/03/21/search-command-coalesce.html
One way would be extract them into 4 different values and use coalesc to get the first non-null values to message_type and message
in your props.conf
EVAL-message_type = coalesce(message_type1,message_type2,message_type3,message_type4)
When more than one regex matches, values from which are you getting? The last one (alphabetically)? Or multivalue?
If it's just a single value, I'd suggest naming and ordering the stanzas in such a way that the preferred one goes last. And listing them in the same (reverse in terms of priority) order in props.conf
. If it becomes a multivalue, try adding MV_ADD=false
(though I doubt it will help).
I am getting multivalue's back, i did try MV_ADD=false
but it is still multivalue.
Might be time to start go back to the drawing board and figure out if there is a better way to do this.
You can use coalesce https://www.splunk.com/blog/2014/03/21/search-command-coalesce.html
One way would be extract them into 4 different values and use coalesc to get the first non-null values to message_type and message
in your props.conf
EVAL-message_type = coalesce(message_type1,message_type2,message_type3,message_type4)
Thank you for the suggestion, were are going to split up the regex and then use coalesce to get the right values.