topic Re: I failed to get parameter value from Splunk regex query result in Splunk Search

I failed to get parameter value from Splunk regex query result

splunkusera — Fri, 27 Apr 2012 05:41:35 GMT

Per this document in splunk (http://docs.splunk.com/Documentation/Splunk/latest/Data/overridedefaulthostassignments), i'd like to change value of a default field, "host" during index time.

There are two fields, fielda="vala" and fieldb="valb" from log event data I need to refer to.
And format of new host value is FORMAT = "host::$1.$2".

I worte 3 regex:

REGEX = fielda="(?P<AAA>[^"]*)"|fieldb="(?P<BBB>[^"]*)"

$2 is always "" and host is set to 'vala.'.

REGEX = fielda="(?P<AAA>[^"]*)".*(fieldb="(?P<BBB>[^"]*)")?

$2 is always "" and host is set to 'vala.'.

REGEX = fielda="(?P<AAA>[^"]*)".*fieldb="(?P<BBB>[^"]*)"

I got expected value, "vala.valb".

In summary, splunk regex performs non-greedy matching. How can I match all paths and get $2 filled?

Thanks,
Kevin

Re: I failed to get parameter value from Splunk regex query result

Ayn — Fri, 27 Apr 2012 06:01:33 GMT

It's not performing non-greedy matching, but it's true that once a complete match is identified it stops. So if you have an OR type regex with (match1|match2) it will indeed not match match2 if it's already found match1. This is not something Splunk specific really.

What's the problem with your third regex that makes you unable to use that? It looks pretty much like what I had suggested if you hadn't already created the correct regex yourself. 🙂
If the issue is that fieldb can occur before fielda, just write a regex to cover that case as well.

Re: I failed to get parameter value from Splunk regex query result

splunkusera — Sat, 28 Apr 2012 02:45:53 GMT

Thank you for your kindly help first. The problem for me is there might be only a fielda in log event without fieldb. So, I can not use regex 3 and we have order assumption for regex 2. The best way for me to use is regex 1.

Can we specific some option to perform a full match?I use RegexBuddy to test regex1 and it can highlight all result.

Re: I failed to get parameter value from Splunk regex query result

sowings — Mon, 30 Apr 2012 18:45:16 GMT

Since you're building up a host:: tag based upon the matches from the REGEX, I nominally thought of that as "fielda=host" and "fieldb=domain". Based upon that, I wrote up this tiny sample:

Mar 20 22:03:49.335 [RATE] host="bogfly"
Mar 21 10:03:19.335 [RATE] host="wibblenog" domain="bar.com"

I then used this regex to capture appropriately:

host="(?<AAA>[^"]*)".*(domain="(?<BBB>[^"]*)")?"

The BBB value is now in $3. The whole match for fieldb (in this case, "domain") is now optional.

Re: I failed to get parameter value from Splunk regex query result

splunkusera — Mon, 28 Sep 2020 11:45:51 GMT

Still failed:-). Is there anything wrong with my configuration?

In props.conf I add following section.
[HostRewrite]
SHOULD_LINEMERGE = false
TRANSFORMS-ZYHostTrans = ZYHostTrans
NO_BINARY_CHECK = 1
pulldown_type = 1

In transforms.conf I add following section:
[ZYHostTrans]
REGEX = host="(?[^"])".(domain="(?[^"]*)")?"
FORMAT = host::$1.$3
DEST_KEY = MetaData:Host

My log content is as following:
Mar 20 22:03:49.335 [RATE] host="bogfly"
Mar 21 10:03:19.335 [RATE] host="wibblenog" domain="bar.com"

Re: I failed to get parameter value from Splunk regex query result

splunkusera — Wed, 02 May 2012 03:24:12 GMT

I got two host values.
One is my pc hostname which is for the first event.
The other is "wibblenog." which is for the second one.
BTW, I test it in splunk 4.3.1 evaluation version in windows 7.

Re: I failed to get parameter value from Splunk regex query result

splunkusera — Mon, 28 Sep 2020 11:48:34 GMT

We finally resolve the issue with another regular expression. The expression is
host="(?[^"])".*domain="(?[^"])"|host=(?[^"]*)"

The reason we can match is for NFA engine Splunk uses, we will match alternatives one by one if there are multiple alternatives. That is it tries to match first alternative first and then second with order.

Thanks a lot for all your help, Splunk guys.