topic Re: Regex to remove optional trailing text from field with transforms/props in Splunk Search

Regex to remove optional trailing text from field with transforms/props

bowesmana — Sat, 17 Jun 2017 07:44:53 GMT

I have a field called Title, where it may sometimes end with the text

 Ends 9 P.M.

or varying case related variants.

I can easily do this in my search

| rex mode=sed field=Title "s/(?i) Ends 9.?p.?m.?//"

which performs the job nicely, but I want to be able to do this as standard, so I tried setting up a transform and field extraction with the following regex

(.*)((?i) ends [0-9]*.?[ap].?m)?

but the optional ? at the end of the 'ends...' group means that the first (.*) will capture all text, including the 'ends...' section, so the result is no change.

If I get rid of the last ? then it works for fields that have the 'ends...' but not for those fields that don't so they lose their value.

Any help on the right regex or a way to setup a 'sed' style regex in conf?

Re: Regex to remove optional trailing text from field with transforms/props

horsefez — Sat, 17 Jun 2017 10:13:45 GMT

Hi bowesmana,

try out this regex and see if it will do the trick.

https://regex101.com/r/3MSGhl/2

(.+)(?:((?i)\sends\s[1]?[0-9]\s[ap]\.m\.))$

Re: Regex to remove optional trailing text from field with transforms/props

bowesmana — Sun, 18 Jun 2017 05:04:55 GMT

That has the same problem as my original, i.e. it does not capture anything in the capture group unless the text does have the trailing "ends..." phrase, e.g.

The phrase

Weekend Unreserved

gets an empty capture group 1 as the regex requires the "ends..." to be present to result in a match

Re: Regex to remove optional trailing text from field with transforms/props

bowesmana — Sun, 18 Jun 2017 05:12:10 GMT

I could make two alternatives within the regex, but then I am not sure how to assign the correct numbered capture group to the Title.

Re: Regex to remove optional trailing text from field with transforms/props

horsefez — Sun, 18 Jun 2017 09:01:52 GMT

How about this one?

https://regex101.com/r/3MSGhl/3

Re: Regex to remove optional trailing text from field with transforms/props

bowesmana — Sun, 18 Jun 2017 10:01:40 GMT

Still has the same issue, how does that work with transforms.conf, where the assignment is done with

Title::$X

where X is the capture group #. Unless it's always the same number how do you assign more than one capture group to the same field?

Re: Regex to remove optional trailing text from field with transforms/props

DalJeanis — Sun, 18 Jun 2017 22:54:16 GMT

Try this -

(?i)(^.*(?=\s*ends\s+\d+\s?[ap]\.?m\.?.*)|^.*)

This is a case-insensitive flag (?i) followed by a single capture group which has two options. The first option is anything, followed by a positive lookahead (?= for a value like " ends 9 pm". You'll notice I've allowed for 2-digit hours, etc. If that one fails, the second option takes everything. Both options require the match to start at the beginning of the string, with the first one ending at the start of the positive lookahead, and the second option taking the entire string.

Re: Regex to remove optional trailing text from field with transforms/props

bowesmana — Mon, 19 Jun 2017 10:22:33 GMT

Ah, that's the trick with the positive lookahead... That single capture group is the key, which means I can use Title::$1 in the transforms.conf and it works.

Out of interest, would lookaround work to remove prefixes to strings? I played around with a few attempts, but I don't see that it would.

Thanks

Re: Regex to remove optional trailing text from field with transforms/props

DalJeanis — Mon, 19 Jun 2017 11:52:22 GMT

Updated. Correct tool is definitely not lookaround.

Just need to take group 2 from this one:

^(drop this prefix )?(.*)