Mapping fields and values using regex and transforms.conf

[hlarimer]

I have a very ugly log file that I need to run a regex against and have it match as many times as possible to map the field name and the value of the field. I have a working regex that I can test through search and its working as expected but when I try to move that regex to a transforms file using REPORT, I'm not seeing any results in my searches. Here are the props.conf and transforms.conf that is running on the SH:

props.conf
[jamfChangeManagement]
REPORT-jamfcm=jamfcm

transforms.conf
[jamfcm]
REGEX=(<_KEY_1>.​*?)\W{5,30}(<_VAL_1>\w.*​)

Any ideas why its not matching the regex in search?

[acharlieh · Accepted solution]

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

[hlarimer]

I confirmed that I was running in Verbose mode and still didn't get the extractions. I also changed the regex to match what you have above and still don't see any results. I think I'm going to try a very simple regex to confirm that the REPORT function is working correctly.

[hlarimer]

This regex is working, thanks @acharlieh!

[acharlieh]

Other thoughts, could it be a permissions or configuration overlaying issue?

[hlarimer]

Shouldn't be, I'm running as admin.

[acharlieh]

But even as admin, app context can matter.

[hlarimer]

I gotcha, I could set global permissions in local.meta just to be sure.

[Yasaswy]

Hi... just curious, does it work with / did you try using "EXTRACT" option?

[hlarimer]

I have not tried the Extract option, I haven't ever used that command but it may be an option for testing.