Splunk Search

Mapping fields and values using regex and transforms.conf

hlarimer
Communicator

I have a very ugly log file that I need to run a regex against and have it match as many times as possible to map the field name and the value of the field. I have a working regex that I can test through search and its working as expected but when I try to move that regex to a transforms file using REPORT, I'm not seeing any results in my searches. Here are the props.conf and transforms.conf that is running on the SH:

props.conf
[jamfChangeManagement]
REPORT-jamfcm=jamfcm

transforms.conf
[jamfcm]
REGEX=(<_KEY_1>.​*?)\W{5,30}(<_VAL_1>\w.*​)

Any ideas why its not matching the regex in search?

0 Karma
1 Solution

acharlieh
Influencer

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

View solution in original post

acharlieh
Influencer

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

hlarimer
Communicator

I confirmed that I was running in Verbose mode and still didn't get the extractions. I also changed the regex to match what you have above and still don't see any results. I think I'm going to try a very simple regex to confirm that the REPORT function is working correctly.

0 Karma

hlarimer
Communicator

This regex is working, thanks @acharlieh!

0 Karma

acharlieh
Influencer

Other thoughts, could it be a permissions or configuration overlaying issue?

0 Karma

hlarimer
Communicator

Shouldn't be, I'm running as admin.

0 Karma

acharlieh
Influencer

But even as admin, app context can matter.

0 Karma

hlarimer
Communicator

I gotcha, I could set global permissions in local.meta just to be sure.

0 Karma

Yasaswy
Contributor

Hi... just curious, does it work with / did you try using "EXTRACT" option?

0 Karma

hlarimer
Communicator

I have not tried the Extract option, I haven't ever used that command but it may be an option for testing.

0 Karma
Get Updates on the Splunk Community!

New Splunk Observability innovations: Deeper visibility and smarter alerting to ...

You asked, we delivered. Splunk Observability Cloud has several new innovations giving you deeper visibility ...

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...