Splunk Search

Mapping fields and values using regex and transforms.conf

hlarimer
Communicator

I have a very ugly log file that I need to run a regex against and have it match as many times as possible to map the field name and the value of the field. I have a working regex that I can test through search and its working as expected but when I try to move that regex to a transforms file using REPORT, I'm not seeing any results in my searches. Here are the props.conf and transforms.conf that is running on the SH:

props.conf
[jamfChangeManagement]
REPORT-jamfcm=jamfcm

transforms.conf
[jamfcm]
REGEX=(<_KEY_1>.​*?)\W{5,30}(<_VAL_1>\w.*​)

Any ideas why its not matching the regex in search?

0 Karma
1 Solution

acharlieh
Influencer

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

View solution in original post

acharlieh
Influencer

Playing with some data you shared in slack and putting it into regex101.com I came up with this for a transforms.conf expression:

[jamfcm]
REGEX = (?m)^\s++(?<_KEY_1>(?:[\w-\/]| (?![ .]))+) [. ]+ (?<_VAL_1>[^\s].*?|)$

Some other questions that you confirmed, the sourcetype of your data is exactly jamfChangeManagement and you restarted after saving changes. You weren't sure if you were running in Fast Mode or not (which obviously would not auto extract fields unless needed).

I then went to try your REGEX from here, and as I pasted into my editor I noticed that copying from here it looks like you have two ZERO WIDTH SPACE (U+200B) characters in your regex which are probably not intended and likely causing havoc as well. (One is before the first asterisk and one after the second)

hlarimer
Communicator

I confirmed that I was running in Verbose mode and still didn't get the extractions. I also changed the regex to match what you have above and still don't see any results. I think I'm going to try a very simple regex to confirm that the REPORT function is working correctly.

0 Karma

hlarimer
Communicator

This regex is working, thanks @acharlieh!

0 Karma

acharlieh
Influencer

Other thoughts, could it be a permissions or configuration overlaying issue?

0 Karma

hlarimer
Communicator

Shouldn't be, I'm running as admin.

0 Karma

acharlieh
Influencer

But even as admin, app context can matter.

0 Karma

hlarimer
Communicator

I gotcha, I could set global permissions in local.meta just to be sure.

0 Karma

Yasaswy
Contributor

Hi... just curious, does it work with / did you try using "EXTRACT" option?

0 Karma

hlarimer
Communicator

I have not tried the Extract option, I haven't ever used that command but it may be an option for testing.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...