topic Re: How to convert a regex to work in transforms.conf? in Splunk Search

How to convert a regex to work in transforms.conf?

hawkik1 — Fri, 14 Apr 2023 16:26:42 GMT

I am attempting (for the first tiume) to convert the following regex search to work in transforms.conf, but can't seem to get it to work. What am I missing?

My search which works:

index="fileshares" sourcetype="fileshares" source="/mnt/auditlog/*"
| rex "\"SubjectUserName\">(?<Username>[^\<]+)"

My attempt with transforms.conf:

[Username]
SOURCE_KEY = Username
REGEX = \"SubjectUserName\">(?<Username>[^\<]+)
MV_ADD = true

Props.conf:

[fileshares]
REPORT-fields = Username

Re: How to convert a regex to work in transforms.conf?

richgalloway — Fri, 14 Apr 2023 16:46:07 GMT

Regular expressions can't be evaluated without sample data.

Setting MV_ADD=true is necessary only when the rex command uses the max_match option with a value greater than zero.

Quotation marks do not need to be escaped in transforms.conf because the regex is not itself quoted.

That said, what are you trying to accomplish with transforms that rex cannot? If you just want to extract the Username field then use EXTRACT rather than REPORT in props and dispense with the transform.

EXTRACT-fields = "SubjectUserName">(?<Username>[^\<]+)

Keep in mind that REPORT transforms are processed at search time rather than index time.

Re: How to convert a regex to work in transforms.conf?

hawkik1 — Fri, 14 Apr 2023 17:44:40 GMT

I'm new at working with transforms.conf and props.conf. To better explain my issue. I've got an XML file that I'm trying to generate fields for.

I'm using this search portion which works:

index="fileshares" sourcetype="fileshares" source="/mnt/auditlog/*"
| rex "SystemTime=\"(?<SystemTime>[^\"]+)"

| rex "\"SubjectDomainName\">(?<Domain>[^\<]+)"
| rex "\"SubjectUserName\">(?<Username>[^\<]+)"
| rex "\"ObjectType\">(?<ObjectType>[^\<]+)"

I'd like to have these fields created so that the rex statements are not necessary. If using EXTRACT-fields is the best method for this, what should the configuration look like for these fields?

Re: How to convert a regex to work in transforms.conf?

hawkik1 — Fri, 14 Apr 2023 17:47:48 GMT

XML:

"" <Data Name="SubjectDomainName">US</Data><Data Name="SubjectUserName">fmtdc</Data><Data Name="ObjectServer">Security</Data><Data Name="ObjectType">File</Data> ""

Re: How to convert a regex to work in transforms.conf?

richgalloway — Fri, 14 Apr 2023 18:51:56 GMT

If you want to extract all of the XML fields then use KV_MODE = xml in props.conf. To extract selected fields then (IMO) EXTRACT is the way. Use your existing regular expressions, modified as I described in my previous answer.

EXTRACT-SystemTime = SystemTime="(?<SystemTime>[^"]+) EXTRACT-SubjectDomainName = SubjectDomainName">(?<Domain>[^\<]+) EXTRACT-SubjectUserName = SubjectUserName">(?<Username>[^\<]+) EXTRACT-ObjectType = ObjectType">(?<ObjectType>[^\<]+)

Re: How to convert a regex to work in transforms.conf?

woodcock — Fri, 14 Apr 2023 19:00:47 GMT

[Username]
REGEX = \"SubjectUserName\">(?<Username>[^\<]+)

Re: How to convert a regex to work in transforms.conf?

hawkik1 — Fri, 14 Apr 2023 21:09:05 GMT

I'm still doing something incorrect. I'm editing /opt/splunk/etc/apps/Splunk_TA_nix/local/props.conf on my standalone enterprise server.

I added this at the bottom of the file. Sourcetype of the search is fileshares:

[fileshares]
EXTRACT-SystemTime = SystemTime="(?<SystemTime>[^"]+)
EXTRACT-SubjectDomainName = SubjectDomainName">(?<Domain>[^\<]+)
EXTRACT-SubjectUserName = SubjectUserName">(?<Username>[^\<]+)
EXTRACT-ObjectType = ObjectType">(?<ObjectType>[^\<]+)

I also tried this:

[fileshares]
KV_MODE=xml

Restarted splunk and neither of the above worked. Using |xmlkv works fine btw, but grabs way too many fields, slowing the search. Thanks ahead of time from the newbe.