topic Re: Can I REGEX a string and assign to field? in Splunk Search

Can I REGEX a string and assign to field?

pfabrizi — Tue, 29 Sep 2020 21:04:31 GMT

I have Graylog forwarding Windows events and I use this command in my props.conf to parser

FIELDALIAS-winlogbeat_as_action = winlogbeat_keywords as action

this sets action to the value of winlogbeat_keywords however it is [audit success] and I want to remove the '[]',

I know I can use an EXTRACT and a REGEX expression, but I am guessing that is against the entire messages string and I want to run it on winlogbeat_keywords and\or action

can I do this?

Thanks!

Re: Can I REGEX a string and assign to field?

harsmarvania57 — Tue, 28 Aug 2018 12:14:15 GMT

Hi @pfabrizi,

Yes you can apply regex to specific fields. I have created run anywhere search

| makeresults | eval winlogbeat_keywords="[abc xyz]"
| rex field=winlogbeat_keywords "\[(?<action>.*)\]"

I hope this helps.

Thanks,
Harshil

Re: Can I REGEX a string and assign to field?

pfabrizi — Tue, 28 Aug 2018 12:49:46 GMT

Will this work in a props.conf or transforms.conf?

Re: Can I REGEX a string and assign to field?

harsmarvania57 — Tue, 28 Aug 2018 12:57:22 GMT

Try this on Search Head

props.conf

[your_sourcetype]
REPORT-fields = actions_extraction

transforms.conf

[actions_extraction]
SOURCE_KEY = winlogbeat_keywords
REGEX = \[(?<action>.*)\]

Re: Can I REGEX a string and assign to field?

pfabrizi — Tue, 29 Sep 2020 21:02:52 GMT

I add thi line to my props.conf

[graylog:windows]
REPORT-fields = get-action

then in transforms.conf

[get-action]
SOURCEKEY_KEY = winlogbeat_keywords
REGEX = [(?.*)]

it doesn't seem to work.

Re: Can I REGEX a string and assign to field?

harsmarvania57 — Wed, 29 Aug 2018 09:28:34 GMT

While testing this configuration it looks like either you need to extract and index this data in another field at Index time or if you want to use search time extraction then you need to apply regex to _raw data.

Based on answers on https://answers.splunk.com/answers/559858/why-does-a-transforms-report-stanza-have-issues-wi.html REPORT has higher precedence then KV_MODE and it looks like in your case winlogbeat_keywords field extracted by KV_MODE=auto so when you define props and transforms for winlogbeat_keywords field it will not work because that field was not extracted due to higher precedence of REPORT than KV_MODE

I hope this clears your query.

Re: Can I REGEX a string and assign to field?

pfabrizi — Tue, 29 Sep 2020 21:03:09 GMT

So can I do this, which is actually what I want to do.
I need to try and match the auto lookups from the splunk_ta_windows into my custom app.
I know we should be using the splunk_ta_windows with the UF, but our operations folks drive what we use for collection,

Thanks!

I need to make [Audit Success] to success or failure...
I see in the SPLUNK_TA_WINDOWS they use a lookup.

LOOKUP-action_for_windows1_security = windows_action_lookup Type OUTPUTNEW action, action AS status
LOOKUP-action_for_windows2_security = windows_action_lookup Type as Keywords OUTPUTNEW action, action AS status

Should I be able to do this in my app? I am trying this:

Props.conf:
LOOKUP-action_for_windows1_security = windows_action_lookup Type OUTPUTNEW action, action AS status
LOOKUP-action_for_windows2_security = windows_action_lookup Type as Keywords OUTPUTNEW action, action AS status

Transforms.conf:
[windows_action_lookup]
filename = windows_actions_graylog.csv

windows_actions_graylog.csv:
Type,action
"audit failure",failure
"Audit Failure",failure
"AUDIT_FAILURE",failure
"failure audit",failure
"Failure Audit",failure
"FAILURE_AUDIT",failure
"audit success",success
"[Audit Success]",success
"AUDIT_SUCCESS",success
"success audit",success
"Success Audit",success
"SUCCESS_AUDIT",success
"success","success"
"failure","failure"

Re: Can I REGEX a string and assign to field?

horsefez — Wed, 29 Aug 2018 13:35:47 GMT

Also you need to put the regex into the REGEX= field like written above. I See from your config file export that you didn't do that correctly.

REGEX=\[(?<action>.*)\]

Re: Can I REGEX a string and assign to field?

pfabrizi — Tue, 29 Sep 2020 21:03:14 GMT

so this is not correct?

props.conf.

REPORT-field = get_action

Transform:
[get_action]
SOURCEKEY=winlogbeat_keywords
REGEX = [(?.*)] ------ should this be winlogbeat_keywords?

Thanks!

Re: Can I REGEX a string and assign to field?

horsefez — Wed, 29 Aug 2018 13:48:07 GMT

If you look what @harsmarvania57 and I wrote then you can see the difference.

Re: Can I REGEX a string and assign to field?

pfabrizi — Wed, 29 Aug 2018 14:42:32 GMT

sorry, it was a type.
I have it as:
REGEX = [(?.*)]