topic Re: How to use regex to reformat a field? in Splunk Search

How to use regex to reformat a field?

ronbuzon — Tue, 14 Aug 2018 15:25:06 GMT

Need assistance regex to reformat the field

the field is Message. And the output is

"Reason: Details: Attributes: folderPathname folder ManagerDisplayName david foster OwnerEmail user@useremail"

when developing the regex to select anything after "Attributes:" i was able to create this rex

(?i)Attributes: (?.+)

It works in regex101.com and displays this field

the SPLUNK query that I wrote is:

(base search)||rex field=Message "Attributes: (?.+)

But the message field still shows the entire message value.

Any assistance will help

Re: How to use regex to reformat a field?

493669 — Tue, 14 Aug 2018 15:32:35 GMT

are you trying this :

   ... |rex field=Message "Attributes: (?<Message>.+)"

Re: How to use regex to reformat a field?

ronbuzon — Tue, 14 Aug 2018 15:35:17 GMT

Got this error-
Error in 'rex' command: Encountered the following error while compiling the regex 'Attributes: (?.+)': Regex: unrecognized character after (? or (?-

Re: How to use regex to reformat a field?

493669 — Tue, 14 Aug 2018 15:37:22 GMT

updated the query ...special characters was missed

Re: How to use regex to reformat a field?

gjanders — Tue, 14 Aug 2018 23:07:54 GMT

regex101 reports a pattern error on:

(?i)Attributes: (?.+)

If you wanted to capture the part after attributes then it would be:

(?i)Attributes: (?P<fieldname>.+)

If you wanted to regex match then it would be | rexgex :

(?i)Attributes: (.+)

Re: How to use regex to reformat a field?

ronbuzon — Wed, 15 Aug 2018 18:36:00 GMT

Gjanders,

Thank you for the recommendation and the feedback.

I used the regex command you have provided. however, the field from the search results still provide the entire value of the field. It seems like the rex command does not work.

(base search)||rex field=Message " (?i)Attributes: (?P.+)"

Re: How to use regex to reformat a field?

sudosplunk — Wed, 15 Aug 2018 19:36:09 GMT

Hi there, are you trying to trim values of Message field after indexing and create a new field new_field with these trimmed values? Or are you trying to trim the values of Message field before indexing?

Re: How to use regex to reformat a field?

gjanders — Wed, 15 Aug 2018 21:59:12 GMT

If your goal was to create a new field then:

 (base search)|rex field=Message " (?i)Attributes: (?P<new_field>.+)"

You could potentially override the Message field at search time, where I'm assuming message is a valid field name:

 (base search)|rex field=Message " (?i)Attributes: (?P<Message>.+)"

If you wanted to do this at index time it's completely different, or if your trying to extract the field with the name message from the event itself:

 (base search)|rex " (?i)Attributes: (?P<Message>.+)"

Re: How to use regex to reformat a field?

horsefez — Tue, 21 Aug 2018 14:33:05 GMT

Hi @ronbuzon,

if you are trying to extract the following data out of the given string, then try something like this.

Your sample data:
Reason: Details: Attributes: folderPathname folder ManagerDisplayName david foster OwnerEmail user@useremail

What you want to extract:
folderPathname folder ManagerDisplayName david foster OwnerEmail user@useremail

How you can do that:
| rex field=Message "Attributes:\s*(?.+)"

Please give me feedback, if that solves your problem.

Re: How to use regex to reformat a field?

ronbuzon — Tue, 21 Aug 2018 14:46:48 GMT

Hi @pyro_wood
thanks for responding and giving your input.

i tried your solution and got this error:

Error in 'rex' command: Encountered the following error while compiling the regex 'Attributes:\s*(?.+)': Regex: unrecognized character after (? or (?-

Re: How to use regex to reformat a field?

niketn — Tue, 21 Aug 2018 15:25:26 GMT

@ronbuzon , I think @493669 has already given you updated query... you need to provide a name for the capturing group, which is your case is Message. You should try the following:

<yourExistingSearch>
| rex field=Message "Attributes: (?<Message>.+)"

You can test the same on regex101 as well https://regex101.com/r/SZzS59/1

Re: How to use regex to reformat a field?

horsefez — Thu, 23 Aug 2018 19:09:02 GMT

Hi @ronbuzon,

sorry for the late reply.

Let me fix that solution:

| rex field=Message "Attributes:\s*(?<Attributes>.+)"