topic Re: Rex not yielding result in Splunk Search

Rex not yielding result

harryhcg — Wed, 06 Sep 2023 05:05:59 GMT

Data: {"Field1":"xxx","message1":"{0}","message2":"xxx","message3":{"TEXT":"xxxx: xxx\r\n.xxxxx: {\"xxxxx\":{\"@CDI\":\"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 \",\"@RETURN\":\"xxxx-xxxxxxxxxx-xx-xxxxx\",\"@message4\":\"xxxxxx:xxx\",\"message5\":{\"message6............

Want to extract new field highlighted above but not getting any result.

This is what I tried:

| rex field=_raw "RETURN\\\"\:\\\"(?<Field2>[^\\]+)"

Re: Rex not yielding result

yuanliu — Wed, 06 Sep 2023 06:19:53 GMT

As I always tell people, do not treat structured data as plain text, and rex is not the right tool for JSON.

Looking at your illustration, I am convinced that your original data is fully compliant; the field message3.TEXT embeds an escaped, fully compliant JSON message with some leading text. Like thus

{"Field1":"xxx","message1":"{0}","message2":"xxx","message3":{"TEXT":"xxxx: xxx\r\n.xxxxx: {\"xxxxx\":{\"@CDI\":\"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 \",\"@RETURN\":\"xxxx-xxxxxxxxxx-xx-xxxxx\",\"@message4\":\"xxxxxx:xxx\",\"message5\":{\"message6\":null}}}"}}

As such, you can use this to directly access the field RETURN

| eval TEXT = replace('message3.TEXT', "^[^{]+", "") | spath input=TEXT path="xxxxx.@RETURN" output=Field2

The illustrated data will give something like

Field1	Field2	message1	message2	message3.TEXT
xxx	xxxx-xxxxxxxxxx-xx-xxxxx	{0}	xxx	xxxx: xxx .xxxxx: {"xxxxx":{"@CDI":"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 ","@RETURN":"xxxx-xxxxxxxxxx-xx-xxxxx","@message4":"xxxxxx:xxx","message5":{"message6":null}}}

Here is an emulation you can play with and compare with raw data

| makeresults | eval _raw = "{\"Field1\":\"xxx\",\"message1\":\"{0}\",\"message2\":\"xxx\",\"message3\":{\"TEXT\":\"xxxx: xxx\\r\\n.xxxxx: {\\\"xxxxx\\\":{\\\"@CDI\\\":\\\"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 \\\",\\\"@RETURN\\\":\\\"xxxx-xxxxxxxxxx-xx-xxxxx\\\",\\\"@message4\\\":\\\"xxxxxx:xxx\\\",\\\"message5\\\":{\\\"message6\\\":null}}}\"}}" | spath ``` data emulation above ```

Re: Rex not yielding result

gcusello — Wed, 06 Sep 2023 06:43:28 GMT

Hi @harryhcg,

this seems to be a json format, so, as @yuanliu hinted, try to use the "spath" command (https://community.splunk.com/t5/Splunk-Enterprise/spath-command/m-p/518343) .

About your regex, try to add another backslash to your regex:

| rex "RETURN\\\\"\:\\\\"(?<Field2>[^\\]+)"

Ciao.

Giuseppe

Re: Rex not yielding result

harryhcg — Wed, 06 Sep 2023 08:50:01 GMT

Regarding regex suggestion, still have issue.

Error - Regex: missing terminating ] for character class.

Analysing raw data to use spath. Thank you @gcusello @yuanliu

Re: Rex not yielding result

gcusello — Wed, 06 Sep 2023 08:59:18 GMT

Hi @harryhcg ,

let us know if we can help you more, or, please, accept one answer for the other people of Community.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

Re: Rex not yielding result

harryhcg — Wed, 06 Sep 2023 09:03:29 GMT

Single field extraction still wondering why it didn't work.

Re: Rex not yielding result

yuanliu — Thu, 07 Sep 2023 06:31:43 GMT

If you have to use regex, you will need more backslashes.

| rex "@RETURN\\\\\":\\\\\"(?<Field2>[^\\\]+)"

Re: Rex not yielding result

gcusello — Thu, 07 Sep 2023 06:34:29 GMT

Hi @harryhcg,

as also @yuanliu hinted, you have to add another backslash to the regex:

| rex "RETURN\\\\\"\:\\\\\"(?<Field2>[^\\]+)"

Ciao.

Giuseppe

Re: Rex not yielding result

PickleRick — Thu, 07 Sep 2023 09:15:17 GMT

While I wholeheartedly agree with the "don't use regex for structured data" it's worth noting that sometimes it's not easy to extract the structured part from the whole event.