Rex not yielding result

[harryhcg]

Data: {"Field1":"xxx","message1":"{0}","message2":"xxx","message3":{"TEXT":"xxxx: xxx\r\n.xxxxx: {\"xxxxx\":{\"@CDI\":\"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 \",\"@RETURN\":\"xxxx-xxxxxxxxxx-xx-xxxxx\",\"@message4\":\"xxxxxx:xxx\",\"message5\":{\"message6............

Want to extract new field highlighted above but not getting any result. 

This is what I tried:

| rex field=_raw "RETURN\\\"\:\\\"(?<Field2>[^\\]+)"

[gcusello]

Hi @harryhcg,

this seems to be a json format, so, as @yuanliu hinted, try to use the "spath" command (https://community.splunk.com/t5/Splunk-Enterprise/spath-command/m-p/518343) .

About your regex, try to add another backslash to your regex:

| rex "RETURN\\\\"\:\\\\"(?<Field2>[^\\]+)"

Ciao.

Giuseppe

[harryhcg]

Regarding regex suggestion, still have issue. 

Error - Regex: missing terminating ] for character class.

Analysing raw data to use spath. Thank you @gcusello @yuanliu 

[yuanliu]

If you have to use regex, you will need more backslashes.

| rex "@RETURN\\\\\":\\\\\"(?<Field2>[^\\\]+)"

[gcusello]

Hi @harryhcg ,

let us know if we can help you more, or, please, accept one answer for the other people of Community.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

[harryhcg]

Single field extraction still wondering why it didn't work. 

[gcusello]

Hi @harryhcg,

as also @yuanliu hinted, you have to add another backslash to the regex:

| rex "RETURN\\\\\"\:\\\\\"(?<Field2>[^\\]+)"

Ciao.

Giuseppe

[yuanliu]

As I always tell people, do not treat structured data as plain text, and rex is not the right tool for JSON.

Looking at your illustration, I am convinced that your original data is fully compliant; the field message3.TEXT embeds an escaped, fully compliant JSON message with some leading text.  Like thus

{"Field1":"xxx","message1":"{0}","message2":"xxx","message3":{"TEXT":"xxxx: xxx\r\n.xxxxx: {\"xxxxx\":{\"@CDI\":\"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 \",\"@RETURN\":\"xxxx-xxxxxxxxxx-xx-xxxxx\",\"@message4\":\"xxxxxx:xxx\",\"message5\":{\"message6\":null}}}"}}

As such, you can use this to directly access the field RETURN

| eval TEXT = replace('message3.TEXT', "^[^{]+", "")
| spath input=TEXT path="xxxxx.@RETURN" output=Field2

 The illustrated data will give something like

Field1	Field2	message1	message2	message3.TEXT
xxx	xxxx-xxxxxxxxxx-xx-xxxxx	{0}	xxx	xxxx: xxx .xxxxx: {"xxxxx":{"@CDI":"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 ","@RETURN":"xxxx-xxxxxxxxxx-xx-xxxxx","@message4":"xxxxxx:xxx","message5":{"message6":null}}}

Here is an emulation you can play with and compare with raw data

| makeresults
| eval _raw = "{\"Field1\":\"xxx\",\"message1\":\"{0}\",\"message2\":\"xxx\",\"message3\":{\"TEXT\":\"xxxx: xxx\\r\\n.xxxxx: {\\\"xxxxx\\\":{\\\"@CDI\\\":\\\"@ABC-123G-dhskdd-ghdkshd122@hkfhksdf12-djkshd12-hkdshd12 \\\",\\\"@RETURN\\\":\\\"xxxx-xxxxxxxxxx-xx-xxxxx\\\",\\\"@message4\\\":\\\"xxxxxx:xxx\\\",\\\"message5\\\":{\\\"message6\\\":null}}}\"}}"
| spath
``` data emulation above ```

[PickleRick]

While I wholeheartedly agree with the "don't use regex for structured data" it's worth noting that sometimes it's not easy to extract the structured part from the whole event.