New source type with regex not working correctly

[tchamp]

I've created a new source type with a regex. It was working but I found an edge case where it was broken. I rewrote the regex using a capture group but the group doesn't seem to be  getting applied. Can someone tell me if this should work?

Here is my regex: s/"message":\s*"{([\s\S]*)}"/"data": {$1}/g put in "SEDCMD-a". 

My json data is as follows:

{
  "message": "{
      "test": "test data"
  }"
}

and my transformed data ends up like this:

{  "data": {$1}}

It isn't making the replacement with the capture group.

Am I doing something wrong? Should this work?

Thanks,

-Tim

[livehybrid]

Hi @tchamp 

Check out my other response first, but you may also be able to achieve this with an INGEST_EVAL

== props.conf ==
[yourSourceType]
TRANSFORMS-extractMessage = myExtract

== transforms.comf ==
[myExtract]
INGEST_EVAL = _raw:=json_extract(_raw,"message")

Visualised as SPL this would be

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[livehybrid · Accepted solution]

Hi @tchamp 

I think you need to replace the $1 capture group with a \1 

Here is an example in SPL 

| makeresults 
|eval _raw="{
  \"message\": \"{
      \"test\": \"test data\"
  }\"
}
"
|rex mode=sed "s/\"message\":\s*\"{([\s\S]*)}\"/\"data\": {\1}/g"

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[tchamp]

This was the answer. I appreciate the help.