Unable to remove prefix by SEDCMD in sourcetype but able to run in search

[Alan_Chan]

I am trying to remove everything before the { character to preserve the JSON format. I am using SEDCMD-keepjson = s/^[^{]{/{/ in the sourcetype configuration, but it fails to apply correctly. However, when I use the search command | rex mode=sed "s/^[^{]{/{/", it successfully removes the unwanted text. I am wondering what could be causing this issue. The sourcetype settings are configured on both the Search Head (SH) and Heavy Forwarder (HF)

Mar 28 13:11:57 abcdeabcdev01w.abcdabcd.local {<json_log>}

[livehybrid]

Hi @Alan_Chan 

Ive checked this config locally and it does work for your sample event so something else isnt right here. I think there is a typo in what you posted so I used the value from the screenshot, but please confirm you have the asterisk in your SEDCMD that is deployed?

Is the screenshot you sent from the Searchhead? Is the exact same config deployed to the Heavy Forwarder? And is this the only (or first) HF that the data lands on? 
How is the data arriving into the HF? If it is via HEC using the event endpoint then this configuration will not work and you would need to use INGEST_EVAL or move to the raw HEC endpoint.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[Alan_Chan]

Hi @livehybrid 
The screenshot I sent is from the Search Head and shows the exact same configuration deployed to the Heavy Forwarder. This is the first Heavy Forwarder that the data lands on. The data is sent to the Heavy Forwarder using rsyslog, and the Heavy Forwarder uses [monitor:] to monitor the logs.

[gcusello]

Hi @Alan_Chan ,

this transformation is unuseful on the SH, but it must be localized in the first HF that dara pass trhough, are you sure that you applied it in the first HF?

Then check if the sourcetype where you associated the SEDCMD command is the correct one, and that there isn't any transformation on this sourcetype.

Then, are you sure that is useful to remove these few chars?

Ciao.

Giuseppe