topic Re: Strip leading junk to index json in Getting Data In

Strip leading junk to index json

ntripp_element — Wed, 01 Jul 2020 18:20:32 GMT

I have some json coming in from syslog and there's a bunch of added junk that's inhibiting the field extraction.

I tried

transforms.conf

[eliminate_header]
REGEX=s/^[^{]+//g
DEST_KEY=queue
FORMAT=nullQueue

and

props.conf

[syslog_cue]
TRANSFORMS-t1=eliminate_header

and a couple other things but can't quite seem to get the outcome I want. Testing the regex works, but isn't helping in the way I need it to.

Re: Strip leading junk to index json

richgalloway — Wed, 01 Jul 2020 18:35:38 GMT

Please provide examples of the original data and the desired output. It's almost impossible to solve regex problems without that.

That said, the REGEX line shown is not valid in transforms.conf. Sed commands are not supported so the regex contains unescaped delimiter characters (slashes).

Additionally, FORMAT=nullQueue discards entire events that match the REGEX, not just the text that matches.

You probably want to use SEDCMD in props.conf.