Replacing backslash not working in SEDCMD after re-directing through transforms.conf and applying it in props.conf.

[chaitalynavare]

Hi,

I am trying to escape backslash character from json data. It works when I apply SEDCMD definations in props.conf soucetype - mysrc. But when I re-direct the definations to transforms.conf (custom_data_one and custom_data_two) to transform data for particular pattern & extract required data from the json event, and then apply the SEDCMD in mentioned sourcetype (mysrc_two) it doesnt works.
Please share your thoughts on this.

Data:

{"docker":{"container_id":"852241528698541tzfjztdgtzjsxf"},"kubernetes":{"container_name":"a-kterminal","namespace_name":"kterminal","pod_name":"a-kterminal-555-85chghv","pod_id":"858gh-zgzh-gjh-ghg-896545213","labels":{"application":"a-kterminal","createdBy":"k-rass-template","deployment":"a-kterminal-555","deploymentConfig":"a-kterminal","deploymentconfig":"a-kterminal"},"host":"sdeb-gv-g58","master_url":"https://kubernetes.default.hgfbsjbgsk","namespace_id":"uzsefgvshj-dsgfvjhdv-ztfvsjhybv","namespace_labels":{"app_code":"mycode","network-policy":"true","splunk":"true","splunkindex":"myindex"}},"message":"2019-11-04 14:07:12.321 TRACE 1 --- [nio-8080-exec-4] c.k.k.d.trackinglogger.TrackingLogger    : {\"timeStamp\":\"2019-11-04T14:07:12.321Z\",\"country\":\"DE\",\"environment\":\"at\",\"payload\":\"/bye/0\",\"loggingVersion\":\"1.0.0\",\"sessionId\":\"uzsefgvshj-dsgfvjhdv-ztfvsjhybv\",\"terminalId\":\"ABC-12345TST0103\",\"storeId\":\"8950\",\"floor\":\"0\",\"type\":\"System\"}\n","level":"info","hostname":"abc-555-g85","pipeline_metadata":{"collector":{"ipaddr4":"123.12.00.123","ipaddr6":"abc::abc5:abc54:a12:12a","inputname":"fluent-plugin-systemd","name":"fluentd","received_at":"2019-11-04T14:07:13.101993+00:00","version":"0.12.43 1.6.0"}},"@timestamp":"2019-11-04T14:07:12.321816+00:00","viaq_index_name":"project.kterminal.uzsefgvshj-dsgfvjhdv-ztfvsjhybv","viaq_msg_id":"uzsefgvshj-dsgfvjhdv-ztfvsjhybv","forwarded_by":"splunk-connect-1-854ik","source_component":"t01"}

Data from which all backslash () need to be removed to view the data in proper json format:

{\"timeStamp\":\"2019-11-04T14:07:12.321Z\",\"country\":\"DE\",\"environment\":\"at\",\"payload\":\"/bye/0\",\"loggingVersion\":\"1.0.0\",\"sessionId\":\"uzsefgvshj-dsgfvjhdv-ztfvsjhybv\",\"terminalId\":\"ABC-12345TST0103\",\"storeId\":\"8950\",\"floor\":\"0\",\"type\":\"System\"}

Configurations :-

OLD CONFIG which works as expected -
props.conf

[mysrc]
TRUNCATE = 0
CHARSET = UTF-8
KV_MODE=JSON
SHOULD_LINEMERGE=false
SEDCMD-remove_header = s/{\"docker.*\,\"message":.*\s+\:\s+//g
SEDCMD-remove_footer = s/\\n"\,\"level"\:.*//g
SEDCMD-replace_backslash = s/\\//g

NEW CONFIG which doesnt remove backslash -

[mysrc_one]
TRUNCATE = 0
CHARSET = UTF-8
KV_MODE=JSON
SHOULD_LINEMERGE=false
TRANSFORMS-kdt-one = custom_data_one
TRANSFORMS-kdt-two = custom_data_two

[mysrc_two]
TRUNCATE = 0
CHARSET = UTF-8
KV_MODE=JSON
SHOULD_LINEMERGE=false
SEDCMD-replace_backslash = s/\\//g

transforms.conf

[custom_data_one]
REGEX = "splunkindex":"myindex"
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::mysrc_two

[custom_data_two]
REGEX = ({\"docker.*"splunkindex":"myindex"}},\"message":.*\s+\:\s+)(.*)(\\n"\,\"level"\:.*)
DEST_KEY = _raw
FORMAT = $2

Thanks!
Chiataly Navare

[FrankVl · Accepted solution]

Any index-time props.conf settings are only applied based on the original sourcetype. Splunk does not process index-time props.conf settings (like SEDCMD) again, after rewriting the sourcetype using TRANSFORMS.

[chaitalynavare]

Yeah looks like you are right. Whatever changes I do to transforms.conf and primary sourcetype they are adapted quikly but the secondry sourcetype doesnt work at all.

But when I went through the below link, I learnt that the props SEDCMD and transforms thing happen at same time. (I am reffering to regex replacement block here) -

https://wiki.splunk.com/Community:HowIndexingWorks

[FrankVl]

Correct, they both take place in the same phase. Not sure how it exactly works under the hood, but basically when the event enters the phase, splunk looks at host, source and sourcetype and based on that determines what props and transforms to apply during typing phase and then applies all those in a certain order. Those props and transforms may change host, source and sourcetype, but this does not trigger Splunk to re-evaluate what props and transforms to apply.

Changing those 3 fields is basically only useful for search time.

The only exception is when you do a CLONE_SOURCETYPE transform, there the cloned events are injected back in to the start of the pipeline and SED etc. is applied as if it had come in originally with the new sourcetype. But I guess you're not interested in cloning anything.

[gcusello]

Hi @chaitalynavare,
in your regexes there are some special chars no escaped (as { or "), so try something like this:
In props.conf:

SEDCMD-remove_header = s/\{\"docker.*,\"message\":.*\s+:\s+//g
SEDCMD-remove_footer = s/\\n\"\,\"level\":.*//g
SEDCMD-replace_backslash = s/\\//g

In transforms.conf:

REGEX = \"splunkindex\":\"myindex\"    
REGEX = (\{\"docker.*\"splunkindex\":\"myindex\"}},\"message\":.*\s+:\s+)(.*)(\\n\",\"level\":.*)

Ciao.
Giuseppe

[chaitalynavare]

Hi @gcusello,

I am able to extract the required data as expected but revoming backslash not working in props (Refer NEW CONFIG from question above). Whereas the same config works if I apply without involving transforms.conf (Refer OLD CONFIG from question above).

NOTE: I am using NEW configs as I want to limit the slash to be removed and extract required data only from specific pattern ("splunkindex":"myindex").

Thanks,
Chaitaly Navare