How can I set _time for a json payload that has to first be restructured with SEDCMD?

[andrewtrobec]

Hello!  I realize that the question is a bit particular, so I will try to explain through an example.

I am indexing a json that looks like this with escaped characters and leading/trailing quotes:

"{\"data\": {\"essentials\": {\"monitorCondition\": \"Resolved\",\"firedDateTime\": \"2022-09-26T14:56:41.7862462Z\",\"resolvedDateTime\": \"2022-09-26T15:02:47.9852843Z\"}}}"

I need to associated _time to the following statement:

If monitorCondition=Fired then parse firedDateTime as _time, otherwise parse resolvedDateTime as _time.

Since the json is not understood directly by Splunk due to the escaped quotes I am attempting the following:

1. format the _raw correctly so that it is interpreted correctly by Splunk.
2. calculate the value to use as timestamp
3. associate timestamp to the _time field 

This is my props.conf so far:

[json_test_st]

KV_MODE = json

DATETIME_CONFIG = 

LINE_BREAKER = ([\r\n]+)

MAX_TIMESTAMP_LOOKAHEAD = 500

NO_BINARY_CHECK = true

TZ = GMT

category = Custom

disabled = false

pulldown_type = 1

SEDCMD-formatjson = s/\\|^\"|\"$//g 

TRANSFORMS = gettime

TIMESTAMP_FIELDS = timestamp

TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%7NZ

This is my transforms.conf

[gettime]

INGEST_EVAL = timestamp=if('data.essentials.monitorCondition' = "Fired",'data.essentials.firedDateTime','data.essentials.resolvedDateTime')

The result is that I can get Splunk to parse the json correctly, but it does not extract the timestamp.

Could anybody give me a push in the right direction?

Thank you and best regards,

Andrew

[johnhuang]

Put this in your transforms.conf instead of INGEST_EVAL.  This regex works off the _raw event example you've provided. Disable the SEDCMD.

REGEX = \x5c\x22monitorCondition\x5c\x22[^\w]*(((Fired)[^\w]*firedDateTime)|(.*resolvedDateTime))[^\w]*(?<timestamp>[\w\d\:\-\.]*)