Getting Data In

How can I set _time for a json payload that has to first be restructured with SEDCMD?

andrewtrobec
Motivator

Hello!  I realize that the question is a bit particular, so I will try to explain through an example.

I am indexing a json that looks like this with escaped characters and leading/trailing quotes:

"{\"data\": {\"essentials\": {\"monitorCondition\": \"Resolved\",\"firedDateTime\": \"2022-09-26T14:56:41.7862462Z\",\"resolvedDateTime\": \"2022-09-26T15:02:47.9852843Z\"}}}"

I need to associated _time to the following statement:

If monitorCondition=Fired then parse firedDateTime as _time, otherwise parse resolvedDateTime as _time.

Since the json is not understood directly by Splunk due to the escaped quotes I am attempting the following:

  1. format the _raw correctly so that it is interpreted correctly by Splunk.
  2. calculate the value to use as timestamp
  3. associate timestamp to the _time field 

This is my props.conf so far:

[json_test_st]
KV_MODE = json
DATETIME_CONFIG = 
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 500
NO_BINARY_CHECK = true
TZ = GMT
category = Custom
disabled = false
pulldown_type = 1
SEDCMD-formatjson = s/\\|^\"|\"$//g 
TRANSFORMS = gettime
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%7NZ
 
This is my transforms.conf
 
[gettime]
INGEST_EVAL = timestamp=if('data.essentials.monitorCondition' = "Fired",'data.essentials.firedDateTime','data.essentials.resolvedDateTime')

 

The result is that I can get Splunk to parse the json correctly, but it does not extract the timestamp.

Could anybody give me a push in the right direction?

Thank you and best regards,

Andrew

Labels (4)
0 Karma

johnhuang
Motivator

Put this in your transforms.conf instead of INGEST_EVAL.  This regex works off the _raw event example you've provided. Disable the SEDCMD.

REGEX = \x5c\x22monitorCondition\x5c\x22[^\w]*(((Fired)[^\w]*firedDateTime)|(.*resolvedDateTime))[^\w]*(?<timestamp>[\w\d\:\-\.]*)

 

0 Karma
Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...