how to compose the _time in index time from two JSON fields?

edigilink — Wed, 18 Jul 2018 16:44:28 GMT

I am developing a Python add-on and I am trying to specify a _time composed by two JSON fields lastTstamp and lastDate in the index time. Therefore, the extraction is getting a different and wrong timestamp.

JSON input:
{    
   lastTstamp: 15:32:02Z    
   lastDate: 2015-10-23        
   id: a4ec1ba0-ab74-11e6-a19f-0a7e67dda05f    
   status: new
}

event output: _time: 2015-11-18T05:55:58.000+00:00

So far I tried two approaches:
1st approach: Using helper.new_event + ew.write_event(event)

utc_dt = datetime.strptime(data_json['lastDate'] + 'T' + data_json['lastTstamp'], '%Y-%m-%dT%H:%M:%SZ')

event = helper.new_event(time=time.mktime(utc_dt.timetuple()),
                                     source=helper.get_input_type(),
                                     index=helper.get_output_index(),
                                     sourcetype=helper.get_sourcetype(),
                                     data=json.dumps(data_json))
ew.write_event(event)

2nd approach: Edit props.conf and transforms.conf

transform.conf:

[alert_time]
REGEX = 'lastDate': u'(\d{4}-\d{2}-\d{2}).*lastTstamp': u'(\d{2}:\d{2}:\d{2})
FORMAT = $1T$2.000+00:00
DEST_KEY = _time

props.conf:

 [json_alert]
KV_MODE = json
SHOULD_LINEMERGE = 0
category = Splunk App Add-on Builder
pulldown_type = 1
TRANSFORMS-datetime = alert_time`

I some cases a time zone difference is expected as normal, but as depicted in the example above, there is a huge gap between input and output timestamp.

Re: how to compose the _time in index time from two JSON fields?

edigilink — Fri, 20 Jul 2018 13:49:44 GMT

Solved by setting DATETIME_CONFIG equals NONE in props.conf, which means the extraction will leave the event time set to whatever time was selected by the input layer.

topic how to compose the _time in index time from two JSON fields? in Splunk Dev

how to compose the _time in index time from two JSON fields?

Re: how to compose the _time in index time from two JSON fields?