Extract timestamp from two json fields

alec_stan — Fri, 15 Mar 2024 21:43:23 GMT

I need to extract timestamp from a JSON log where date and time are on two separate fields. Example below:

{ "Date": 240315, "EMVFallback": false, "FunctionCode": 80, "Time": 154915 }

Date here is equivalent of 2024-March-15 and the time is 15:49:15 pm.

I am struggling to find a way to extract timestamp using props.conf. May you please assist.

Re: Extract timestamp from two json fields

tscroggins — Sat, 16 Mar 2024 16:45:11 GMT

Hi @alec_stan,

You can extract the timestamp using INGEST_EVAL in transforms.conf referenced by a TRANSFORMS setting in props.conf.

If your source type has INDEXED_EXTRACTIONS = json, you can reference the Date and Time fields directly in your INGEST_EVAL expression; otherwise, you can use JSON eval functions to extract the Date and Time values from _raw.

### with INDEXED_EXTRACTIONS # props.conf [alec_stan_json] INDEXED_EXTRACTIONS = json TRANSFORMS-alec_stan_json_time = alec_stan_json_time # transforms.conf [alec_stan_json_time] INGEST_EVAL = _time:=strptime(tostring(Date).tostring(Time), "%y%m%d%H%M%S") ### without INDEXED_EXTRACTIONS # props.conf [alec_stan_json] TRANSFORMS-alec_stan_json_time = alec_stan_json_time # transforms.conf [alec_stan_json_time] INGEST_EVAL = _time:=strptime(tostring(json_extract(json(_raw), "Date")).tostring(json_extract(json(_raw), "Time")), "%y%m%d%H%M%S")

If the event time zone differs from the receiver time zone, add a time zone string (%Z) or offset (%z) to the eval expression:

[alec_stan_json_time] INGEST_EVAL = _time:=strptime(tostring(Date).tostring(Time)."EDT", "%y%m%d%H%M%S%Z")

In a typical environment, deploy props.conf to universal forwarders and props.conf and transforms.conf to receivers (heavy forwarders and indexers).

If you haven't already, you should add SHOULD_LINEMERGE, LINE_BREAKER, etc. settings to props.conf to correctly break your input into events. You can also set DATETIME_CONFIG = CURRENT or DATETIME_CONFIG = NONE to help Splunk skip automatic timestamp extraction logic since you'll be extracting the timestamp using INGEST_EVAL.

topic Extract timestamp from two json fields in Getting Data In

Extract timestamp from two json fields

Re: Extract timestamp from two json fields