topic Re: How come I can't override _time with props? in Getting Data In

How come I can't override _time with props?

nickdewijer — Tue, 26 Feb 2019 10:08:34 GMT

Using an HTTP event collector on a heavy forwarder, I receive JSON that comes in as follows:

{
    "env":  "prod",
    "org":  "xxx",
    "percentile":  "95",
    "proxy":  "xxx",
    "region":  "europe-west1",
    "target":  "ALL",
    "time":  "2019-02-26T10:54:00.000+01:00",
    "totalLatency":  362,
    "targetLatency":  359
}

I want to override the indexing _time field with the timefield from the event. I've tried all forms of the following in Props:

[stansa]

TIMESTAMP_FIELDS = time
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX = time\":\s*\"
KV_MODE = json

but the _time sticks to indexing time with the Splunk event looking like this:

time
2019-02-26T10:54:00.000+01:00

_time
2019-02-26T10:55:11.000+01:00

Please help me understand why Splunk stubbornly refuses to recognize me passing it a timestamp.

Re: How come I can't override _time with props?

tiagofbmm — Tue, 29 Sep 2020 23:21:20 GMT

Forget about the TIMESTAM_FIELDS.

Keep the following configurations and make sure these are in the first Heavy Forwarder or Indexer through which the data is going

KV_MODE=json
TIME_PREFIX=time\":\s*\"
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=24

Re: How come I can't override _time with props?

nickdewijer — Wed, 27 Feb 2019 12:54:55 GMT

I've implemented this exactly, on the Heavy Forwarder that is hosting the HEC. No effect.

Re: How come I can't override _time with props?

tiagofbmm — Wed, 27 Feb 2019 15:11:38 GMT

Here is the reason why no TIME parsing is done on JSON Endpoints:

https://answers.splunk.com/answers/411892/json-timestamps-not-parsed-via-http-event-collecto.html

Re: How come I can't override _time with props?

nickdewijer — Thu, 28 Feb 2019 11:42:23 GMT

That's amazing, never knew. I've switched over to raw, had to play around with the props AND the data coming into the HEC but it's exactly as I want it now.