Getting Data In

How come I can't override _time with props?

nickdewijer
Explorer

Using an HTTP event collector on a heavy forwarder, I receive JSON that comes in as follows:

{
    "env":  "prod",
    "org":  "xxx",
    "percentile":  "95",
    "proxy":  "xxx",
    "region":  "europe-west1",
    "target":  "ALL",
    "time":  "2019-02-26T10:54:00.000+01:00",
    "totalLatency":  362,
    "targetLatency":  359
}

I want to override the indexing _time field with the timefield from the event. I've tried all forms of the following in Props:

[stansa]

TIMESTAMP_FIELDS = time
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
TIME_PREFIX = time\":\s*\"
KV_MODE = json

but the _time sticks to indexing time with the Splunk event looking like this:

time
2019-02-26T10:54:00.000+01:00

_time
2019-02-26T10:55:11.000+01:00

Please help me understand why Splunk stubbornly refuses to recognize me passing it a timestamp.

0 Karma
1 Solution

tiagofbmm
Influencer

tiagofbmm
Influencer

Here is the reason why no TIME parsing is done on JSON Endpoints:

https://answers.splunk.com/answers/411892/json-timestamps-not-parsed-via-http-event-collecto.html

nickdewijer
Explorer

That's amazing, never knew. I've switched over to raw, had to play around with the props AND the data coming into the HEC but it's exactly as I want it now.

0 Karma

tiagofbmm
Influencer

Forget about the TIMESTAM_FIELDS.

Keep the following configurations and make sure these are in the first Heavy Forwarder or Indexer through which the data is going

KV_MODE=json
TIME_PREFIX=time\":\s*\"
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=24

0 Karma

nickdewijer
Explorer

I've implemented this exactly, on the Heavy Forwarder that is hosting the HEC. No effect.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...