topic Re: timestamp extraction not working in Getting Data In

timestamp extraction not working

davidbann — Thu, 04 Feb 2021 18:40:46 GMT

I have an http event collector configured with a heavy forwarder in the DMZ forwarding to an internal Indexer. The timtestamp of all events is being set to the time received, it's not picking up the "time" value from the body despite my props.conf settings. No errors or warnings in "_internal" around timestamp or anything close to it.

Test event sent to the collector:

curl --location --request POST 'https://<redacted>.com/services/collector' \ --header 'Authorization: Splunk <redacted>' \ --header 'Content-Type: application/json' \ --data-raw '{"event": {"time":"2021-02-04 20:20:20.123-05:00","userSettings":{"userId":"ab12345","userName":"ab12345,"site":"000"},"version":5070004},"sourcetype": "st-test"}'

shows up as expected in Search results as expected (raw):

{"time":"2021-02-04 20:20:20.123-05:00","userSettings":{"userId":"ab12345","userName":"ab12345","site":"901"},"version":5070004}

props.conf for this sourcetype is configured on both the heavy forwarder and internal indexer:

[st-test] TRUNCATE = 100000 INDEXED_EXTRACTIONS = json KV_MODE = none LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true category = Structured disabled = false pulldown_type = 1 TIME_PREFIX = "time":" TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N%:z MAX_TIMESTAMP_LOOKAHEAD = 32

Any ideas?

Re: timestamp extraction not working

richgalloway — Thu, 04 Feb 2021 19:12:09 GMT

When you look at the raw event in Search, what sourcetype is shown?

Re: timestamp extraction not working

davidbann — Thu, 04 Feb 2021 19:42:49 GMT

st-test as expected

Re: timestamp extraction not working

bowesmana — Thu, 04 Feb 2021 21:48:49 GMT

@davidbann

I believe you're possible using the wrong endpoint to support event timestamp extraction. See

https://docs.splunk.com/Documentation/Splunk/8.1.2/RESTREF/RESTinput#services.2Fcollector.2Fevent

timestamp extraction is a bit finicky with HEC, but there is a short discussion of timestamp extraction there. There is an envelope timestamp and event timestamp and I recall when using this some months back, that you need to use the raw collector endpoint to get timestamp extracted using settings from props.conf.

I forget all the detail, In our case we needed to resort to using auto_extract_timestamp=true in the /event endpoint. Have a play with this and the /raw endpoints.

Re: timestamp extraction not working

davidbann — Thu, 04 Feb 2021 23:05:47 GMT

yep that did it. using:

/services/collector/event?auto_extract_timestamp=true

resulted in the timestamp being picked out of the body instead of event time.

Re-reading https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/FormateventsforHTTPEventCollector

I noticed the following: "The HTTP Event Collector endpoint extracts the events from the HTTP request and parses them before sending them to indexers."

I think that explains the "finicky" behavior as it doesn;t follow the same path as other inputs.

Thanks @bowesmana !