Getting Data In

timestamp extraction not working

davidbann
Explorer

I have an http event collector configured with a heavy forwarder in the DMZ forwarding to an internal Indexer. The timtestamp of all events is being set to the time received, it's not picking up the "time" value from the body despite my props.conf settings. No errors or warnings in "_internal" around timestamp or anything close to it.

Test event sent to the collector:

curl --location --request POST 'https://<redacted>.com/services/collector' \
--header 'Authorization: Splunk <redacted>' \
--header 'Content-Type: application/json' \
--data-raw '{"event": {"time":"2021-02-04 20:20:20.123-05:00","userSettings":{"userId":"ab12345","userName":"ab12345,"site":"000"},"version":5070004},"sourcetype": "st-test"}'

 

shows up as expected in Search results as expected (raw):

{"time":"2021-02-04 20:20:20.123-05:00","userSettings":{"userId":"ab12345","userName":"ab12345","site":"901"},"version":5070004}

 

props.conf for this sourcetype is configured on both the heavy forwarder and internal indexer:

[st-test]
TRUNCATE = 100000
INDEXED_EXTRACTIONS = json
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Structured
disabled = false
pulldown_type = 1
TIME_PREFIX = "time":"
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N%:z
MAX_TIMESTAMP_LOOKAHEAD = 32

 

Any ideas?

0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

@davidbann 

I believe you're possible using the wrong endpoint to support event timestamp extraction.  See

https://docs.splunk.com/Documentation/Splunk/8.1.2/RESTREF/RESTinput#services.2Fcollector.2Fevent

timestamp extraction is a bit finicky with HEC, but there is a short discussion of timestamp extraction there. There is an envelope timestamp and event timestamp and I recall when using this some months back, that you need to use the raw collector endpoint to get timestamp extracted using settings from props.conf. 

I forget all the detail, In our case we needed to resort to using auto_extract_timestamp=true in the /event endpoint. Have a play with this and the /raw endpoints.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

@davidbann 

I believe you're possible using the wrong endpoint to support event timestamp extraction.  See

https://docs.splunk.com/Documentation/Splunk/8.1.2/RESTREF/RESTinput#services.2Fcollector.2Fevent

timestamp extraction is a bit finicky with HEC, but there is a short discussion of timestamp extraction there. There is an envelope timestamp and event timestamp and I recall when using this some months back, that you need to use the raw collector endpoint to get timestamp extracted using settings from props.conf. 

I forget all the detail, In our case we needed to resort to using auto_extract_timestamp=true in the /event endpoint. Have a play with this and the /raw endpoints.

 

davidbann
Explorer

yep that did it. using:

/services/collector/event?auto_extract_timestamp=true

resulted in the timestamp being picked out of the body instead of event time.

Re-reading https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/FormateventsforHTTPEventCollector

I noticed the following: "The HTTP Event Collector endpoint extracts the events from the HTTP request and parses them before sending them to indexers."

I think that explains the "finicky" behavior as it doesn;t follow the same path as other inputs.

 

Thanks @bowesmana !

richgalloway
SplunkTrust
SplunkTrust

When you look at the raw event in Search, what sourcetype is shown?

---
If this reply helps you, Karma would be appreciated.
0 Karma

davidbann
Explorer

st-test as expected

0 Karma
Get Updates on the Splunk Community!

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...