Getting Data In

timestamp extraction not working

davidbann
Explorer

I have an http event collector configured with a heavy forwarder in the DMZ forwarding to an internal Indexer. The timtestamp of all events is being set to the time received, it's not picking up the "time" value from the body despite my props.conf settings. No errors or warnings in "_internal" around timestamp or anything close to it.

Test event sent to the collector:

curl --location --request POST 'https://<redacted>.com/services/collector' \
--header 'Authorization: Splunk <redacted>' \
--header 'Content-Type: application/json' \
--data-raw '{"event": {"time":"2021-02-04 20:20:20.123-05:00","userSettings":{"userId":"ab12345","userName":"ab12345,"site":"000"},"version":5070004},"sourcetype": "st-test"}'

 

shows up as expected in Search results as expected (raw):

{"time":"2021-02-04 20:20:20.123-05:00","userSettings":{"userId":"ab12345","userName":"ab12345","site":"901"},"version":5070004}

 

props.conf for this sourcetype is configured on both the heavy forwarder and internal indexer:

[st-test]
TRUNCATE = 100000
INDEXED_EXTRACTIONS = json
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Structured
disabled = false
pulldown_type = 1
TIME_PREFIX = "time":"
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N%:z
MAX_TIMESTAMP_LOOKAHEAD = 32

 

Any ideas?

0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

@davidbann 

I believe you're possible using the wrong endpoint to support event timestamp extraction.  See

https://docs.splunk.com/Documentation/Splunk/8.1.2/RESTREF/RESTinput#services.2Fcollector.2Fevent

timestamp extraction is a bit finicky with HEC, but there is a short discussion of timestamp extraction there. There is an envelope timestamp and event timestamp and I recall when using this some months back, that you need to use the raw collector endpoint to get timestamp extracted using settings from props.conf. 

I forget all the detail, In our case we needed to resort to using auto_extract_timestamp=true in the /event endpoint. Have a play with this and the /raw endpoints.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

@davidbann 

I believe you're possible using the wrong endpoint to support event timestamp extraction.  See

https://docs.splunk.com/Documentation/Splunk/8.1.2/RESTREF/RESTinput#services.2Fcollector.2Fevent

timestamp extraction is a bit finicky with HEC, but there is a short discussion of timestamp extraction there. There is an envelope timestamp and event timestamp and I recall when using this some months back, that you need to use the raw collector endpoint to get timestamp extracted using settings from props.conf. 

I forget all the detail, In our case we needed to resort to using auto_extract_timestamp=true in the /event endpoint. Have a play with this and the /raw endpoints.

 

davidbann
Explorer

yep that did it. using:

/services/collector/event?auto_extract_timestamp=true

resulted in the timestamp being picked out of the body instead of event time.

Re-reading https://docs.splunk.com/Documentation/Splunk/8.1.1/Data/FormateventsforHTTPEventCollector

I noticed the following: "The HTTP Event Collector endpoint extracts the events from the HTTP request and parses them before sending them to indexers."

I think that explains the "finicky" behavior as it doesn;t follow the same path as other inputs.

 

Thanks @bowesmana !

richgalloway
SplunkTrust
SplunkTrust

When you look at the raw event in Search, what sourcetype is shown?

---
If this reply helps you, Karma would be appreciated.
0 Karma

davidbann
Explorer

st-test as expected

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...