Getting Data In
Highlighted

Unable to extract timestamp from incoming API POST

Contributor

I have an index cluster with load balancer
a curl sending a JSON event to HEC

curl http://indexers-amazonaws.com:8088/services/collector -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","event": {"timestamp": "Sun Aug 11 19:00:00 GMT+10:00 2019","Username": "joblogs", "requestID": "???", "access-level": "1", "authentication": "success"}}'

Props that appears to work when I do it manually through data input

[bma]
INDEXED_EXTRACTIONS = json
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
TIME_PREFIX = {"timestamp": "
category = Structured
disabled = false
pulldown_type = 1

I've tried numerous variation of props

What am I missing???

0 Karma
Highlighted

Re: Unable to extract timestamp from incoming API POST

SplunkTrust
SplunkTrust

Have you tried adding TIME_FORMAT = %a %b %d %H:%M:%S %Z%:z %Y" to props.conf?

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: Unable to extract timestamp from incoming API POST

Contributor

Thanks Rich, yes I had

0 Karma
Highlighted

Re: Unable to extract timestamp from incoming API POST

Esteemed Legend

When using the /collector/event endpoint, you need to supply your timestamp while formatting your event, along with sourcetype, source and host; if you want to extract the timestamp from your raw data then you need to use the /collector/raw HEC endpoint instead.

Highlighted

Re: Unable to extract timestamp from incoming API POST

Contributor

Thank you sir, very good.
I figured after to much playing around with props that my problem was with the event.
I had tried adding time as suggested but got errors, didn't realise it needed to be epoch time.

For those who are interested this format works:
curl http://indexers-amazonaws.com:8088/services/collector/event -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","time": "1565561700","event": {"Username": "jobloggs", "tokenID": "???", "access-level": "1", "authentication": "success"}}'

And the raw option also does the trick but the final event is not as tidy

Highlighted

Re: Unable to extract timestamp from incoming API POST

Esteemed Legend

You should unaccept my answer and accept your because mine was not it. Feel free to UpVote, though!

0 Karma
Highlighted

Re: Unable to extract timestamp from incoming API POST

Contributor

I had tried adding time as suggested but got errors, didn't realise it needed to be epoch time.
For those who are interested this format works:
curl http://indexers-amazonaws.com:8088/services/collector/event -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","time": "1565561700","event": {"Username": "jobloggs", "tokenID": "???", "access-level": "1", "authentication": "success"}}'

View solution in original post

0 Karma