Getting Data In

Unable to extract timestamp from incoming API POST

proylea
Contributor

I have an index cluster with load balancer
a curl sending a JSON event to HEC

curl http://indexers-amazonaws.com:8088/services/collector -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","event": {"timestamp": "Sun Aug 11 19:00:00 GMT+10:00 2019","Username": "joblogs", "requestID": "???", "access-level": "1", "authentication": "success"}}'

Props that appears to work when I do it manually through data input

[bma]
INDEXED_EXTRACTIONS = json
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
TIME_PREFIX = {"timestamp": "
category = Structured
disabled = false
pulldown_type = 1

I've tried numerous variation of props

What am I missing???

0 Karma
1 Solution

proylea
Contributor

I had tried adding time as suggested but got errors, didn't realise it needed to be epoch time.
For those who are interested this format works:
curl http://indexers-amazonaws.com:8088/services/collector/event -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","time": "1565561700","event": {"Username": "jobloggs", "tokenID": "???", "access-level": "1", "authentication": "success"}}'

View solution in original post

0 Karma

proylea
Contributor

I had tried adding time as suggested but got errors, didn't realise it needed to be epoch time.
For those who are interested this format works:
curl http://indexers-amazonaws.com:8088/services/collector/event -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","time": "1565561700","event": {"Username": "jobloggs", "tokenID": "???", "access-level": "1", "authentication": "success"}}'

0 Karma

woodcock
Esteemed Legend

When using the /collector/event endpoint, you need to supply your timestamp while formatting your event, along with sourcetype, source and host; if you want to extract the timestamp from your raw data then you need to use the /collector/raw HEC endpoint instead.

proylea
Contributor

Thank you sir, very good.
I figured after to much playing around with props that my problem was with the event.
I had tried adding time as suggested but got errors, didn't realise it needed to be epoch time.

For those who are interested this format works:
curl http://indexers-amazonaws.com:8088/services/collector/event -H 'Authorization: Splunk ???' -d '{"sourcetype": "bma","time": "1565561700","event": {"Username": "jobloggs", "tokenID": "???", "access-level": "1", "authentication": "success"}}'

And the raw option also does the trick but the final event is not as tidy

woodcock
Esteemed Legend

You should unaccept my answer and accept your because mine was not it. Feel free to UpVote, though!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have you tried adding TIME_FORMAT = %a %b %d %H:%M:%S %Z%:z %Y" to props.conf?

---
If this reply helps you, Karma would be appreciated.
0 Karma

proylea
Contributor

Thanks Rich, yes I had

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...