topic Re: How do you set custom timestamp for data ingested via Splunk SDK? in Getting Data In

How do you set custom timestamp for data ingested via Splunk SDK?

awalton — Sun, 08 Aug 2021 06:27:49 GMT

Using the Splunk SDK, I am ingesting json data into a splunk index via this line of code:

index.submit(event, host="localhost", sourcetype="covid_vacc_data_ingest")

This line of code is working and data is ingested, but the timestamp is always the ingestion time rather then the date field on the event. Here is a screenshot of my settings in Splunk enterprise for this sourcetype:

Here is a screenshot of what the ingested data looks like:

I want the _time field on the left to be the date field on the right. Any suggestions? Not sure what I am doing wrong.

Thank you!

Re: How do you set custom timestamp for data ingested via Splunk SDK?

jhanvidattani — Sun, 08 Aug 2021 12:57:04 GMT

@awalton

Can you add below setting in props.conf for json data ingested:

[covid_vacc_data_ingest] DATETIME_CONFIG = None TIME_PREFIX = "date": TIME_FORMAT = %Y-%M-%D ...

DATETIME_CONFIG: Splunk will not to take any timestamp by default by this config
TIME_FORMAT and TIME_PREFIX: Splunk will take timestamp of format %Y-%M-%D from date field

If you find my solution fruitful an upvote will be appreciated.

Re: How do you set custom timestamp for data ingested via Splunk SDK?

awalton — Tue, 10 Aug 2021 00:43:04 GMT

@jhanvidattani Thank you for the response.

This solution still did not work for me. What's strange is that if i manually upload the data via a file and assign it my sourcetype, it actually extracts the date field and assigns that as the timestamp. So, it must be something specifically happening when i ingest via the splunk sdk, correct?

Re: How do you set custom timestamp for data ingested via Splunk SDK?

jhanvidattani — Tue, 10 Aug 2021 17:10:32 GMT

Can you confirm whether the sdk sets the value of _time from code?