Getting Data In

How do you set custom timestamp for data ingested via Splunk SDK?

awalton
Splunk Employee
Splunk Employee

Using the Splunk SDK, I am ingesting json data into a splunk index via this line of code: 

index.submit(event, host="localhost", sourcetype="covid_vacc_data_ingest")

This line of code is working and data is ingested, but the timestamp is always the ingestion time rather then the date field on the event. Here is a screenshot of my settings in Splunk enterprise for this sourcetype: 

Screen Shot 2021-08-07 at 11.04.59 PM.png

Here is a screenshot of what the ingested data looks like: 

Screen Shot 2021-08-07 at 11.23.42 PM.png

I want the _time field on the left to be the date field on the right. Any suggestions? Not sure what I am doing wrong.

Thank you! 

Labels (1)
0 Karma

jhanvidattani
Path Finder

@awalton 

Can you add below setting in props.conf for json data ingested:

 

[covid_vacc_data_ingest]
DATETIME_CONFIG = None 
TIME_PREFIX = "date": 
TIME_FORMAT = %Y-%M-%D
...

 

DATETIME_CONFIG: Splunk will not to take any timestamp by default by this config
TIME_FORMAT and TIME_PREFIX: Splunk will take timestamp of format %Y-%M-%D from date field

If you find my solution fruitful an upvote will be appreciated.

0 Karma

awalton
Splunk Employee
Splunk Employee

@jhanvidattani  Thank you for the response.

This solution still did not work for me. What's strange is that if i manually upload the data via a file and assign it my sourcetype, it actually extracts the date field and assigns that as the timestamp. So, it must be something specifically happening when i ingest via the splunk sdk, correct?

0 Karma

jhanvidattani
Path Finder

Can you confirm whether the sdk sets the value of _time from code?

 

 

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...