Getting Data In

How do you set custom timestamp for data ingested via Splunk SDK?

awalton
Splunk Employee
Splunk Employee

Using the Splunk SDK, I am ingesting json data into a splunk index via this line of code: 

index.submit(event, host="localhost", sourcetype="covid_vacc_data_ingest")

This line of code is working and data is ingested, but the timestamp is always the ingestion time rather then the date field on the event. Here is a screenshot of my settings in Splunk enterprise for this sourcetype: 

Screen Shot 2021-08-07 at 11.04.59 PM.png

Here is a screenshot of what the ingested data looks like: 

Screen Shot 2021-08-07 at 11.23.42 PM.png

I want the _time field on the left to be the date field on the right. Any suggestions? Not sure what I am doing wrong.

Thank you! 

Labels (1)
0 Karma

jhanvidattani
Path Finder

@awalton 

Can you add below setting in props.conf for json data ingested:

 

[covid_vacc_data_ingest]
DATETIME_CONFIG = None 
TIME_PREFIX = "date": 
TIME_FORMAT = %Y-%M-%D
...

 

DATETIME_CONFIG: Splunk will not to take any timestamp by default by this config
TIME_FORMAT and TIME_PREFIX: Splunk will take timestamp of format %Y-%M-%D from date field

If you find my solution fruitful an upvote will be appreciated.

0 Karma

awalton
Splunk Employee
Splunk Employee

@jhanvidattani  Thank you for the response.

This solution still did not work for me. What's strange is that if i manually upload the data via a file and assign it my sourcetype, it actually extracts the date field and assigns that as the timestamp. So, it must be something specifically happening when i ingest via the splunk sdk, correct?

0 Karma

jhanvidattani
Path Finder

Can you confirm whether the sdk sets the value of _time from code?

 

 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Cloud Application Management in Terraform

Now On-Demand   We’re diving into how you can bring Infrastructure as Code (IaC) principles to your Splunk ...

What's New in Splunk Enterprise Security (ES) 8.6

Purpose-Built AI Agents for the Agentic SOC  Splunk Enterprise Security 8.6 expands AI in Security with ...

From Raw Data to Executive-Ready Stories, Faster

Build Data Stories for Every Audience  A dashboard is rarely just a dashboard. It might be the view an ...