Getting Data In

How to get system time for each events indexed file splunk

snehalk
Communicator

Hello Everyone,

I have text files where there is no datetime in it, but my required is need to get each line as one event with indexing time ( that willbe system time).

I have used below props.conf but still its having same datetime for all the events in the file.

[test]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false

That is one file is having 100 lines as events and for all of that it has same timestamp.

Can any one help me where am going wrong

Thanks you

0 Karma

somesoni2
Revered Legend

You wanted to assign current time (time when Splunk sees/read the event) to be assigned as timestamp of the event, which it's doing correctly. Splunk has capacity to process multiple events at almost at the same time and they'll have same timestamp. What is your expected behivour?

0 Karma

snehalk
Communicator

Hello somesoni2,

Thanks for reply, Yes, as you mentioned its taking almost all the events at same time, and because of this the splunk search performance is not good and also am getting other error too, so because of this i though if splunk has at least each event in different timestamp, then it will resolve all other issues. Is there any way to achieve this?

Thanks

0 Karma

yannK
Splunk Employee
Splunk Employee

I think that it cannot be achieved if your events do not have a timestamp in it.

As best splunk can assign the current timestamp to the events (DATETIME_CONFIG = CURRENT), it will be the indextime timestamp of the time splunk read the events from the file (not necessarily the mod time of the file or the line in the file).
As splunk will read them in a batch, several events will have the same timestamp.

If you can change your application to write the timestamp in the event, it will be possible.

0 Karma

nit123
Path Finder

use time.time() to mark each one event by its time of creation to Splunk.

0 Karma

snehalk
Communicator

Hello nit123,

Thank you for response, where i need to use this attribute? in props.conf file?

0 Karma

nit123
Path Finder

In the python script that pulls data into Splunk and ingests it to some index.
The script will set value to CREATION_DATETIME and LAST_UPDATE_DATETIME in props.conf
In prop.conf , have something like

[StanzaName]
SHOULD_LINEMERGE = true
KV_MODE = auto
TIME_PREFIX=:\s|CREATION_DATETIME="|LAST_UPDATE_DATETIME="
TIME_FORMAT=%Y-%m-%dT%H:%M:%

If this information helps, reward points and accept answer. Thanks.

0 Karma

snehalk
Communicator

Hello Nit123,

i have used below props.conf, but still am not getting different timestamp for each events.

[test]
SHOULD_LINEMERGE = true
KV_MODE = auto
TIME_PREFIX=:\s|CREATION_DATETIME="|LAST_UPDATE_DATETIME="
TIME_FORMAT=%Y-%m-%dT%H:%M:%

0 Karma

nit123
Path Finder

Can you share the extract of your code for better understanding.

0 Karma

snehalk
Communicator

Hello Nit123,
I have fixed length text files where there is no timestamp, and because of this Splunk by default adding 10000(say one file contain) events in one timestamp,
Is there any ways where i can atleast bundle 100 events ?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...