topic Re: How to get system time for each events indexed file splunk in Getting Data In

How to get system time for each events indexed file splunk

snehalk — Fri, 02 Jun 2017 09:12:33 GMT

Hello Everyone,

I have text files where there is no datetime in it, but my required is need to get each line as one event with indexing time ( that willbe system time).

I have used below props.conf but still its having same datetime for all the events in the file.

[test]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE = false

That is one file is having 100 lines as events and for all of that it has same timestamp.

Can any one help me where am going wrong

Thanks you

Re: How to get system time for each events indexed file splunk

nit123 — Fri, 02 Jun 2017 09:59:17 GMT

use time.time() to mark each one event by its time of creation to Splunk.

Re: How to get system time for each events indexed file splunk

snehalk — Fri, 02 Jun 2017 10:00:55 GMT

Hello nit123,

Thank you for response, where i need to use this attribute? in props.conf file?

Re: How to get system time for each events indexed file splunk

nit123 — Tue, 29 Sep 2020 14:21:58 GMT

In the python script that pulls data into Splunk and ingests it to some index.
The script will set value to CREATION_DATETIME and LAST_UPDATE_DATETIME in props.conf
In prop.conf , have something like

[StanzaName]
SHOULD_LINEMERGE = true
KV_MODE = auto
TIME_PREFIX=:\s|CREATION_DATETIME="|LAST_UPDATE_DATETIME="
TIME_FORMAT=%Y-%m-%dT%H:%M:%

If this information helps, reward points and accept answer. Thanks.

Re: How to get system time for each events indexed file splunk

snehalk — Tue, 29 Sep 2020 14:22:00 GMT

Hello Nit123,

i have used below props.conf, but still am not getting different timestamp for each events.

[test]
SHOULD_LINEMERGE = true
KV_MODE = auto
TIME_PREFIX=:\s|CREATION_DATETIME="|LAST_UPDATE_DATETIME="
TIME_FORMAT=%Y-%m-%dT%H:%M:%

Re: How to get system time for each events indexed file splunk

nit123 — Mon, 05 Jun 2017 11:20:18 GMT

Can you share the extract of your code for better understanding.

Re: How to get system time for each events indexed file splunk

snehalk — Wed, 07 Jun 2017 07:42:54 GMT

Hello Nit123,
I have fixed length text files where there is no timestamp, and because of this Splunk by default adding 10000(say one file contain) events in one timestamp,
Is there any ways where i can atleast bundle 100 events ?

Re: How to get system time for each events indexed file splunk

yannK — Wed, 07 Jun 2017 19:56:55 GMT

I think that it cannot be achieved if your events do not have a timestamp in it.

As best splunk can assign the current timestamp to the events (DATETIME_CONFIG = CURRENT), it will be the indextime timestamp of the time splunk read the events from the file (not necessarily the mod time of the file or the line in the file).
As splunk will read them in a batch, several events will have the same timestamp.

If you can change your application to write the timestamp in the event, it will be possible.

Re: How to get system time for each events indexed file splunk

somesoni2 — Wed, 07 Jun 2017 20:26:05 GMT

You wanted to assign current time (time when Splunk sees/read the event) to be assigned as timestamp of the event, which it's doing correctly. Splunk has capacity to process multiple events at almost at the same time and they'll have same timestamp. What is your expected behivour?

Re: How to get system time for each events indexed file splunk

snehalk — Wed, 28 Jun 2017 08:22:51 GMT

Hello somesoni2,

Thanks for reply, Yes, as you mentioned its taking almost all the events at same time, and because of this the splunk search performance is not good and also am getting other error too, so because of this i though if splunk has at least each event in different timestamp, then it will resolve all other issues. Is there any way to achieve this?

Thanks