topic Re: DATETIME_CONFIG = NONE doesn't work in Getting Data In

DATETIME_CONFIG = NONE doesn't work

hongduan — Mon, 28 Sep 2020 16:08:29 GMT

I have this stanza in my props.conf
[test_test]
TZ = US/Eastern
SHOULD_LINEMERGE = false
MAX_DAYS_HENCE = 5
TRUNCATE = 0
DATETIME_CONFIG = NONE

And this setting in inputs.conf
[monitor:///test_folder/test.txt]
disabled = 0
index = test_index
sourcetype = test_test
crcSalt =

why splunk is still trying to parse the timestamp in each line of test.txt instead of using the modified time of test.txt?

Re: DATETIME_CONFIG = NONE doesn't work

linu1988 — Fri, 14 Mar 2014 15:42:05 GMT

where is you props.conf file placed?

Re: DATETIME_CONFIG = NONE doesn't work

hongduan — Fri, 14 Mar 2014 17:18:18 GMT

I am sure the props.conf file is placed in the correct location. cause other changes I made to props.conf are picked up by splunk, only this DATATIME_CONFIG = NONE not work.

Re: DATETIME_CONFIG = NONE doesn't work

linu1988 — Mon, 28 Sep 2020 16:08:42 GMT

if you don't need the date time config why is the TZ and Max_days_hence are there in your config?

Re: DATETIME_CONFIG = NONE doesn't work

hongduan — Fri, 14 Mar 2014 19:11:38 GMT

Does that matter, thought splunk will ignore those fields. Let me try remove those fields see if it works.

Re: DATETIME_CONFIG = NONE doesn't work

starcher — Sun, 16 Mar 2014 14:45:06 GMT

Are you running a single instance of splunk? Or distributed with dedicated indexers? Timestamp manipulations are an index time thing so it needs to be on the indexers.

Re: DATETIME_CONFIG = NONE doesn't work

hongduan — Mon, 17 Mar 2014 13:55:55 GMT

distributed with decicated indexers

Re: DATETIME_CONFIG = NONE doesn't work

linu1988 — Mon, 17 Mar 2014 14:20:57 GMT

is it stored in the indexers in your distributed environment?

Re: DATETIME_CONFIG = NONE doesn't work

starcher — Mon, 17 Mar 2014 14:33:20 GMT

You will need to put the props stanza on each of the indexers.

Re: DATETIME_CONFIG = NONE doesn't work

hongduan — Thu, 20 Mar 2014 14:43:28 GMT

it is not on the indexers. it is on the sourcetype. Why cannot I have different configurations for each sourcetype in an indexer?

Re: DATETIME_CONFIG = NONE doesn't work

hongduan — Thu, 20 Mar 2014 14:53:14 GMT

when test this configuration in splunk data preview, it works very well. all records are using the modification time on the file
but after I add my file into splunk using that sourcetype, splunk tries to parse the timestamp in the log record again.

Re: DATETIME_CONFIG = NONE doesn't work

bnorthway — Wed, 15 Jul 2015 15:58:36 GMT

Unfortunately, the docs say:

"NONE" will leave the event time set to whatever time was selected by the input layer
For data sent by splunk forwarders over the splunk protocol, the input layer will be the time that was selected on the forwarder by its input behavior (as below).

My solution was to use a batch input:

[batch://D:\data*.csv]
sourcetype = foo
move_policy = sinkhole

and the sourcetype stanza in props.conf has DATETIME_CONFIG = NONE.

NOTE batch input will DELETE files as it imports them! So make a copy first!