Archive

DATETIME_CONFIG = NONE doesn't work

Explorer

I have this stanza in my props.conf
[testtest]
TZ = US/Eastern
SHOULD
LINEMERGE = false
MAXDAYSHENCE = 5
TRUNCATE = 0
DATETIME_CONFIG = NONE

And this setting in inputs.conf
[monitor:///testfolder/test.txt]
disabled = 0
index = test
index
sourcetype = test_test
crcSalt =

why splunk is still trying to parse the timestamp in each line of test.txt instead of using the modified time of test.txt?

Tags (1)

Path Finder

Unfortunately, the docs say:

"NONE" will leave the event time set to whatever time was selected by the input layer
For data sent by splunk forwarders over the splunk protocol, the input layer will be the time that was selected on the forwarder by its input behavior (as below).

My solution was to use a batch input:

[batch://D:\data*.csv]
sourcetype = foo
move_policy = sinkhole

and the sourcetype stanza in props.conf has DATETIME_CONFIG = NONE.

NOTE batch input will DELETE files as it imports them! So make a copy first!

0 Karma

SplunkTrust
SplunkTrust

Are you running a single instance of splunk? Or distributed with dedicated indexers? Timestamp manipulations are an index time thing so it needs to be on the indexers.

0 Karma

Explorer

when test this configuration in splunk data preview, it works very well. all records are using the modification time on the file
but after I add my file into splunk using that sourcetype, splunk tries to parse the timestamp in the log record again.

0 Karma

Explorer

it is not on the indexers. it is on the sourcetype. Why cannot I have different configurations for each sourcetype in an indexer?

0 Karma

SplunkTrust
SplunkTrust

You will need to put the props stanza on each of the indexers.

0 Karma

Champion

is it stored in the indexers in your distributed environment?

0 Karma

Explorer

distributed with decicated indexers

0 Karma

Explorer

Does that matter, thought splunk will ignore those fields. Let me try remove those fields see if it works.

0 Karma

Champion

if you don't need the date time config why is the TZ and Maxdayshence are there in your config?

0 Karma

Explorer

I am sure the props.conf file is placed in the correct location. cause other changes I made to props.conf are picked up by splunk, only this DATATIME_CONFIG = NONE not work.

0 Karma

Champion

where is you props.conf file placed?

0 Karma