Getting Data In
Highlighted

handling future dates from csv

Communicator

hi,
I have a csv file, having time field 'Start Time'. This field will have entries of current week dates as well as future week dates. Eg: If today is 6/13/2017, It will have 6/16/2017, 6/23/2017 etc. When I try to upload the csv it shows warning 'Could not use regex to parse timestamp '6/23/2017'. And when start searching on the csv, it shows 'NO result found'.
I have to have future dates in the csv.
Can somebody help to overcome this problem?

alt text

0 Karma
Highlighted

Re: handling future dates from csv

Esteemed Legend

The root cause of this problem is that the limit MAX_DAYS_HENCE (which is 2 by default) is deliberately disallowing your events.
It is very easy to increase MAX_DAYS_HENCE in props.conf to tolerate your amazing events from the future.

To double-check if this is the case, you should also be seeing warning logs like this:

10-12-2010 12:53:32.619 WARN DateParserVerbose - The TIME_FORMAT specified is matching timestamps (Tue Oct 12 12:53:32 2010) outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE.

You can run this search to see:

index="internal" sourcetype="splunkd" DateParserVerbose MAX_DAYS* "outside of the acceptable time window"
| rex "matching timestamps ((?<ts>[^)]+))"
| eval ts2=strptime(ts, "%a %b %d %T %Y")
| eval time_diff=round((_time-ts2)/60,1)
| where abs(time_diff)<=1440
| table _time, ts, time_diff, host

You will have to restart all splunk instances on your Indexers after you change this setting.

View solution in original post

0 Karma
Highlighted

Re: handling future dates from csv

Communicator

I changed in file C:\Program Files\Splunk\etc\system\default\props.config, MAXDAYSHENCE=30. But it is not reflecting.

0 Karma
Highlighted

Re: handling future dates from csv

Esteemed Legend

Did you deploy to your INDEXERS and restart all splunk instances there? This will only apply to events you index AFTER the change (older broken events will "stay missing").

0 Karma
Highlighted

Re: handling future dates from csv

Communicator

'Did you deploy to your INDEXERS' can you pls tell me how to do that.

0 Karma
Highlighted

Re: handling future dates from csv

Communicator

i am uploading csv and using it without specify any index

0 Karma
Highlighted

Re: handling future dates from csv

Esteemed Legend

You should change that back, create a C:\Program Files\Splunk
etc\system\local\props.conf
file and put in it this:

[default]
MAX_DAYS_HENCE=300

Then, assuming that you are using a splunk all-in-one instance on your personal computer, go to Settings -> Server controls -> Restart.

0 Karma
Highlighted

Re: handling future dates from csv

Communicator

thanks Woodcock..its working!

0 Karma