Getting Data In

Setting timestamp to minus one month of ingestion

Builder

I am getting some csv files in start of each month but actually they are the billing data for the last month. I want to set the timestamp to last month not the month it is being ingested in. Any ideas how this can be done?

PS: there is no field in the files that I can set as timestamp neither I want to change the files.

0 Karma

Esteemed Legend

Given your constraints, it is not possible; you will have to pre-process your file with other software to modify it such that one of the other answers that will not work as-is, will work when-then.

0 Karma

SplunkTrust
SplunkTrust

In props.conf:

[sourcetypeName]
DATETIME_CONFIG=NONE

This will work assuming the modified date of the file is last month.

0 Karma

Builder

No, unfortunately they get the file out of the system the first/second day of the month so the timestamp is current month

0 Karma

Esteemed Legend

You can set the timestamp based on the filename so arrange to have the filenames as you like and do this:

http://answers.splunk.com/answers/40247/timestamp-from-file-name.html
http://answers.splunk.com/answers/94763/set-timestamp-based-on-file-source-path.html

Be sure to sent MAX_DAYS_PAST appropriately!

0 Karma

SplunkTrust
SplunkTrust

@woodcock - what would be the proper stanzas to use SOURCE_KEY = _indextime to recalculate the _time? Like, how would you do the equivalent of this in an index-time transform?

_time=relative_time(_indextime,"-1mon@mon")

If you can't do anything so "programmatic" in a stanza, is there any place that you could get a SOURCE_KEY value that gave the first day (or last day) of the preceding month, in order to use it to override _time?

0 Karma

Builder

No, unfortunately they get the file out of the system the first/second day of the month so the timestamp is current month

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!