I am getting some csv files in start of each month but actually they are the billing data for the last month. I want to set the timestamp to last month not the month it is being ingested in. Any ideas how this can be done?
PS: there is no field in the files that I can set as timestamp neither I want to change the files.
Given your constraints, it is not possible; you will have to pre-process your file with other software to modify it such that one of the other answers that will not work as-is, will work when-then.
You can set the timestamp based on the filename so arrange to have the filenames as you like and do this:
Be sure to sent
@woodcock - what would be the proper stanzas to use SOURCEKEY = `indextime` to recalculate the _time? Like, how would you do the equivalent of this in an index-time transform?
If you can't do anything so "programmatic" in a stanza, is there any place that you could get a SOURCE_KEY value that gave the first day (or last day) of the preceding month, in order to use it to override _time?