How to edit the time field in the index?

[klim]

I want to create a 30 day index of data that changes it's indexed timestamp as each day passes. Therefore the data will always show up when I do a last 30 day search and don't need to pick out the specific 30 days I saved. Ie if I started with January data, in June 1st the original data from January should represent the month of May.

Is there any way to change the time of the data in the index every day? Or does it have to be deleted from index and readded? 

[somesoni2]

What type of data is this? How is being ingested and how frequently it updates? How much data (size/number of events) are we talking about here? If it's small data, you might be able to achieve this via lookups (csv OR kvstore). As @gcusello said, indexed data can't be modified. You can re-ingest after updating the date it but it doesn't seem to be normal process.

[gcusello]

Hi @klim,

no, when a log is indexed isn't possible to modify it.

If you have wrong dates, you have to check your ingestion process, probably there's an error in the TimeStamp parsing.

Ciao.

Giuseppe

[klim]

@gcusello  thanks for the reply. 

There isn't a problem with the ingestion process. I just wanted to be able to have an index contain the same 30 days of data but the timestamp always changes. ie March 1st data would become March 2nd and so on and would always be the most recent data. 

[gcusello]

Hi @klim,

sorry but I don't understand your need: 

you ingested data and the timestamp is correct (it's the same of the one in the log),

What do you mean that you want an index that contains the same 30 days?, in an index you usually put data, also etherogeneous, it isn't relevant data, in other words, it isn't a good idea to have an index for each month.

Mayber you mean that you have the same data in many days and you want always the last.

If this is your need, you could use a stats command using the "last" option.

Could you better describe your data and share some sample?

Ciao.

Giuseppe

[PickleRick]

I suppose OP might want a "convenience feature" - for example to use a staticaly built subsearch or something like that without the need to select a time range.

@klimIt doesn't work like that. Once the data is indexed, it can't be modified. You can "delete" it (in reality - just mark it as unavailable for searching; it will be physically deleted along with the whole bucket it resides in when the buckets rolls out to frozen unless you have frozen storage set up) but that's it. Once it got into the index, it's immutable. You can of course change the search-time extractions, calculated fields and so on but indexed fields and raw event data are immutable.

You could do a scheduled search that would search for old data, modify the timestamp and collect it back into an index but that's ugly. And it would count against your license since the data would be indexed anew as if it came straight from the sources.

[klim]

The main idea was to always have the same data in the past 30 days for demo purposes. I was thinking of doing the scheduled report and saving it to an index.

"it would count against your license since the data would be indexed anew as if it came straight from the sources."

What determines if it counts against the license or not? I thought if the events have a source type of stash it won't count.

[PickleRick]

Yes, if you collect it with a default sourcetype of stash, they won't count against the license.

But you'll have problems with parsing them like if they had the original sourcetype.

[somesoni2]

The output of a schedule report (or ad-hoc report) saved in a summary index (https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/Usesummaryindexing) will not be counted against the license (sourcetype=stash). 

You can also look at EventGen to dynamically generate event data for demos.