Sequential file processing

[pramodkumar]

Hi Team,
Have an scenario, lets say in first 12 hours i get some 1000 sales files and i index then and in second 12 hours i get around 2000 files and do same. When i fetch the second 2000 files will it again fetches the previous 1000 files or not.
And at the end of the day i want to see sales amount for first 1000 files and carry the value to the second 2000 files (lets say first 1000 sales $1000 and second 2000 files $3000 i want total of ($1000+$3000) ) for each and every file the sales amount should keep on move.. is this possible if so how.. please help me. My sales values has to float (increase) file by file as it process.. how to do..

[kristian_kolb · Accepted solution]

Normally splunk will not index the same file twice. So if you monitor a directory where the files get written/copied, splunk will usually 'know' whether it has already indexed a particular file.

Once the events are indexed, you can then do searching/reporting on the indexed information.

I do not really understand your requirement regarding how you want to report on the different file sets, or what the files contain. Please provide some sample data, and an idea of how you want the results presented.

You could also have a look at the following search commands, which may be helpful to you;

• accum
• sort
• streamstats

Hope this helps,

Kristian

[pramodkumar]

ur answer really helped me a lot..

[Ayn]

Splunk will not reindex files that it has already indexed. More information on how it deals with recognizing files that have already been indexed is available here: http://docs.splunk.com/Documentation/Splunk/latest/Data/Howlogfilerotationishandled