I am ingesting events from log files. There are 50 log files, each with 10,000 lines a day, and they get rolled daily with retention of 10 days. The file formats are identical, so there is only 1 source type. So I have 500 files in total of which 50 are changing at any time, and maybe 5,000,000 total events in Splunk.
My question relates to best practice for indexing for query performance. I don't believe that there are good reasons in my use case for go in any particular direction due to access control or retention.
At the moment I just have 1 index for everything. But I could create a new index each day across all log files, including the date in the index name. Alternatively I could have a separate index for each log file. Or both.
I would like to hear about what would be best practice in terms of theory and your practical experience, please.
... View more