I have a Windows 2008 R2 Server. We are importing 2,160,617 Files in 86 Folders.
The indexing process starts off fast and now has slowed down to a crawl. Is there any practical way that this can be speeded up.
I must stress that I am evaluating Splunk, so any solution will need to be spoon fed.
I did read one possible solution on Splunkbase which talked about clearing out data from the Learned App. I do not know whether this is a safe course of action and when I make the Learned App visible and search for my *.aud files, there are massive returns, but no visible method of removing them from the App.
Performance troubleshooting can be complicated and requires a good knowledge of Splunk.
As Initial analysis, these problems with Splunk can be divided in three categories:
1, The Indexes are based in disks with slow write performance
2, The source files are based in disks with slow read performance (in particular network devices)
3, The OS has limited resources. A common example is the number of open files it can handle at a certain time, which might be smaller of the number of files Splunk is required to monitor, causing the OS to wait until it ca reuse the file descriptors.
4, Splunk hasn't been instructed how to extract the timestamp from those specific events.
You might want to review the System Requirement documentation page: