topic Messed up input data ruining/slowing down search? in Splunk Search

Messed up input data ruining/slowing down search?

monicato — Fri, 13 Jul 2012 23:29:14 GMT

Hi!

I accidentally indexed really bizarre logs (partially downloaded logs) and assigned it to a sourcetype. Now searches in this sourcetype is extremely slow and I get inconsistent and bizarre results. I then deleted them from my input directory but it seems like they still exist. Is there anyway to clear or erase the history?

I used the | delete command and it looks like it's gone, but my search is still messed up and extremely slow (it takes a long time to output results)... Can anyone tell me what's going on?

please help ;__;

Re: Messed up input data ruining/slowing down search?

MHibbin — Sat, 14 Jul 2012 11:05:29 GMT

Not sure "what's going on?", but for you question in cleaning your "history", I assume you mean events/eventdata, this can be done by using the CLI command "$SPLUNK_HOME/bin/splunk clean". Before using this, you should read the documentation, as you will lose ALL data in the indexes you choose to clean..

http://docs.splunk.com/Documentation/Splunk/latest/Admin/RemovedatafromSplunk

Once you have cleaned a specific index (e.g. main), you may wish to re-index certain files that were previously monitored, as Splunk maintains a record of what it has indexed through CRC checks,etc, you may also need to look in to cleaning the fishbucket... BUT BE WARNED, this could result in duplicated data in other indexes.

Hope this helps,

MHIbbin

Re: Messed up input data ruining/slowing down search?

monicato — Mon, 16 Jul 2012 18:35:42 GMT

Thanks for the answer! I identified the source of the weird log and piped it to the delete command and that did the trick! No need to clear the index. Thanks for the help!

Re: Messed up input data ruining/slowing down search?

wilsonchua — Fri, 15 Jan 2016 08:37:54 GMT

I did a 'clean' operation but the issue is still the same.
Running a search query for the "last 15 minutes" takes more than 25 minutes to complete.
Any ideas would be appreciated.