topic Re: slow splunk seaches in Splunk Search

slow splunk seaches

gauravmsharma — Tue, 06 Oct 2020 10:27:33 GMT

A simple search(index="xx" source="/aa/bb/cc.log") made on my searchead takes 4 minutes to display 7.5 millon events for past 4 hours. This seems to be a very slow performance. My architecture contains 2 peer nodes and a master plus searchead which are dedicated machines.

More complex searches with regex takes enormous time. Where do i start troubleshooting this slowness.

Does inceasing IOPS for hot db (/var/opt/splunk/db) on my peer nodes, will have a postive effect on my perfomance or any other things to check on this.

Re: slow splunk seaches

inventsekar — Tue, 06 Oct 2020 10:35:40 GMT

Hi @gauravmsharma

iops improvement is good.. also, generally improving search speed is a complex task, requires lot of analysis...

https://conf.splunk.com/files/2017/slides/speed-up-your-searches.pdf

https://docs.splunk.com/Documentation/Splunk/8.0.6/Search/Writebettersearches

https://docs.splunk.com/Documentation/Splunk/8.0.6/Search/Quicktipsforoptimization

the summary indexing, data model acceleration ideas will improve search performance good.

(PS - i have given around 500+ karma points so far, received badge for that, if an answer helped you, a karma point would be nice!. we all should start "Learn, Give Back, Have Fun")

Re: slow splunk seaches

leonard_dupray — Tue, 06 Oct 2020 11:10:52 GMT

How many diff apps do you have installed on your search head?

Re: slow splunk seaches

gauravmsharma — Tue, 06 Oct 2020 11:29:16 GMT

8 apps