Installation

After upgrading my search head and indexer to Splunk 6.3.1, why am I seeing a decrease in performance and increase in CPU and memory usage?

rmsit
Communicator

Hello, all.

I upgraded my single search head and indexer to version 6.3.1. I am aware of the CPU resource improvements with this release, however, I've noticed a slight decrease in performance (UI responsiveness) and increase CPU and Memory usage overall. Should I just assume this is the new norm and add resources to hopefully improve system response? I am running Splunk on virtual machines and have applied the best practices.

Thanks,
James

Labels (2)
0 Karma

risgupta_splunk
Splunk Employee
Splunk Employee

This message means your search processes are taking >1s to read initial configuration information from disk. What does the I/O subsystem underneath $SPLUNK_HOME/etc look like in your environment? If $SPLUNK_HOME/etc is networked storage, for example, there might be disk/network performance issues affecting search startup time.

0 Karma

rmsit
Communicator

I'm also receiving the alert below after upgrading:

Configuration initialization for Drive:\Program Files\Splunk\etc took longer than expected (1289ms) when dispatching a search (search ID: xxx__xxx__search__search5_1447719591.31937); this typically reflects underlying storage performance issues

Is this really a disk I/O issue?

0 Karma

stevepraz
Path Finder

What OS are you running on? I have seem similar issues and log messages on my Windows search head after upgrading to 6.3 and 6.3.1.

0 Karma

rmsit
Communicator

I'm running on Windows Server 2008 R2 x64 Enterprise SP1.

0 Karma

woodcock
Esteemed Legend

The majority of the benefits are on Indexers; did you upgrade them or just the Search Head?

0 Karma

rmsit
Communicator

Yes, I upgraded the search head first, then the indexer.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.