Go to the command line and change to the $SPLUNK_HOME/bin directory.
Use splunk show kvstore-port to see what port has been assigned to the KVstore.
Use splunk set kvstore-port 8191 to set the port to 8191, then restart Splunk.
After Splunk has been restarted, it should be fine. You can use the show command above to check on it.