Please who can give me a help !!
I'm not able to start splunk.
bash-4.1$ /opt/splunk/bin/splunk start Splunk> Be an IT superhero. Go home early. Checking prerequisites... Checking http port : open Checking mgmt port : open Checking appserver port [127.0.0.1:8065]: open Checking kvstore port : open Checking configuration... Done. Checking critical directories... Done Checking indexes... Validated: _audit _internal _introspection _thefishbucket access_summary access_summary2 algosec audit_summary audit_summary2 bcoat bro cim_summary cisco_acs cisco_ise cisco_router cisco_wc endpoint_summary endpoint_summary2 firedalerts fireeye guardium history ioc juniper_isg main mcafee_eg mcafee_ips misc network_summary network_summary2 network_summary3 nexthink notable notable_summary oim os os_aix os_hpux os_linux os_sunos os_windows paloalto_pa perfmon proxy_center_summary proxy_center_summary2 qualys risk rsa_ecat rsa_sa session_end session_start summary symantec_dlp symantec_encryption symantec_sep te test threat_activity tpam traffic_center_summary traffic_center_summary2 ueba vasco venafi web_inspect websense whois windows wineventlog wrla xtreme_contexts Done Bypassing local license checks since this instance is configured with a remote license master. Checking filesystem compatibility... Done Checking conf files for problems... Done Checking default conf files for edits... Validating installed files against hashes from '/opt/splunk/splunk-6.3.2-aaff59bb082c-linux-2.6-x86_64-manifest' All installed files intact. Done Checking replication_port port : open All preliminary checks passed. Starting splunk server daemon (splunkd)... Done [ OK ] Waiting for web server at http://127.0.0.1:8000 to be available.splunkd 3600 was not running. Stopping splunk helpers... [ OK ] Done. Stopped helpers. Removing stale pid file... done. WARNING: web interface does not seem to be available!
Indexer seemed to work fine, no web interface no 8000 listening port.
Our issue was these files were corrupted likely by cisco AMP.
-rw-------. 1 splunk splunk 5165 Nov 20 12:55 times.pyo
-rw-------. 1 splunk splunk 13008 Nov 20 12:55 routes.pyo
-rw-------. 1 splunk splunk 15667 Nov 20 12:55 message.pyo
-rw-------. 1 splunk splunk 8204 Nov 20 12:56 startup.pyo
Copied from other index server and restarted, web interface was available again.
Please look for Splunkd.log.
1. bucket replication or bucket duplication
2. Indexes execution issue (if cluster enviornment)
1. check carefully entire (try to find for "error", you may get duplicate buckets.
2. In case of indexes, you may not have your master cluster speaking, or bucket under other indexes may be creating this issue.
I just had this issue with bucket conflict. Look for the following in splunkd.log:
Error IndexerService - Error initilizing IndexerService: idx=* bucket=rb_* Detected directory manually copied into its database, causing id conflics ...
even mine were replicated buckets and i deleted them and tried restarting but at last i am getting the error.
10-22-2016 14:27:19.243 -0700 FATAL IndexerService - One or more indexes could not be initialized. Cannot disable indexes on a clustering slave.
Several reasons that can cause this issue. without looking at splunkd.log, it's hard to tell
i can list few possibilities from my experience
if it's an indexer,
1. see if there are any bucket clashes....if there are two buckets with same id, example db*120 in db and db*120 in colddb
if it's a search head,
1. if you pushed an encrypted password file from deployer, shc member fail to parse the file as it doesn't know what the encrypted password is.
Again, hard to tell without seeing the splunkd.log.
Hope this helps!