Splunk is unable to start

[rubeniturrieta]

Hi to eveeryone:

I have this problem when i try to start splunk. Here's the error message:

./splunk start

Splunk> Take the sh out of IT.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]:
open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
Validated: _audit _blocksignature _internal _introspection _thefishbucket access_summary access_summary2 audit_summary audit_summary2 bro cim_summary ciscokcc endpoint_summary endpoint_summary2 firedalerts history main netflow network_summary network_summary2 network_summary3 notable notable_summary os proxy_center_summary proxy_center_summary2 risk session_end session_start summary traffic_center_summary traffic_center_summary2 whois
Done
Checking filesystem compatibility... Done
Checking conf files for problems...
Invalid key in stanza [samplelog_css.cisco-wsa-squid] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 2: sourcetype (value: cisco:wsa:squid)
Invalid key in stanza [samplelog_css.cisco-wsa-squid] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 7: hourOfDayRate (value: { "0":0.1, "1":0.1, "2":0.1, "3":0.1, "4":0.1, "5":0.25, "6":0.35, "7":0.45, "8":0.65, "9":0.8, "10":1.0, "11":1.0, "12":1.0, "13":1.0, "14":1.0, "15":1.0, "16":1.0, "17":0.9, "18":0.8, "19":0.7, "20":0.6, "21":0.4, "22":0.2, "23":0.1 })
Invalid key in stanza [samplelog_css.cisco-wsa-squid] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 8: dayOfWeekRate (value: { "0":0.5, "1":1.0, "2":1.0, "3":1.0, "4":1.0, "5":1.0, "6":0.75 })
Invalid key in stanza [samplelog_css.cisco-wsa-squid] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 9: randomizeCount (value: 0.2)
Invalid key in stanza [samplelog_css.cisco-wsa-squid] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 10: randomizeEvents (value: true)
Invalid key in stanza [samplelog_css.cisco-wsa-squid] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 11: sampletype (value: csv)
Invalid key in stanza [samples_css.search] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 36: hourOfDayRate (value: { "0":0.1, "1":0.1, "2":0.1, "3":0.1, "4":0.1, "5":0.25, "6":0.35, "7":0.45, "8":0.65, "9":0.8, "10":1.0, "11":1.0, "12":1.0, "13":1.0, "14":1.0, "15":1.0, "16":1.0, "17":0.9, "18":0.8, "19":0.7, "20":0.6, "21":0.4, "22":0.2, "23":0.1 })
Invalid key in stanza [samples_css.search] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 37: dayOfWeekRate (value: { "0":0.5, "1":1.0, "2":1.0, "3":1.0, "4":1.0, "5":1.0, "6":0.75 })
Invalid key in stanza [samples_css.search] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 38: randomizeCount (value: 0.2)
Invalid key in stanza [samples_css.search] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 39: randomizeEvents (value: true)
Invalid key in stanza [samples_css.search] in /opt/splunk/etc/apps/Splunk_CiscoSecuritySuite/default/eventgen.conf, line 40: sampletype (value: csv)
Invalid key in stanza [CIM-Alerts] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 6: outputMode (value: spool)
Invalid key in stanza [CIM-Application_State] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 56: outputMode (value: spool)
Invalid key in stanza [CIM-Authentication] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 126: outputMode (value: spool)
Invalid key in stanza [CIM-Authentication] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 128: randomizeEvents (value: True)
Invalid key in stanza [CIM-Inventory] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 156: outputMode (value: spool)
Invalid key in stanza [CIM-Inventory] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 158: randomizeEvents (value: True)
Invalid key in stanza [CIM-Database] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 277: outputMode (value: spool)
Invalid key in stanza [CIM-Database] in /opt/splunk/etc/apps/Splunk_SA_CIM/default/eventgen.conf, line 279: randomizeEvents (value: True)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 4: recursive (value: False)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 6: store_dir (value: $SPLUNK_HOME/var/spool/splunk)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 8: bro_bin (value: /opt/bro/bin/bro)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 9: bro_opts (value: -C)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 10: bro_script (value: None)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 11: bro_seeds (value: None)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 12: bro_merge (value: False)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 15: content_maxsize (value: 1024)
Invalid key in stanza [pcap_monitor] in /opt/splunk/etc/apps/Splunk_TA_bro/default/inputs.conf, line 18: run_maxtime (value: 1800)
Invalid key in stanza [samplelog.cisco.asa] in /opt/splunk/etc/apps/Splunk_TA_cisco-asa/default/eventgen.conf, line 6: sourcetype (value: cisco:asa)
Invalid key in stanza [samplelog.cisco.fwsm] in /opt/splunk/etc/apps/Splunk_TA_cisco-asa/default/eventgen.conf, line 76: sourcetype (value: cisco:fwsm)
Invalid key in stanza [samplelog.cisco.pix] in /opt/splunk/etc/apps/Splunk_TA_cisco-asa/default/eventgen.conf, line 131: sourcetype (value: cisco:pix)
Invalid key in stanza [syslog.ciscowsa.access] in /opt/splunk/etc/apps/Splunk_TA_cisco-wsa/default/eventgen.conf, line 2: sourcetype (value: cisco:wsa:squid)
Invalid key in stanza [syslog.ciscowsa.access] in /opt/splunk/etc/apps/Splunk_TA_cisco-wsa/default/eventgen.conf, line 7: maxIntervalsBeforeFlush (value: 1)
Invalid key in stanza [samplelog.ciscowsa.access] in /opt/splunk/etc/apps/Splunk_TA_cisco-wsa/default/eventgen.conf, line 42: sourcetype (value: cisco:wsa:squid)
Invalid key in stanza [samplelog.ciscowsa.l4tm] in /opt/splunk/etc/apps/Splunk_TA_cisco-wsa/default/eventgen.conf, line 79: sourcetype (value: cisco:wsa:l4tm)
Invalid key in stanza [sample.v4.mcafee_epo] in /opt/splunk/etc/apps/Splunk_TA_mcafee/default/eventgen.conf, line 9: source (value: mcafee_v4.sample)
Invalid key in stanza [sample.v4.mcafee_epo] in /opt/splunk/etc/apps/Splunk_TA_mcafee/default/eventgen.conf, line 10: sourcetype (value: mcafee:epo)
Invalid key in stanza [sample.v5.mcafee_epo] in /opt/splunk/etc/apps/Splunk_TA_mcafee/default/eventgen.conf, line 40: source (value: mcafee_v5.sample)
Invalid key in stanza [sample.v5.mcafee_epo] in /opt/splunk/etc/apps/Splunk_TA_mcafee/default/eventgen.conf, line 41: sourcetype (value: mcafee:epo)
Invalid key in stanza [sample.mcafee_ids] in /opt/splunk/etc/apps/Splunk_TA_mcafee/default/eventgen.conf, line 80: source (value: mcafee_ids.sample)
Invalid key in stanza [sample.mcafee_ids] in /opt/splunk/etc/apps/Splunk_TA_mcafee/default/eventgen.conf, line 81: sourcetype (value: mcafee:ids)
Value in stanza [app=/network/ntp:default] in /opt/splunk/etc/apps/Splunk_TA_nix/default/tags.conf, line 783 not URI encoded: app = /network/ntp:default
Value in stanza [shell=/bin/bash] in /opt/splunk/etc/apps/Splunk_TA_nix/default/tags.conf, line 835 not URI encoded: shell = /bin/bash
Value in stanza [shell=/bin/sh] in /opt/splunk/etc/apps/Splunk_TA_nix/default/tags.conf, line 838 not URI encoded: shell = /bin/sh
Value in stanza [shell=/usr/bin/bash] in /opt/splunk/etc/apps/Splunk_TA_nix/default/tags.conf, line 841 not URI encoded: shell = /usr/bin/bash
Value in stanza [shell=/usr/bin/pfksh] in /opt/splunk/etc/apps/Splunk_TA_nix/default/tags.conf, line 844 not URI encoded: shell = /usr/bin/pfksh
Value in stanza [shell=/usr/bin/pfsh] in /opt/splunk/etc/apps/Splunk_TA_nix/default/tags.conf, line 847 not URI encoded: shell = /usr/bin/pfsh
Value in stanza [Service_Name=kadmin/changepw] in /opt/splunk/etc/apps/Splunk_TA_windows/default/tags.conf, line 121 not URI encoded: Service_Name = kadmin/changepw
Value in stanza [app=win:local] in /opt/splunk/etc/apps/Splunk_TA_windows/default/tags.conf, line 184 not URI encoded: app = win:local
Value in stanza [app=win:remote] in /opt/splunk/etc/apps/Splunk_TA_windows/default/tags.conf, line 187 not URI encoded: app = win:remote
Value in stanza [signature=Credit Card Number detected in Clear Text] in /opt/splunk/etc/apps/TA-snort/default/tags.conf, line 8 not URI encoded: signature = Credit Card Number detected in Clear Text
Value in stanza [signature=SENSITIVE-DATA Credit Card Numbers] in /opt/splunk/etc/apps/TA-snort/default/tags.conf, line 13 not URI encoded: signature = SENSITIVE-DATA Credit Card Numbers
Your indexes and inputs configurations are not internally consistent. For more information, run 'splunk btool check --debug'
Done
All preliminary checks passed.

Starting splunk server daemon (splunkd)...
Done

Waiting for web server at https://127.0.0.1:8000 to be available..

WARNING: web interface does not seem to be available!

Please help me with this error. Any help will be very appreciated.

Regards

[rubeniturrieta · Accepted solution]

I solved it. I deleted the /opt/splunk/var/lib/splunk/defaultdb/thaweddb directory, and then splunk started without problem. Thanks to stepahnefotso anyways.

[stephanefotso]

Are you the only user on your machine? If not, check if another user did not use the 8000 port on your machine.
You can also think on changing your splunk-web port default value by reading here:
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Changedefaultvalues#Change_network_ports

[rubeniturrieta]

I'm the only user on my machine. I have changed the port to 9000 how you suggested, but i have the same error messages

[stephanefotso]

Did you change splunkd default port also?

[rubeniturrieta]

Yes, i changed splunkd default por also