- Find Answers
- :
- Splunk Administration
- :
- Admin Other
- :
- Installation
- :
- splunk not start
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hello
recently my Splunk not start, it happens suddenly,after i notice splunk web not work,login to windows server and see it crash and have auto restart,after that i start splunk but get this :
Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
Failed to determine if running as service user: LookupAccountName: No mapping between account names and security IDs was done.
(skipping validation of index paths because not running as ASADC\Mediterranean)
Validated: _audit _internal _introspection _telemetry _thefishbucket history main msad mssql perfmon summary vmware-esxilog vmware-inv vmware-perf vmware-taskevent vmware-vclog windows wineventlog winevents
Done
Checking filesystem compatibility... Done
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from 'C:\Program Files\Splunk\splunk-7.1.2-a0c72a66db66-windows-64-manifest'
File 'C:\Program Files\Splunk\etc/system/default/indexes.conf' changed.
File 'C:\Program Files\Splunk\etc/system/default/inputs.conf' changed.
File 'C:\Program Files\Splunk\etc/system/default/limits.conf' changed.
Problems were found, please review your files and move customizations to local
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Splunkd: Stopped
what can i do?i chek log file of splunk and fined this :
10-26-2019 08:02:54.889 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.904 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.904 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.904 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.904 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.904 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.920 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.920 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.920 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.920 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.920 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.920 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.935 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.935 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.935 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.935 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.935 +0330 ERROR STMgr - dir='D:\Warm\defaultdb\db\hot_v1_10953' out of memory failure rc=1 warm_rc[-2,8] from st_txn_start
10-26-2019 08:02:54.935 +0330 ERROR StreamGroup - unexpected rc=1 from IndexableValue->index
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADGetFullServerPath: Failed to bind to root 'LDAP://pri02.eng.ad.splunk.com/rootDSE': err='0x8007203a' - 'The server is not operational.'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADGetFullServerPath: Failed to bind to root 'LDAP://pri01.eng.ad.splunk.com/rootDSE': err='0x8007203a' - 'The server is not operational.'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADGetServerPath: Failed to bind to root: err='0x8007203a' - 'The server is not operational.'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::GetDCAttributes: Failed to get AD server path.
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed: (0x80004005)Unspecified error -- attempting to reload server path
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADGetServerPath: Failed to bind to root: err='0x8007203a' - 'The server is not operational.'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::GetDCAttributes: Failed to get AD server path.
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed: (0x80004005)Unspecified error -- attempting to reload server path
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdQuery::OutputStartEvent: Failed to search attributes of root object: err='0xa'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::OutputStartEvent: Failed in OutputStartEvent,
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed again with DCName='Asa-Dc.AsaDc.local': (0x80004005)Unspecified error -- no more retries
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADMonitor::init: Failed to initialize Active Directory usn context.
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADMonitorThread::launchADMonitor: Failed to initialize ADMonitor='admon://SecondTargetDC', targedDC='pri02.eng.ad.splunk.com'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdQuery::OutputStartEvent: Failed to search attributes of root object: err='0xa'
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::OutputStartEvent: Failed in OutputStartEvent,
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - AdEventCollector::InitCollector: LoadContextState failed again with DCName='Asa-Dc.AsaDc.local': (0x80004005)Unspecified error -- no more retries
10-26-2019 08:02:54.967 +0330 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\bin\splunk-admon.exe"" splunk-admon - ADMonitor::init: Failed to initialize Active Directory usn context.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We solve it, just restart Splunk, Splunk web services with local administrator privileges
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We solve it, just restart Splunk, Splunk web services with local administrator privileges
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Great! Now come back and click Accept on your answer to close the question and UpVote any other answers or comments that helped you.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It looks like Splunk is complaining about the Index values defined on your D drive. Is your D drive mounted? Is your D drive full? If so, fix that and it shoud be OK. The long term solution is to stop running your Splunk infrastructure (Indexers) on Windows OS:
https://answers.splunk.com/answers/516059/what-are-the-pain-points-with-deploying-your-splun.html
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I checked Splunk version from Splunk.version and splunk.exe,they show my current version 7.1.2 wich it mean not updated at all
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Several issues that could be related.
Seems like could be a permissions issue of the user permissions Splunk as a service is running as
I would check the password for the splunkd service if not using managed service accounts. The two errors listed below points to this.
Skipping validation of index paths error due to not running as ASADC\Mediterranean
Failed to determine if running as service user: LookupAccountName: No mapping between account names and security IDs was done.
Another possibility looks like could be a GPO error and example of a fix for this is below. Not sure what version of windows server that is running.
http://www.rebeladmin.com/2016/01/how-to-fix-error-no-mapping-between-account-names-and-security-ids...
Another issue that could cause problems is placing changes in default not local directory - will cause issues anytime you update Splunk because changes will be overwritten.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
that error(mediteranian user) probably be there in past couple months
I using Windows Server 2016
does Splunk can automatically update?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you upgrade to v8.0?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
no, I do not upgrade it, can Splunk automatically update???
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
No, it cannot.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What kind of instance is it. If it is an EC2, try stopping and starting the instance. Then go to the terminal and start the splunk.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I cant understand, what is EC2??? I stop all Splunk in task manager and start it again, nothing changed
Enterprise Security Content Update (ESCU) | New Releases
Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?
Index This | What are the 12 Days of Splunk-mas?
