Getting Data In

Splunk forwarder preventing Docker rebuild


I am wondering if anyone has come accross this issue before:

System and application versions:
• Docker version 18.09.4
• Splunk version 7.2.6 (?)
• Windows Server 2019 1809 Build

A summary of what we’ve discovered and background information:
• Splunk seems to prevent docker from starting docker containers, they are stuck in a “Created” state
• We do not use Splunk explicitly as our docker logging service, i.e. Splunk is not referenced in any docker config
• Docker and the SplunkForwarder service both start up on host boot
• Changing the dependencies on the service (i.e. docker start first or splunk start first) doesn’t fix the issue
• Stopping splunk while docker is running and then creating the containers works
o As soon as one container has started successfully, we can start splunk and still create more containers
• Restarting splunk while docker is running and then creating the containers does not work

Steps to reproduce on a machine:
1. Boot server up, docker and splunk start automatically
2. Attempt to run docker-compose to create our containers with no containers already running or in an exited state, docker gets stuck with containers in a “Created” state

Steps to mitigate issue:
1. When there are no containers running, stop the splunk service
2. Run docker-compose to create at least one container successfully
3. Start the splunk service
4. Run docker-compose to bring up any remaining containers

Any help or ideas to get a work around would be appreciated

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.