Building for the Splunk Platform

Splunk Docker: committing container as new image

breid1313
Path Finder

Hi all,

I'm leveraging the splunk/splunk:8.0.4.1 image for some development. At this time, I've pulled the splunk/splunk image, and configured it according to my needs. I now want to commit that container to save it as a new image, so that I have a baseline container to fall back on and start. However, my newly saved image fails to start when I perform a `docker run`. I've checked the original container for the `cacert.pem` file in question and it does exist. Since the new image I'm attempting to start is essentially a clone of the running container, I'm surprised that it fails to start.

Is this a use case that can be achieved with the splunk/splunk image? Thank you.

```
TASK [splunk_common : Start Splunk via CLI] ************************************
fatal: [localhost]: FAILED! => {
"changed": false,
"cmd": [
"/opt/splunk/bin/splunk",
"start",
"--accept-license",
"--answer-yes",
"--no-prompt"
],
"delta": "0:00:01.506092",
"end": "2020-07-15 19:59:26.957025",
"rc": 1,
"start": "2020-07-15 19:59:25.450933"
}

STDOUT:


Splunk> Take the sh out of IT.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Creating: /opt/splunk/var/lib/splunk
Creating: /opt/splunk/var/run/splunk
Creating: /opt/splunk/var/run/splunk/appserver/i18n
Creating: /opt/splunk/var/run/splunk/appserver/modules/static/css
Creating: /opt/splunk/var/run/splunk/upload
Creating: /opt/splunk/var/run/splunk/search_telemetry
Creating: /opt/splunk/var/spool/splunk
Creating: /opt/splunk/var/spool/dirmoncache
Creating: /opt/splunk/var/lib/splunk/authDb
Creating: /opt/splunk/var/lib/splunk/hashDb


STDERR:

The CA file specified (/opt/splunk/etc/auth/cacert.pem) does not exist. Cannot continue.
SSL certificate generation failed.


MSG:

non-zero return code

PLAY RECAP *********************************************************************
localhost : ok=43 changed=6 unreachable=0 failed=1 skipped=44 rescued=0 ignored=0

```
Labels (1)
0 Karma
1 Solution

breid1313
Path Finder
0 Karma

tuckcodes
Engager

Having the same issue. This makes it very difficult to leverage modern tools like Gitlab, where having a base image is important. Furthermore, "-e SPLUNK_APPS_URL=<url>" does not seam to work either in the docker run command. These are very common requirement for modern deployment via CI/CD, regardless of the platform..

  • There is no way to build an image and automate the installation of apps via '-e' in the docker run command.
  • There is also no way to build the vanilla splunk/spunk:latest image, install the apps manually via localhost, and then commit the container state to a new image. 

It appears as though deploying Splunk via containers is not yet a viable option. Please let me know if I am missing something, as deploying Splunk via containers (rather than a VM) is my preferred path.

0 Karma

breid1313
Path Finder

@tuckcodes have a look at this. Hope it helps!

https://github.com/breid1313/splunk-docker

0 Karma

breid1313
Path Finder

@tuckcodes I have something hacky working that I can generalize and push to github if you're interested. It's a docker-compose setup that creates a new slave server off a master license server we have running remotely. It's pretty convenient for quick testing or to more closely replicate someone else's environment.

0 Karma

breid1313
Path Finder

I have also tried to copy /opt/splunk/etc/ and /opt/splunk/var/ to my local file system and mount them into a new container with the below command:

 

docker run -it \
-p 8000:8000 \
-e "SPLUNK_START_ARGS=--accept-license" \
-e "SPLUNK_PASSWORD=securePassw0rd" \
-v /local/path/etc/:/opt/splunk/etc \
-v /local/path/var/:/opt/splunk/var \
splunk/splunk

 

 

but ansible still fails to provision the new server

 

 

TASK [splunk_common : Start Splunk via CLI] *********************************************************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {
"changed": false,
"cmd": [
"/opt/splunk/bin/splunk",
"start",
"--accept-license",
"--answer-yes",
"--no-prompt"
],
"delta": "0:00:16.962175",
"end": "2020-07-15 22:28:05.187999",
"rc": 10,
"start": "2020-07-15 22:27:48.225824"
}

STDOUT:


Splunk> Take the sh out of IT.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...


STDERR:

homePath='/opt/splunk/var/lib/splunk/audit/db' of index=_audit on unusable filesystem.
Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve the issue(s) above after consulting documentation, please file a case online at http://www.splunk.com/page/submit_issue


MSG:

non-zero return code

 

 

Get Updates on the Splunk Community!

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...