Is anyone using CI/CD to deploy Splunk apps?

[mugurelmargarit]

Hello,

We are trying to achieve one-click deployments with Splunk applications. Our desired workflow is below:

1) we develop the app and push the changes to the develop branch
2) we have a pipeline listening for any changes on the branch and when changes are pushed, deployment is triggered on a WIP environment
3) we pack the Splunk applications for the search head and deployment server in tar.gz archives which are extracted on the search head/deployment server (in etc/apps on the search head and etc/deployment-apps on the deployment server) and rsynced using the --delete flag so that if an app has been removed from git, it will also be removed from the search head/deployment server apps - this process is being executed by Chef
4) we manually reload the deployment server or, in the case of adding a new index, we manually apply the cluster bundle

We are facing the following difficulties:

• we plan on testing the applications before pushing them to live and that entitles using live data - since we're using AWS I've created a snapshot of the live Splunk server, mounted the snapshot on the WIP environment, put the cluster in maintenance mode and symlinked /opt/splunk/var/lib from the snapshot and started the indexers and disabled maintenance mode. This worked just fine however, if there is new data in Splunk, we need to repeat the process. Q: What is the best way of replicating live data on the WIP environment?
• when we deploy applications, we don't want the WIP apps to interact w/ the live environment at all. This means that if we develop an app and we create an outputs.conf or anything referring to the WIP infrastructure, we will use something like wip-domain.com. This defeats our purpose as if we develop an application we will need to maintain an additional Git repo for live so that we reference live-domain.com. Q: How do we overcome the environment naming convention issue so that we only maintain one repository?
• how can we run tests on our apps? is there a framework that can be used?
• we have some apps on the search head that are pretty static and those are maintained through Chef - our developers don't have access to the Chef repo so if they make a change to an existing app, they need to rely on a sysadmin. Q: Is there any reason why we shouldn't deploy applications on the search head using serverclass.conf?

There are just some of the challenges we've faced so far. Is anyone using CI/CD (Continuous Integration / Continuous Deployment) to deploy Splunk applications? We are using GoCD if this helps.

Thanks in advance.

[imgarytan]

Hi Vince

How is Appetite going so far? seems like not many activities going on in recent 2 years.

Any updates, latest status?

Thanks
Gary

[goodsellt]

For your DEV/QA environment, you should try using data cloning at the forwarder level over to those indexers instead of snapshot copying. Then just set the data retention in the DEV/QA environment to a much smaller level. You should be able to have the same apps on the search head and indexer as normal but they would be separate from your prod environment (much more cost effective if you're not using a cluster / multisite cluster as DEV/QA as well).

You can deploy apps on the search head using the deployment server (I use the DS to deploy tech addons and additional configurations to the search heads, however be careful of deploying apps to a prod environment where you allow users to edit dashboards, reports, and saved searches with global permissions, as the deployment server likes to keep the apps' synced).

[vliggio · Accepted solution]

http://conf.splunk.com/sessions/2016-sessions.html#search=open-sourced&

Will be out soon! I'll update this when it's released.

[vliggio]

Appetite has finally been released to open source! https://github.com/Bridgewater/appetite

[laurie_gellatly]

Hi Vince,
Any chance we can get a 'beta' copy to look at ? Keen to get things moving 🙂

  ...Laurie:{)

[vliggio]

Release is days away!

[laurie_gellatly]

Excellent. Thanks

[gjanders]

Any updates yet or approximate dates? Looking forward to the release...

[ppeterson]

Can't wait, looking forward to it!

[warwicks]

Glad to hear that this is still in the pipeline.
Was looking good in Orlando.
Thanks Vince, the hard work is appreciated.

[gjanders]

The name referred to is:
Unified Open-Sourced Splunk Configuration Management System
Recording Slides
I've also been eagerly awaiting it's release! Thanks for the update.

[vliggio]

Right, thanks! The search doesn't seem to carry over even though it's in the URL.

[gjanders]

Yes I noticed ! Not sure why, I've put links into my post to assist with the recording/slides as well 🙂

[mugurelmargarit]

Very interested in this, looking forward to its release!

[ppeterson]

Great news, I've been on the lookout for this since I saw the recording! Thanks