All Apps and Add-ons

Parallel development in Splunk on the same app - use GIT for management and source control??

Path Finder

Hi all,

Just wondering if anyone has had any experience using GIT as a tool to manage Splunk development work across multiple branches?

e.g. if I have two DEV environments and one master environment..

  1. Clone DEV1/DEV2 from master.
  2. Develop searches/reports/eventtypes on DEV1
  3. Develop searches/report/eventttypes on DEV2
  4. Merge DEV1 and DEV2 onto Master.

Anyone done this successfully? any issues encountered? Just wondering how GIT would handle multiple changes to the same configuration file.. it seems to be that we would always get conflicts that would have to be manually resolved? but I am no expert in GIT so would appreciate any help.

Alternatively if someone else has experience with merging parallel streams of work in Splunk together that would be good too.


Tags (2)


You will need to use git hooks. I use Stash by Atlassian where I have implemented pull requests to merge to master, reject commits directly to maste. If there is a merge conflict on a pull request users must resolve those conflict before pull request can be approved. I recommend user pull from master before creating a pull request.

Check out my pervious post:


I'll answer this with an example for changes to a single file. Let's use server.conf just to show that this could be any file at all, not specific to searches. Here's our sample file:



Dev1 clones the repo and wants to change the timeout to 8h. So he creates a branch so that the changes can be tracked separately from master. Edit is made and now our file reads:


Before Dev1 checks his changes in, Dev2 comes along and wants to enable SSL. So, they clone master, make a branch, and now the file might look like this:


enableSplunkdSSL = true
useClientSSLCompression = true

Dev1 commits their change, and pushes it up to the central repo, so now the new branch is visible to all. At this point, maybe you'd have someone else review the change, or do a quick smoke test on a lab server. Or, if you are already down that path, you'd do a full suite of automated tests. Whatever. Then when you are happy that things are not broken, you merge into master and then on your production server(s), you sync them with master, and restart as needed.

Dev2 comes along and is like, "somebody test my changes, I just enabled SSL". They commit the changes locally, then push them up to the server so that others can see and help test. Tests all looks good. When it's time for Dev2 to merge into master (but prior to syncing production to the new-new changes), this is what happens:

Git (automatically, because this is a core feature of revision control features and git in particular) will do a line-by-line merge of "changeset dev1" and "changeset dev2". This is a pretty straightforward case, and the end result will be exactly as you predict, like this:


enableSplunkdSSL = true
useClientSSLCompression = true

If git cannot figure out the changes, you can then take a variety of actions. The onus is on the one merging to figure it out in the end, and they will be shown all the changes that have been made by multiple contributors.

There is a lot to read on this subject, which can make things confusing for folks new to source control systems. To simplify things, you could omit the branch step, and have everyone work in master, but this can make testing and merging more difficult. One of my favorite tips is to automate the "master sync" in production (have cron do a git pull && splunk restart during your maintenance window). This might sound scary at first--until you realize that you REALLY don't push to master until everything has already been thoroughly tested.

So, in order to take advantage of this automated deployment step, at least use two branches. On a small team, you could get away with a "test" and "master" branch. One or more folks work in test, and then the person who gets the short straw merges test into master.

To go back around to your initial question, file changes are compared line-by-line. Git will usually figure things out as if by magic. Many editors will allow you to show diffs visually for those times when you have to merge things manually.

Get Updates on the Splunk Community!

Infographic provides the TL;DR for the 2023 Splunk Career Impact Report

We’ve been shouting it from the rooftops! The findings from the 2023 Splunk Career Impact Report showing that ...

Splunk Lantern | Getting Started with Edge Processor, Machine Learning Toolkit ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...