Splunk Search

Auto generate a lookup file from SVN or GIT in Splunk?

mikaelsandquist
Explorer

Is it possible to automatically generate a lookup file from SVN or GIT inside Splunk or should it be done by a cron script from the OS?
I'm thankful and open for all creative suggestions 🙂

0 Karma
1 Solution

barakreeves
Splunk Employee
Splunk Employee

My suggestion is to use Bash (*nix/BSD), Powershell (Windows), Python, or a combination to generate the CSV lookup file. This will give you ultimate control over what goes in the file to make sure that it is formatted in a Splunk-friendly manner.

What I have done in the past is to run a Python script on cron. Then have the script do the following:
- Make a backup of the existing lookup
- Generate a new lookup file using SVN/Git commands; format the output
- Delete the old file
- Restart Splunk

Another thing you may wish to consider is to create a transaction log to show if the new lookup file was successfully created or not; and if not where the errors occurred. If that is something you are interested in, I will be more than happy to post the code in the answer...the log is Splunk-friendly 🙂

Hope this helps.

View solution in original post

0 Karma

barakreeves
Splunk Employee
Splunk Employee

My suggestion is to use Bash (*nix/BSD), Powershell (Windows), Python, or a combination to generate the CSV lookup file. This will give you ultimate control over what goes in the file to make sure that it is formatted in a Splunk-friendly manner.

What I have done in the past is to run a Python script on cron. Then have the script do the following:
- Make a backup of the existing lookup
- Generate a new lookup file using SVN/Git commands; format the output
- Delete the old file
- Restart Splunk

Another thing you may wish to consider is to create a transaction log to show if the new lookup file was successfully created or not; and if not where the errors occurred. If that is something you are interested in, I will be more than happy to post the code in the answer...the log is Splunk-friendly 🙂

Hope this helps.

0 Karma

MuS
SplunkTrust
SplunkTrust

Just a small remark: you don't need to restart splunk to read a new lookup csv. You can reload it by accessing http[s]://yourSplunkServer/debug/refresh if you have the sufficient rights to do so.

0 Karma

mikaelsandquist
Explorer

Thanks for your answer! I think I'll go for a cron scheduled bash script to generate the CSV lookup file once a day 🙂

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...