Splunk Dev

Running script to parse data and output csv

kooixiuhong
New Member

Hi everyone, I am fairly new to Splunk and I have question about scripts. Currently I have a folder with some custom log files that is monitored by Splunk. I have a Python scripts that uses the Splunk SDK and use the rex command to extract fields from the logs and then outputcsv for every log in the folder. The csv files are also monitored by Splunk. I want the script to be ran at specific time each day/week. I am aware Splunk allows scripts scheduling. Does that work for scripts like this or is it better to schedule the script to run on the OS as oppose to on Splunk?

Thanks

0 Karma
1 Solution

renjith_nair
SplunkTrust
SplunkTrust

Hi @kooixiuhong,

You could do it as a scripted input to put all pieces together in splunk itself as mentioned in https://docs.splunk.com/Documentation/Splunk/7.1.1/AdvancedDev/ScriptedInputsIntro

Detailed example : https://docs.splunk.com/Documentation/Splunk/7.1.1/AdvancedDev/ScriptSetup

Happy Splunking!

View solution in original post

0 Karma

woodcock
Esteemed Legend

If you do this as a scripted input, you can still keep the | outputcsv foo as the last line and it will both create the csv and index the results.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Hi @kooixiuhong,

You could do it as a scripted input to put all pieces together in splunk itself as mentioned in https://docs.splunk.com/Documentation/Splunk/7.1.1/AdvancedDev/ScriptedInputsIntro

Detailed example : https://docs.splunk.com/Documentation/Splunk/7.1.1/AdvancedDev/ScriptSetup

Happy Splunking!
0 Karma

kooixiuhong
New Member

HI, from all the documentation I read it's seems most scripts using Splunk's scripted inputs provide some output that would be indexed by Splunk. However my scripts just run a search query on log files and then use outputcsv to create csv files. For example if there is 5 logs in the monitor folder, there is a for loop that run the query for each file and outputs 5 csv.

My question is does a script like that qualifies as scripted input? Sorry if it is confusing.

0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Hi @kooixiuhong , you could either output as csv stream and directly post to splunk or could do as you do now to create a file and post it. Below almost matches with your requirement

https://docs.splunk.com/Documentation/Splunk/7.1.1/AdvancedDev/ScriptedInputsIntro#Use_cases_for_scr...

https://docs.splunk.com/Documentation/Splunk/7.1.1/AdvancedDev/ScriptSetup

Happy Splunking!
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...