How to add scripts as data input?

[tamduong16]

This is my first time dealing with script input. I tried to place my python script in both of these directories (C:\Program Files\Splunk\bin\scripts, C:\Program Files\Splunk\etc\apps\search\bin) and it hasn't work. I run in python idle and the scripts actually index data into my splunk enviroment. But when I want to add it as a script input, I don't get anything back. It looks like python script never been run. Did I have it in the right file? I didn't create a splunk app for this. What can I do inorder for it to be run in splunk? Here is my python script. Thanks so much!

[niketn · Accepted solution]

@tamduong16, yes you need to configure Data Input similar to how you would setup File Monitor, Performance Monitors etc. Splunk would need to know what to read, from where to read and how frequently to read, where to index and setting up source/sourcetype etc. These you would need to configure in inputs.conf either through Splunk Web or CLI. Refer to the documentation: http://docs.splunk.com/Documentation/Splunk/latest/Data/Getdatafromscriptedinputs#Add_a_scripted_inp...

Also read about Writing Reliable scripts documentation, as most of the time scripted inputs have a wrapper script as well as maintain your own last indexed data/recovery/parallel execution etc: https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptSetup

Once you have completely tested and made your scripted input robust for your scenario, you may be able to build an Add on using Splunk Add On Builder or move towards creating your Modular Input to Splunk

[anissabnk]

Hello,

I hope you feel good. I have a little question which deals with "Getting Data in" to Splunk with an url.  I understood that I have to write a python scipt for that, but I don't know how to do that, and where I have to include  thois script.

Thank you for your answer.

Have a good day

[Elsurion]

It seems to me, that you don't have enabled the script itself. Have you configured an inputs.conf in the local folder of your search App?
And have you an entry in the inputs.conf like this? (modified for windows support):

splunk@monsterine ✓  09:53 $ cat inputs.conf
[script://.\bin\cfgpull_srv.py]
disabled = false
index = main
interval = 60
source = cfgpull_srv.py
sourcetype = cfgpull_srv_stdout

[logloganathan]

i have mentioned alternative as you are facing issue.

[Elsurion]

what if he likes it the splunk way?

[tamduong16]

@Elsurion I think that's what I'm missing. Since I don't know anything about this. All I did was placing the file inside of the search/bin and expecting it to run on it own. How can I do this? What do I have to do in my input.conf file? I'm very new to this. Thanks for the help

[logloganathan]

Alternative way is
You can get the output of script and save that in the log file without splunk
Then input the log file generated to splunk
Which is very easy way

[niketn]

@logloganathan, setting up a scripted input through web is a matter of few clicks and sets up the inputs.conf appropriately.

What you are calling a easy way is actually roundabout. Since after logging data to a file, file monitor input needs to be configured exactly the same way scripted input needs to be setup. Also script will have to be modified to write to a log file as additional change.

Also scripted input capability to ingest data directly to Splunk. Most of the times it is requirement as well as necessity (for example compliance to not to store unencrypted data as flat file). Imagine DB Connect Input or HTTP Event Collector writing to log file and then ingesting to Splunk.

[niketn]

@tamduong16 if your python script is working on its own and you have configured/scheduled the execution of Scripted input which is enabled, I am surprised as to why nothing is being indexed (success or exception log). However, I would request you to push some checkpoint information from your script to Splunk's _internal index using sys.stderror.write()

sys.stderr.write("Python script is starting up\n")       

Also try to search in internal index for any other metrics captured for your scripted input when you have setup the same.

[tamduong16]

@niketnilay I'm a little lost. What do you mean by configuring the execution of Scripted input. I thought all I have to do is place the file into search/bin folder and it should run. What could I do to configure this? Thanks!