topic Re: How to add scripts as data input? in Getting Data In

How to add scripts as data input?

tamduong16 — Thu, 22 Mar 2018 21:17:35 GMT

This is my first time dealing with script input. I tried to place my python script in both of these directories (C:\Program Files\Splunk\bin\scripts, C:\Program Files\Splunk\etc\apps\search\bin) and it hasn't work. I run in python idle and the scripts actually index data into my splunk enviroment. But when I want to add it as a script input, I don't get anything back. It looks like python script never been run. Did I have it in the right file? I didn't create a splunk app for this. What can I do inorder for it to be run in splunk? Here is my python script. Thanks so much!

Re: How to add scripts as data input?

niketn — Fri, 23 Mar 2018 06:29:56 GMT

@tamduong16 if your python script is working on its own and you have configured/scheduled the execution of Scripted input which is enabled, I am surprised as to why nothing is being indexed (success or exception log). However, I would request you to push some checkpoint information from your script to Splunk's _internal index using sys.stderror.write()

sys.stderr.write("Python script is starting up\n")

Also try to search in internal index for any other metrics captured for your scripted input when you have setup the same.

Re: How to add scripts as data input?

logloganathan — Fri, 23 Mar 2018 06:41:03 GMT

Alternative way is
You can get the output of script and save that in the log file without splunk
Then input the log file generated to splunk
Which is very easy way

Re: How to add scripts as data input?

Elsurion — Fri, 23 Mar 2018 08:58:35 GMT

It seems to me, that you don't have enabled the script itself. Have you configured an inputs.conf in the local folder of your search App?
And have you an entry in the inputs.conf like this? (modified for windows support):

splunk@monsterine ✓  09:53 $ cat inputs.conf
[script://.\bin\cfgpull_srv.py]
disabled = false
index = main
interval = 60
source = cfgpull_srv.py
sourcetype = cfgpull_srv_stdout

Re: How to add scripts as data input?

logloganathan — Fri, 23 Mar 2018 09:12:58 GMT

i have mentioned alternative as you are facing issue.

Re: How to add scripts as data input?

Elsurion — Fri, 23 Mar 2018 09:26:08 GMT

what if he likes it the splunk way?

Re: How to add scripts as data input?

tamduong16 — Sat, 24 Mar 2018 03:44:36 GMT

@Elsurion I think that's what I'm missing. Since I don't know anything about this. All I did was placing the file inside of the search/bin and expecting it to run on it own. How can I do this? What do I have to do in my input.conf file? I'm very new to this. Thanks for the help

Re: How to add scripts as data input?

tamduong16 — Sat, 24 Mar 2018 03:47:36 GMT

@niketnilay I'm a little lost. What do you mean by configuring the execution of Scripted input. I thought all I have to do is place the file into search/bin folder and it should run. What could I do to configure this? Thanks!

Re: How to add scripts as data input?

niketn — Sat, 24 Mar 2018 04:32:26 GMT

@tamduong16, yes you need to configure Data Input similar to how you would setup File Monitor, Performance Monitors etc. Splunk would need to know what to read, from where to read and how frequently to read, where to index and setting up source/sourcetype etc. These you would need to configure in inputs.conf either through Splunk Web or CLI. Refer to the documentation: http://docs.splunk.com/Documentation/Splunk/latest/Data/Getdatafromscriptedinputs#Add_a_scripted_input_in_Splunk_Web

Also read about Writing Reliable scripts documentation, as most of the time scripted inputs have a wrapper script as well as maintain your own last indexed data/recovery/parallel execution etc: https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptSetup

Once you have completely tested and made your scripted input robust for your scenario, you may be able to build an Add on using Splunk Add On Builder or move towards creating your Modular Input to Splunk

Re: How to add scripts as data input?

niketn — Sat, 24 Mar 2018 04:51:36 GMT

@logloganathan, setting up a scripted input through web is a matter of few clicks and sets up the inputs.conf appropriately.

What you are calling a easy way is actually roundabout. Since after logging data to a file, file monitor input needs to be configured exactly the same way scripted input needs to be setup. Also script will have to be modified to write to a log file as additional change.

Also scripted input capability to ingest data directly to Splunk. Most of the times it is requirement as well as necessity (for example compliance to not to store unencrypted data as flat file). Imagine DB Connect Input or HTTP Event Collector writing to log file and then ingesting to Splunk.

Re: How to add scripts as data input?

anissabnk — Fri, 07 May 2021 10:57:59 GMT

Hello,

I hope you feel good. I have a little question which deals with "Getting Data in" to Splunk with an url. I understood that I have to write a python scipt for that, but I don't know how to do that, and where I have to include thois script.

Thank you for your answer.

Have a good day