topic Re: How to allow shell wildcard in scripted input's argument? in Getting Data In

How to allow shell wildcard in scripted input's argument?

yuanliu — Wed, 19 Apr 2017 18:15:58 GMT

For example, if I put this in inputs.conf

[script:/bin/ls /*/lib /var/lib /usr/lib ]
sourcetype = ls

The latter two will produce desired output, but the first one generates errors like

ERROR ExecProcessor - message from /bin/ls "/\/lib" ls : cannot access `//lib': No such file or directory

Basically splunkd refuses to honor the basic shell wildcard; it even internally puts a backslash to protect the argument, apparently together with adding double quotes which in tells a shell to interpret special characters. This is very strange because in "script", I expect the behavior to be the same as if I am running in a script.

Re: How to allow shell wildcard in scripted input's argument?

MuS — Fri, 16 Jun 2017 02:30:31 GMT

Hi yuanliu,

Maybe I get something wrong here, but this is not the way you should use this kind of input. I see multiple problems here:

The input is missing // so it should be [script:///bin/ls]
The /bin directory is outside of the default allowed script paths $SPLUNK_HOME/etc/system/bin/, $SPLUNK_HOME/etc/apps/$YOUR_APP/bin/ or $SPLUNK_HOME/bin/scripts/
You're listing multiple possible scripts in one single monitor, even though /var/lib and /usr/lib are not really scripts

So, for example to get /bin/ls working in a custom app, you need an inputs stanza like this:

[script://./bin/myScript.path]
sourcetype = ls

and in $SPLUNK_HOME/etc/apps/$YOUR_APP/bin/ a file called myScript.path. This file then contains this entry:

/bin/ls

Only call one executable per script:// stanza and test one by one.

Hope this helps to get you started ...

cheers, MuS

Re: How to allow shell wildcard in scripted input's argument?

MuS — Fri, 16 Jun 2017 02:31:40 GMT

Ah and I forget to answer the initial question: wildcards are not allowed in the path, the path must be be an absolute path.

Re: How to allow shell wildcard in scripted input's argument?

yuanliu — Fri, 16 Jun 2017 04:00:50 GMT

The input is missing // so it should be [script:///bin/ls]

The example is composed, not the real one. The real script is relative in the same app path.

You're listing multiple possible scripts in one single monitor, even though /var/lib and /usr/lib are not really scripts

What follow the script name are arguments to the very script just like in a shell, not scripts. Again, the example is composed; I'm not really interested in /var/lib and /usr/lib.

Is it documented anywhere that wildcard would not be allowed in script arguments? What is the reason for this? I would argue that it is a bug not to expand shell wildcards when the syntax is fully shell compliant.

Re: How to allow shell wildcard in scripted input's argument?

MuS — Fri, 16 Jun 2017 04:04:13 GMT

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf
search for the script:// stanza, it says

The path to <cmd> can be an absolute path, make use of an environment
  variable such as $SPLUNK_HOME, or use the special pattern of an initial '.'
  as the first directory to indicate a location inside the current app.

It does not mention wildcards in these two options.

Re: How to allow shell wildcard in scripted input's argument?

MuS — Fri, 16 Jun 2017 04:17:16 GMT

And if you want to provided arguments to a command, use the filename.path example which then called the final command with your command arguments.

Re: How to allow shell wildcard in scripted input's argument?

yuanliu — Fri, 16 Jun 2017 17:49:20 GMT

Not mentioning and forbidding are two different propositions. Splunk's [monitor] stanza accepts wildcards in path name, what is a reason why [script] must not?

Re: How to allow shell wildcard in scripted input's argument?

MuS — Fri, 16 Jun 2017 21:16:23 GMT

The script stanza has been limited for security reasons to the default directories $SPLUNK_HOME/etc/system/bin/, $SPLUNK_HOME/etc/apps/$YOUR_APP/bin/ and $SPLUNK_HOME/bin/scripts/. This was done years ago to prevent a scripted input to run executables outside of Splunk.
If you still want to run executables outside of Splunk you can use the filename.path file approach, but I can tell you right now that in this file wildcards are also not support 😉

If you want to get an answer why it is not mentioned in the docs, just submit a feedback on the docs page of inputs.conf with you question and the docs team will take care of it.

cheers, MuS