topic Re: How to run rex commands from CLI mode in Splunk Search

How to run rex commands from CLI mode

vkakani60 — Sat, 10 Sep 2016 04:11:28 GMT

I want to run Splunk query from the cmd prompt.

It works just fine with basic error search, but when I tried with rex extraction it doesn't recognize the pipe | and rex commands.

splunk search "index=apps sourcetype=weblogs | rex field=_raw  "Source: (?<Source>.*)" |  dedup Source | outputtext usexml=false "  user admin password changeme

Re: How to run rex commands from CLI mode

MuS — Sat, 10 Sep 2016 05:08:41 GMT

Hi vkakani60,

You need to escape the " of the regex

 splunk search "index=apps sourcetype=weblogs | rex field=_raw  \"Source: (?<Source>.*)\" |  dedup Source | outputtext usexml=false "  user admin password changeme

Hope this helps ...

Cheers, MuS

Re: How to run rex commands from CLI mode

vkakani60 — Sat, 10 Sep 2016 16:38:21 GMT

It doesn't work for me, I tried with single quote ' and it works.

Re: How to run rex commands from CLI mode

vkakani60 — Sat, 10 Sep 2016 16:40:53 GMT

It works with single quote ', for \" it throws me an error There is no such file.

splunk search "index=apps sourcetype=weblogs | rex field=_raw  'Source: (?<Source>.*)' |  dedup Source | outputtext usexml=false "  user admin password changeme

Re: How to run rex commands from CLI mode

dwaddle — Sat, 10 Sep 2016 17:58:12 GMT

When using Splunk's CLI you have to remember that you are running under the context of the Unix Shell first and foremost. Things you try to do will be parsed by the shell, possibly resulting in unexpected side effects, well before the Splunk CLI sees it. Many special characters in SPL are also special characters to the shell.

Drawing an example. Let's take a sample Splunk search, ran from the search bar:

sourcetype=access_combined host = server*

This returns all events from my hosts named server* where the sourcetype is access_combined. Now, let's run this exact same search, copied and pasted, using the CLI. For the sake of argument, let's assume my shell is in a directory that has files named server1.txt server2.txt and server3.txt. So I run:

splunk search sourcetype=access_combined host = server*

Surprise! I don't get the same results that I got running the search via the search bar. Why? Because the Shell interpreted the wildcard in server* for me. What got actually passed to Splunk was:
splunk search sourcetype=access_combined host = server1.txt server2.txt server3.txt

Not at ALL what we expected! You can see this effect with Bash and most other bash-like shells by doing a set -x on the command line to enable extra output from the shell itself for debugging purposes.

I'm not going to try to explain all of the different shell quoting and escaping rules, because they are rather complex and there are certainly better references - ask Google. But, at a high level the pipe character |, the back-tick character (which I cannot show here because it's also special to Answers' markup, the asterisk *, single quotes, ', double quotes, ", and the dollar sign $ are all special to both the shell and Splunk in some certain way. And this is not an exhaustive list.

Trying to run an arbitrarily complex searches from the CLI will require you to know the arcane Shell quoting rules. There is no easy way to avoid it.

Re: How to run rex commands from CLI mode

MuS — Sat, 10 Sep 2016 20:03:21 GMT

Another fantastic answer by duckfez!