topic Re: Pass a variable to a rex command in Splunk Search

Pass a variable to a rex command

JohannesGmelin — Tue, 06 Jun 2017 07:46:57 GMT

Hey Community,

I'm trying to pass a variable including the pattern to a rex command mode=sed.
This is my approach but it doesn't work. My screen just give me a message: Search is waiting for input.
Whats wrong with my code?

| eval rex_langing_page=if("$landing_page$"=="*", "s/^(\/[^\/]+\/).*$/\1/", ".*") //This should be the pattern for my rex command
| rex field=Web.uri mode=sed "$rex_landing_page$"

This is my whole code:

| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview "Web.http_session_pageviews">1 GROUPBY "Web.site","Web.http_session",_time span=1s,"Web.http_referer","Web.uri","Web.http_session_channel"
| eval rex_langing_page=if("$landing_page$"=="*", "s/^(\/[^\/]+\/).*$/\1/", ".*")
| rex field=Web.uri mode=sed "$rex_landing_page$"
| rename Web.uri AS to "Web.http_referer" AS from "Web.http_session_channel" AS channel "Web.site" AS site "Web.http_session" AS http_session
| rex field=from "https?://.+?(?<from_path>/[^\\?]+)" 
| rename from_path AS from
| eval from=if(isnull(from),"$site$",from)
| streamstats global=f count AS interaction by http_session 
| eval interaction=interaction-1
| where interaction<=round($limit$/20,0)
| stats list(from) AS from, list(to) AS to,earliest(to) AS landing_page, list(channel) as channel, list(interaction) as interaction by http_session
| search to="$to$" landing_page="$landing_page$"
| eval fields = mvzip(from,mvzip(to,mvzip(interaction,channel))) 
| fields http_session fields
| mvexpand fields
| rex field=fields "(?<from>[^\,]+),(?<to>[^\,]+),(?<interaction>[^\,]+),(?<channel>.+)"
| eval from=if(interaction=0, "", from)
| eval from=if("$landing_page$"!="*", "$site$"."$landing_page$", from)
| table http_session from to interaction
| where interaction>=1
| where to!="$landing_page$"
| where to!=from
| stats count by from,to,interaction
| sort interaction,-count,from,to
| streamstats global=f count AS interaction_rank by interaction
| where interaction_rank<=round($limit$/10,0)
| stats sum(count) AS count by from,to
| sort from to count
| head $limit$

Thanks a lot
Johannes

Re: Pass a variable to a rex command

niketn — Tue, 06 Jun 2017 08:03:29 GMT

Which version of Splunk are you using? Also where are you setting the $landing_page$ token? Would it be possible for you to use eval instead of set to set the token for regular expression directly?

Re: Pass a variable to a rex command

JohannesGmelin — Tue, 06 Jun 2017 08:08:14 GMT

The value $landing_page" is coming from a dropdown filter. I need this because I want to set different patterns to my rex according to the value in my dropdown filter.

<input type="dropdown" token="landing_page" searchWhenChanged="true">
      <label>Landing page</label>
      <choice value="*">Any</choice>
      <search>
        <query>| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview GROUPBY "Web.site","Web.http_session","Web.uri" 
| stats last("Web.uri") AS landing_page by "Web.http_session" 
| rex field=landing_page mode=sed "s/^(\/[^\/]+\/).*$/\1/"
| top limit=20 landing_page 
| eval landing_page_description=landing_page+" ("+count+")" 
| fields landing_page landing_page_description</query>
        <earliest>0</earliest>
        <latest>now</latest>
      </search>
      <fieldForLabel>landing_page_description</fieldForLabel>
      <fieldForValue>landing_page</fieldForValue>
      <default>*</default>
    </input>

Re: Pass a variable to a rex command

niketn — Tue, 29 Sep 2020 14:19:35 GMT

@JohannesGmelin, you can try the following on <change> event to your dropdown as shown below. <eval> will set the $rex_landing_page$ token based on dropdown selection, which you can use in your query:

 <input type="dropdown" token="landing_page" searchWhenChanged="true">
       <label>Landing page</label>
       <choice value="*">Any</choice>
       <search>
         <query>| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview GROUPBY "Web.site","Web.http_session","Web.uri" 
 | stats last("Web.uri") AS landing_page by "Web.http_session" 
 | rex field=landing_page mode=sed "s/^(\/[^\/]+\/).*$/\1/"
 | top limit=20 landing_page 
 | eval landing_page_description=landing_page+" ("+count+")" 
 | fields landing_page landing_page_description</query>
         <earliest>0</earliest>
         <latest>now</latest>
       </search>
       <fieldForLabel>landing_page_description</fieldForLabel>
       <fieldForValue>landing_page</fieldForValue>
       <default>*</default>
       <change>
            <eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", ".*")</eval>
       </change>
     </input>

PS: In your query 3rd line you are having a typo with variable name as rex_langing_page. Even if you correct this type you can use it as token in subsequent query (you might have to check out documentation on map command in Splunk if you want to set the token within a query being run.)

Updated answer all forward slashes need to be escaped in eval tag: s/^(\\/[^\\/]+\\/).*$/\\1/

Re: Pass a variable to a rex command

JohannesGmelin — Tue, 06 Jun 2017 08:57:45 GMT

I'm getting an error: Error in 'rex' command: Regex: missing terminating ] for character class

Re: Pass a variable to a rex command

niketn — Tue, 29 Sep 2020 14:23:05 GMT

Can you print $rex_landing_page$ in your dashboard to check whether the rex is getting escaped?

<row>
  <panel>
    <html>
      rex_landing_page: $rex_landing_page$
    </html>
  </panel>
</row>

Also I hope you have already validated regular expressions by hard-coding in your query.

Re: Pass a variable to a rex command

niketn — Tue, 06 Jun 2017 10:21:51 GMT

Let me convert this to comment, until this is resolved. Specially for others to provide their solutions/workaround.

Re: Pass a variable to a rex command

JohannesGmelin — Tue, 29 Sep 2020 14:23:33 GMT

This is the result:

rex_landing_page: s/^([^]+).*$//

Re: Pass a variable to a rex command

niketn — Wed, 07 Jun 2017 09:07:52 GMT

@JohannesGmelin, slashes need to be escaped in eval. I have updated my comment. Please try with the following eval condition:

<eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", ".*")</eval>

Re: Pass a variable to a rex command

niketn — Wed, 07 Jun 2017 09:09:48 GMT

@JohannesGmelin, hope this worked for you. Let me convert this to answer so that you can accept and mark your question as answered.

Re: Pass a variable to a rex command

JohannesGmelin — Wed, 07 Jun 2017 09:14:21 GMT

Oh great it's working. But the second command "replace with nothing (don't change something)" don't work too:

Error in 'rex' command: Failed to initialize sed. cannot find sed command: .

Re: Pass a variable to a rex command

niketn — Wed, 07 Jun 2017 10:12:42 GMT

When you pass static values to your rex field does it work?

| rex field=Web.uri mode=sed ".*"

As per your question this is the other value that the eval expression will set

Re: Pass a variable to a rex command

JohannesGmelin — Wed, 07 Jun 2017 10:24:34 GMT

<eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", "s/(.*)$/\\1/")</eval>

This is the solution. Thanks for your help!

Bye
Johannes

Re: Pass a variable to a rex command

niketn — Wed, 07 Jun 2017 12:27:57 GMT

Great that it worked 🙂