Hey Community,
I'm trying to pass a variable including the pattern to a rex command mode=sed.
This is my approach but it doesn't work. My screen just give me a message: Search is waiting for input.
Whats wrong with my code?
| eval rex_langing_page=if("$landing_page$"=="*", "s/^(\/[^\/]+\/).*$/\1/", ".*") //This should be the pattern for my rex command
| rex field=Web.uri mode=sed "$rex_landing_page$"
This is my whole code:
| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview "Web.http_session_pageviews">1 GROUPBY "Web.site","Web.http_session",_time span=1s,"Web.http_referer","Web.uri","Web.http_session_channel"
| eval rex_langing_page=if("$landing_page$"=="*", "s/^(\/[^\/]+\/).*$/\1/", ".*")
| rex field=Web.uri mode=sed "$rex_landing_page$"
| rename Web.uri AS to "Web.http_referer" AS from "Web.http_session_channel" AS channel "Web.site" AS site "Web.http_session" AS http_session
| rex field=from "https?://.+?(?<from_path>/[^\\?]+)"
| rename from_path AS from
| eval from=if(isnull(from),"$site$",from)
| streamstats global=f count AS interaction by http_session
| eval interaction=interaction-1
| where interaction<=round($limit$/20,0)
| stats list(from) AS from, list(to) AS to,earliest(to) AS landing_page, list(channel) as channel, list(interaction) as interaction by http_session
| search to="$to$" landing_page="$landing_page$"
| eval fields = mvzip(from,mvzip(to,mvzip(interaction,channel)))
| fields http_session fields
| mvexpand fields
| rex field=fields "(?<from>[^\,]+),(?<to>[^\,]+),(?<interaction>[^\,]+),(?<channel>.+)"
| eval from=if(interaction=0, "", from)
| eval from=if("$landing_page$"!="*", "$site$"."$landing_page$", from)
| table http_session from to interaction
| where interaction>=1
| where to!="$landing_page$"
| where to!=from
| stats count by from,to,interaction
| sort interaction,-count,from,to
| streamstats global=f count AS interaction_rank by interaction
| where interaction_rank<=round($limit$/10,0)
| stats sum(count) AS count by from,to
| sort from to count
| head $limit$
Thanks a lot
Johannes
@JohannesGmelin, you can try the following on <change>
event to your dropdown as shown below. <eval>
will set the $rex_landing_page$ token based on dropdown selection, which you can use in your query:
<input type="dropdown" token="landing_page" searchWhenChanged="true">
<label>Landing page</label>
<choice value="*">Any</choice>
<search>
<query>| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview GROUPBY "Web.site","Web.http_session","Web.uri"
| stats last("Web.uri") AS landing_page by "Web.http_session"
| rex field=landing_page mode=sed "s/^(\/[^\/]+\/).*$/\1/"
| top limit=20 landing_page
| eval landing_page_description=landing_page+" ("+count+")"
| fields landing_page landing_page_description</query>
<earliest>0</earliest>
<latest>now</latest>
</search>
<fieldForLabel>landing_page_description</fieldForLabel>
<fieldForValue>landing_page</fieldForValue>
<default>*</default>
<change>
<eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", ".*")</eval>
</change>
</input>
PS: In your query 3rd line you are having a typo with variable name as rex_langing_page. Even if you correct this type you can use it as token in subsequent query (you might have to check out documentation on map command in Splunk if you want to set the token within a query being run.)
Updated answer all forward slashes need to be escaped in eval tag: s/^(\\/[^\\/]+\\/).*$/\\1/
@JohannesGmelin, you can try the following on <change>
event to your dropdown as shown below. <eval>
will set the $rex_landing_page$ token based on dropdown selection, which you can use in your query:
<input type="dropdown" token="landing_page" searchWhenChanged="true">
<label>Landing page</label>
<choice value="*">Any</choice>
<search>
<query>| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview GROUPBY "Web.site","Web.http_session","Web.uri"
| stats last("Web.uri") AS landing_page by "Web.http_session"
| rex field=landing_page mode=sed "s/^(\/[^\/]+\/).*$/\1/"
| top limit=20 landing_page
| eval landing_page_description=landing_page+" ("+count+")"
| fields landing_page landing_page_description</query>
<earliest>0</earliest>
<latest>now</latest>
</search>
<fieldForLabel>landing_page_description</fieldForLabel>
<fieldForValue>landing_page</fieldForValue>
<default>*</default>
<change>
<eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", ".*")</eval>
</change>
</input>
PS: In your query 3rd line you are having a typo with variable name as rex_langing_page. Even if you correct this type you can use it as token in subsequent query (you might have to check out documentation on map command in Splunk if you want to set the token within a query being run.)
Updated answer all forward slashes need to be escaped in eval tag: s/^(\\/[^\\/]+\\/).*$/\\1/
I'm getting an error: Error in 'rex' command: Regex: missing terminating ] for character class
Can you print $rex_landing_page$ in your dashboard to check whether the rex is getting escaped?
<row>
<panel>
<html>
rex_landing_page: $rex_landing_page$
</html>
</panel>
</row>
Also I hope you have already validated regular expressions by hard-coding in your query.
This is the result:
rex_landing_page: s/^([^]+).*$//
@JohannesGmelin, slashes need to be escaped in eval. I have updated my comment. Please try with the following eval condition:
<eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", ".*")</eval>
Oh great it's working. But the second command "replace with nothing (don't change something)" don't work too:
Error in 'rex' command: Failed to initialize sed. cannot find sed command: .
When you pass static values to your rex field does it work?
| rex field=Web.uri mode=sed ".*"
As per your question this is the other value that the eval expression will set
<eval token="rex_landing_page">if($value$=="*", "s/^(\\/[^\\/]+\\/).*$/\\1/", "s/(.*)$/\\1/")</eval>
This is the solution. Thanks for your help!
Bye
Johannes
Great that it worked 🙂
@JohannesGmelin, hope this worked for you. Let me convert this to answer so that you can accept and mark your question as answered.
Let me convert this to comment, until this is resolved. Specially for others to provide their solutions/workaround.
Which version of Splunk are you using? Also where are you setting the $landing_page$ token? Would it be possible for you to use eval instead of set to set the token for regular expression directly?
The value $landing_page" is coming from a dropdown filter. I need this because I want to set different patterns to my rex according to the value in my dropdown filter.
<input type="dropdown" token="landing_page" searchWhenChanged="true">
<label>Landing page</label>
<choice value="*">Any</choice>
<search>
<query>| tstats summariesonly=t count FROM datamodel=Web WHERE Web.site="$site$" "Web.eventtype"=pageview GROUPBY "Web.site","Web.http_session","Web.uri"
| stats last("Web.uri") AS landing_page by "Web.http_session"
| rex field=landing_page mode=sed "s/^(\/[^\/]+\/).*$/\1/"
| top limit=20 landing_page
| eval landing_page_description=landing_page+" ("+count+")"
| fields landing_page landing_page_description</query>
<earliest>0</earliest>
<latest>now</latest>
</search>
<fieldForLabel>landing_page_description</fieldForLabel>
<fieldForValue>landing_page</fieldForValue>
<default>*</default>
</input>