https://community.splunk.com/t5/Splunk-Search/How-to-collapse-variable-part-of-url-to-get-a-list-of-urls/m-p/324119#M96740 <P>Hi,</P> <P>I'm trying to get a list of urls that users are visiting for each of the customer sites that we manage. </P> <P>I have a lookup table that links hosts to a particular customer site.</P> <P>I have gotten as far as:</P> <PRE><CODE>| rex field=uri_path mode=sed "s/([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})/email/g" | lookup host_grouping Host AS host OUTPUTNEW Customer, Environment | dedup Customer uri_path | fields _time Customer Environment uri_path </CODE></PRE> <P>The lookup works just fine, but a lot of my url's have a variable portion that is easily described with regex. If I try to tabulate the output of this search I end up with thousands of entries.</P> <P>For example, the values 28400 and 212 are id's of the channel and stream respectively.</P> <PRE><CODE> /api/channel/28400/stream/212/play </CODE></PRE> <P>Instead of listing every combination of this endpoint that has been reached I want to count it just once, for example by matching those integers with regex.</P> <P>There will be different url formats, but the pieces I need to regex out can be assumed to be either integers or email addresses.</P> <P>I think what I'm looking for is Rex (<A href="http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/SearchReference/Rex">http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/SearchReference/Rex</A>) but it doesn't seem to accept PCRE. </P> <P>Any clues where I can find this in the documentation? Or give an example of "flattening" fields that have predictable variable content?</P> Mon, 09 Apr 2018 15:26:34 GMT andrewbeak 2018-04-09T15:26:34Z https://community.splunk.com/t5/Splunk-Search/How-to-collapse-variable-part-of-url-to-get-a-list-of-urls/m-p/324119#M96740 <P>Hi,</P> <P>I'm trying to get a list of urls that users are visiting for each of the customer sites that we manage. </P> <P>I have a lookup table that links hosts to a particular customer site.</P> <P>I have gotten as far as:</P> <PRE><CODE>| rex field=uri_path mode=sed "s/([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})/email/g" | lookup host_grouping Host AS host OUTPUTNEW Customer, Environment | dedup Customer uri_path | fields _time Customer Environment uri_path </CODE></PRE> <P>The lookup works just fine, but a lot of my url's have a variable portion that is easily described with regex. If I try to tabulate the output of this search I end up with thousands of entries.</P> <P>For example, the values 28400 and 212 are id's of the channel and stream respectively.</P> <PRE><CODE> /api/channel/28400/stream/212/play </CODE></PRE> <P>Instead of listing every combination of this endpoint that has been reached I want to count it just once, for example by matching those integers with regex.</P> <P>There will be different url formats, but the pieces I need to regex out can be assumed to be either integers or email addresses.</P> <P>I think what I'm looking for is Rex (<A href="http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/SearchReference/Rex">http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/SearchReference/Rex</A>) but it doesn't seem to accept PCRE. </P> <P>Any clues where I can find this in the documentation? Or give an example of "flattening" fields that have predictable variable content?</P> Mon, 09 Apr 2018 15:26:34 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collapse-variable-part-of-url-to-get-a-list-of-urls/m-p/324119#M96740 andrewbeak 2018-04-09T15:26:34Z https://community.splunk.com/t5/Splunk-Search/How-to-collapse-variable-part-of-url-to-get-a-list-of-urls/m-p/324120#M96741 <P>This query actually works - but the search results don't show the edited field. If you expand the result to show the fields then you'll see that they have been changed.</P> Mon, 09 Apr 2018 15:49:11 GMT https://community.splunk.com/t5/Splunk-Search/How-to-collapse-variable-part-of-url-to-get-a-list-of-urls/m-p/324120#M96741 andrewbeak 2018-04-09T15:49:11Z