Splunk Search

How to collapse variable part of url to get a list of urls?

andrewbeak
Path Finder

Hi,

I'm trying to get a list of urls that users are visiting for each of the customer sites that we manage.

I have a lookup table that links hosts to a particular customer site.

I have gotten as far as:

| rex field=uri_path mode=sed "s/([a-z0-9_\.-]+)@([\da-z\.-]+)\.([a-z\.]{2,6})/email/g"
| lookup host_grouping Host AS host OUTPUTNEW Customer, Environment 
| dedup Customer uri_path 
| fields _time Customer Environment uri_path

The lookup works just fine, but a lot of my url's have a variable portion that is easily described with regex. If I try to tabulate the output of this search I end up with thousands of entries.

For example, the values 28400 and 212 are id's of the channel and stream respectively.

 /api/channel/28400/stream/212/play

Instead of listing every combination of this endpoint that has been reached I want to count it just once, for example by matching those integers with regex.

There will be different url formats, but the pieces I need to regex out can be assumed to be either integers or email addresses.

I think what I'm looking for is Rex (http://docs.splunk.com/Documentation/SplunkCloud/7.0.0/SearchReference/Rex) but it doesn't seem to accept PCRE.

Any clues where I can find this in the documentation? Or give an example of "flattening" fields that have predictable variable content?

0 Karma
1 Solution

andrewbeak
Path Finder

This query actually works - but the search results don't show the edited field. If you expand the result to show the fields then you'll see that they have been changed.

View solution in original post

0 Karma

andrewbeak
Path Finder

This query actually works - but the search results don't show the edited field. If you expand the result to show the fields then you'll see that they have been changed.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...