Large number of regular expressions in search field

[tamnor]

Hi

I am a new Splunk user and at the moment I am using it to monitor the performance of the web applications of the insurance company that I work for. I want to be able to generate a report that shows the average response time against each web request as well as the number of instances of that request that were submitted for a given time period. A large number of the web requests have dynamic values in them for customer id, policy number, claim id, repair booking etc. I am using regular expressions to filter out the unique info and group the request together. So at the moment I have a search that looks like the following ..

index=arena_repairs_prod responseTime>5000 lodge | rex field=uri_path mode=sed "s/repair\/summary.html\?repairId=[0-9]+/repair\/summary.html?repairId=/" | rex field=uri_path mode=sed ....

So I just keep repeating the regular expressions one after the other in the search field. I can see that my search is going to become quite large.

Is this the best way to create a search like this, or is there someone of saving each of the regular expressions and just referring to them by a variable name?

Any advice would be much appreciated.

[tamnor]

Hi. Thanks for the responses. This is no longer a problem as the indexing was changed so that the uri_path is an extracted field so I can group by this rather than the full url with all the dynamic values. There are still some dynamic values in the uri_path but not that many so my queries aren't too long.

[GKC]

you might also find this website as a useful tool for regex

http://gskinner.com/RegExr/

[martin_mueller]

There is a way to save regular expressions to have fields extracted automatically each time you search, without cluttering your query. You can use the interactive field extractor, or create extractions/transformations yourself.

IFX how-to: http://www.splunk.com/view/SP-CAAADUY
Field extraction in general: http://docs.splunk.com/Documentation/Splunk/5.0/Knowledge/Aboutfields

[Ayn · Accepted solution]

It would be better not to mess with the raw data but instead extract the endpoints you're interested in as fields and then group by that. So instead of

 ... | rex field=uri_path mode=sed "s/repair\/summary.html?repairId=[0-9]+/repair\/summary.html?repairId=/"

Couldn't you just check all requests to summary.html?

... | rex field=uri_path "^(?<requestedPage>[^?]+)"