How to truncate a field value after a regex pattern?

[mbolostk]

How can I truncate a field value after a given pattern. For example, if I am looking at web page logs, how can I truncate everything after .html so that no parameters or variables are reported in my web page count?

[mbolostk · Accepted solution]

I figured it out. Nevermind. It was based on the first post - but had to redo the regex as the uri didn't, in this case, start with http://

[cpetterborg]

If you already have the field extracted, then you can use eval or rex to create a new field to extract the first part of the URL with something like (using eval):

eval mainpart=replace(origurl,"(.*)[?].*","\1")

Where origurl is the already extracted URL field, and ? is the ? in the URL for separating the Parameters from the rest of the URL. That will enable you to have more than .html at the end of the URL (like jpeg, js css, etc). The REX would be like the example already given by aljohnson_splunk. If your logs don't include the http:// (as many apache log files do), then your rex would need to allow for finding the URL differently from his example.

[aljohnson_splun]

Things that will help us help you:

1. Post sample data
2. Post sample search
3. Post desired output

It sorta sounds like you want to use the rex command.
E.g.

| rex field=url_field "http://(?<url_path>.+html)"
| stats count by url_path

[jrodman]

For this particular goal, I would usually make the .+ be ungreedy with .+?

e.g.

| rex field=url_field "http://(?<url_path>.+?html)"