How do I use regex to extract URL parameter field names

[DPWSplunkPOC]

I want to extract the field names from a URL's parameters. For example my raw event might look like this:

action=accept host=myserver timestamp=01/01/2016:12:00:00 src_ip=1.1.1.1 domain=mydomain process=GET url=mywebpage.com/requestedpage.resquest?field1=value1&field2=value2&field3=value3

I would like the regex to capture all field names from the parameters into one capture group called url_parameter.

I have the following regex:

^[^\?\n]\*\?(?P<\url_parameter\>\w+)*(?:[^&\n]\*&(\w+)) 

*Note I added slashes around url_parameter because it looked like an HTML tag.

It captures the first field after the question mark and places it in url_parameter capture group. It captures the the second field in a different capture group. Finally it does not capture field 3 or any number of remaining fields in the parameter. I'm far from a regex expert but I'm trying to teach myself. Any help is appreciated. Thank you in advance.

[javiergn]

Hi,

Try this:

your search here
| rex max_match=0 "(?msi)(?<url_parameter>field\d)"

It'll create a multvalue field with all your field names that you can later on used the way you like.

[somesoni2 · Accepted solution]

Try something like this

your base search | rex  "^[^\?\n]*\?(?P<url_parameter>\S+)"  | rex max_match=0 field=url_parameter "(?<url_parameter>\w+)="

[DPWSplunkPOC]

That works! How could I make so that the user could search url_parameters without typing the rex command? Can this be added to props.conf or transforms.conf?

In the meantime you have given me a great head start!

[somesoni2]

See this Splunk documentation page
http://docs.splunk.com/Documentation/Splunk/6.0.3/Knowledge/Createandmaintainsearch-timefieldextract...

Props.conf

[yoursourcetype]
REPORT-urlparams = geturlparams

Transforms.conf

[geturlparams]
REGEX = (?<url_parameter>\w+)=
MV_ADD = true