topic Re: Can I specify a regex in a lookup table to group similar requests into the same output lookup? in Splunk Search

Can I specify a regex in a lookup table to group similar requests into the same output lookup?

xvxt006 — Tue, 17 Sep 2013 01:58:47 GMT

Hi,

We would like to create a look up table based on some user agents.

Mozilla/5.0 (compatible; Traverse/0.1; ABC 22175)
Mozilla/5.0 (compatible; Traverse/0.1; ABC 23457)
Mozilla/5.0 (compatible; Capture/0.4; ABC 56439)
Mozilla/5.0 (compatible; Capture/0.2; ABC 98123)

I would like to group similar kind of requests in the look up table and save them into Field XXX.

So field XXX should show
Traverse 2 requests
Catpure 2 requests

So can i specify reg ex in the look up table as there will be multiple patterns which i would like to group them.

Re: Can I specify a regex in a lookup table to group similar requests into the same output lookup?

lguinn2 — Tue, 17 Sep 2013 04:18:08 GMT

There is an app that provides a dynamic lookup for user agent strings; it is called TA-uas_parser. Download it from

http://apps.splunk.com/app/1007

It's free. The user agent string can be very complex. I don't recommend that you build this yourself.

If you really want to do it youself, you can use wildcards (regular expressions) in the input field of a lookup table.
See How to use wildcards in a lookup table for more info.

Re: Can I specify a regex in a lookup table to group similar requests into the same output lookup?

xvxt006 — Tue, 17 Sep 2013 22:27:35 GMT

Thank you. I will try this.

Re: Can I specify a regex in a lookup table to group similar requests into the same output lookup?

xvxt006 — Thu, 19 Sep 2013 01:05:12 GMT

Hi, i tried the match_type = WILDCARD(useragent) and then i have in the csv file (Look up file).
BOTs useragent Traverse *Traverse* Capture *Capture*

But i am not getting them grouped. One thing i want to mention is, i already have BOTs filed which extracts all the legitimate BOTs (which have +http://....). I want to add these others into the same field which does not have standard user agent (+http://.. format).

Do you think it would work that way?

Re: Can I specify a regex in a lookup table to group similar requests into the same output lookup?

xvxt006 — Thu, 19 Sep 2013 01:06:07 GMT

Sorry in the csv i have like this.

BOTs useragent Traverse *Traverse* Capture *Capture*

Re: Can I specify a regex in a lookup table to group similar requests into the same output lookup?

xvxt006 — Thu, 19 Sep 2013 01:10:17 GMT

i think it is not showing asterisks in the comments