Can I specify a regex in a lookup table to group similar requests into the same output lookup?

[xvxt006]

Hi,

We would like to create a look up table based on some user agents.

Mozilla/5.0 (compatible; Traverse/0.1; ABC 22175)
Mozilla/5.0 (compatible; Traverse/0.1; ABC 23457)
Mozilla/5.0 (compatible; Capture/0.4; ABC 56439)
Mozilla/5.0 (compatible; Capture/0.2; ABC 98123)

I would like to group similar kind of requests in the look up table and save them into Field XXX.

So field XXX should show
Traverse 2 requests
Catpure 2 requests

So can i specify reg ex in the look up table as there will be multiple patterns which i would like to group them.

[lguinn2 · Accepted solution]

There is an app that provides a dynamic lookup for user agent strings; it is called TA-uas_parser. Download it from

http://apps.splunk.com/app/1007

It's free. The user agent string can be very complex. I don't recommend that you build this yourself.

If you really want to do it youself, you can use wildcards (regular expressions) in the input field of a lookup table.
See How to use wildcards in a lookup table for more info.

[xvxt006]

i think it is not showing asterisks in the comments

[xvxt006]

Sorry in the csv i have like this.

BOTs useragent
Traverse *Traverse*
Capture *Capture*

[xvxt006]

Hi, i tried the match_type = WILDCARD(useragent) and then i have in the csv file (Look up file).
BOTs useragent
Traverse *Traverse*
Capture *Capture*

But i am not getting them grouped. One thing i want to mention is, i already have BOTs filed which extracts all the legitimate BOTs (which have +http://....). I want to add these others into the same field which does not have standard user agent (+http://.. format).

Do you think it would work that way?

[xvxt006]

Thank you. I will try this.