Splunk Search

Does someone have a lookup table for user agent strings (browser, os, versions, ect.) that they can share?

swb03
Explorer

I've seen a Python script and App for this, but not a lookup table. Since my admin is not willing to install either one of those, I'm hoping someone has a lookup example, hopefully current (ex. IE 11), that they can share?

Thanks in advance

1 Solution

kaufmanm
Communicator

We use a Python script as you mentioned for this, but the logic in the Python script is simple enough to build into the Splunk search language directly. You can use the eval match() command to map out the different fields.

This example assumes you've already extracted out the user-agent string to a field named useragent, you could use _raw to start with if you don't have that field extracted.

* | eval os = case(match(useragent,"Windows .. 5\.1"),"Windows XP",match(useragent,"droid"),"Android",match(useragent,"Windows NT 6.1"),"Windows 7") | eval browser = case(match(useragent,"MSIE 10.0"),"Internet Explorer 10.0",match(useragent,"Chrome"),"Chrome",match(useragent,"Safari/"),"Safari") | eval arch = case(match(useragent,"droid"),"android",match(useragent,"iPad"),"ipad",match(useragent,"iPod"),"ipod")

I just picked three from each mapping, the search will get quite long implementing this way, and you'd want to add something to handle the case where it doesn't match anything. I'd definitely advise getting the approval from whoever necessary to put this into a script instead.

We use this script internally, you'd have to update it for IE 11:

https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py

Funny fact, when I tested this on our data, someone was in fact viewing our site with an iPod in the first 20 results. Who knew?

If you really really want a lookup, you could append "| fields useragent os browser arch | outputlookup useragentfields.csv" to the search and generate a lookup file from the results. You could then maybe schedule this as a saved search to run every five minutes, appending to your lookup file. The disadvantage is anytime you use this lookup, it'll only work if you've already seen the exact useragent before.

View solution in original post

kaufmanm
Communicator

We use a Python script as you mentioned for this, but the logic in the Python script is simple enough to build into the Splunk search language directly. You can use the eval match() command to map out the different fields.

This example assumes you've already extracted out the user-agent string to a field named useragent, you could use _raw to start with if you don't have that field extracted.

* | eval os = case(match(useragent,"Windows .. 5\.1"),"Windows XP",match(useragent,"droid"),"Android",match(useragent,"Windows NT 6.1"),"Windows 7") | eval browser = case(match(useragent,"MSIE 10.0"),"Internet Explorer 10.0",match(useragent,"Chrome"),"Chrome",match(useragent,"Safari/"),"Safari") | eval arch = case(match(useragent,"droid"),"android",match(useragent,"iPad"),"ipad",match(useragent,"iPod"),"ipod")

I just picked three from each mapping, the search will get quite long implementing this way, and you'd want to add something to handle the case where it doesn't match anything. I'd definitely advise getting the approval from whoever necessary to put this into a script instead.

We use this script internally, you'd have to update it for IE 11:

https://github.com/JustinAzoff/splunk-scripts/blob/master/ua2os.py

Funny fact, when I tested this on our data, someone was in fact viewing our site with an iPod in the first 20 results. Who knew?

If you really really want a lookup, you could append "| fields useragent os browser arch | outputlookup useragentfields.csv" to the search and generate a lookup file from the results. You could then maybe schedule this as a saved search to run every five minutes, appending to your lookup file. The disadvantage is anytime you use this lookup, it'll only work if you've already seen the exact useragent before.

View solution in original post

justdan23
Path Finder

Do you have an example of how this is hooked up to the eval to launch the script?

(I'll look it up, but was just curious. I had a Splunk SME say it was more efficient to do it with an eval and not a script though. He wouldn't share the script though and said he needed us to give him a quote to build it. Ugh!)

0 Karma

swb03
Explorer

Thank you both for your feedback.

0 Karma

lguinn2
Legend

One of the issues is that the useragent is browser-reported info. So a connection can tell the web server anything that it wants - the connection could be completely lying or it could be using compatibility mode... so I look at this as interesting information, but not authoritative.

Also, if this App or script makes your Splunk admin nervous - they can install it so that it is private to you - no one else would be able to see or use it... but that's a separate Q&A 🙂

kaufmanm
Communicator

Yes, it would be simple and if you look at the script the case for Trident 4.0 is already handled. The key thing is the script goes through the mapping and returns the first match. So put this sort of match at the top of the list, that way the Trident user agents will match that instead of the generic IE7 match that can be further down the list.

0 Karma

swb03
Explorer

Thanks for your reply.

Sounds like my best bet is to try and get the script installed.

Let me ask you this, is there any way to update this script to account for compatibility mode with IE (ex. Trident info)? For instance, using the script as is, it may show someone as having IE 7, but in reality they could be using IE 11 in compatibility mode: http://msdn.microsoft.com/library/ms537503.aspx#TriToken

Thanks again.

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!