Splunk Search

User agent - Difficult in extracting Field

jaibalaraman
Path Finder

Hi 

I am trying to extract field from the user agent details like ( Operating system, Software, Software version, Software type, Os version, Hardware type) 

However i am finding some difficulty extracting the field . For example Operation system in Android, IOS & desktop are in the different field which highlighted below. 

Android userMozilla/5.0 (Linux; Android 10; SAMSUNG SM-T590) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser / 12.1 Chrome/79.0.3945.136 Safari/537.36

 
 

Iphone userMozilla/5.0 (iPhone; CPU iPhone OS 14_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Mobile/15E148 Safari/604.1

Desktop userMozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36

can someone help me how do extract field from the above user agent 

Software, Software version, Hardware type, Operation System,  Operating system name , Operation system version. 

 

Thanks 

 

Labels (1)
Tags (1)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

There is no single agreed standard for user agent strings. Probably the best you could do is to use rex to pick out matching strings and if none is found tag it as unrecognised e.g.

| rex "\(.*(?<OS>Android\s\d+|OS \d+_\d+|Windows NT\s\d+\.\d+)\;?.*\)"
| fillnull value="unrecognised" OS

View solution in original post

jaibalaraman
Path Finder

I am expecting the outcome " 

OSDeviceBrowser details Browser Version
0 Karma

jaibalaraman
Path Finder

Hi 

I tried creating the below REX " 
\(.*(?<Device>SAMSUNG\s+SM-\d+|Windows NT\s\d+|iPhone;|SAMSUNG\sSM-\d+)"

Its not working 100%  the output i can see only window & Iphone not samsung, hp etc 

 

jaibalaraman_0-1604976955317.png

 

0 Karma

jaibalaraman
Path Finder

Hi 

Thanks for the rex code, yes its working however i am able to extract only OS( Operation System) also i am looking for the below 

I am trying to extract 

Text color - Represent user device 

Text color  - Represent Software (Browser )

Android user Mozilla/5.0 (LinuxAndroid 10SAMSUNG SMT590AppleWebKit/537.36 (KHTMLlike GeckoSamsungBrowser / 12.1 Chrome/79.0.3945.136 Safari/537.36

 

Iphone user Mozilla/5.0 (iPhoneCPU iPhone OS 14_1 like Mac OS XAppleWebKit/605.1.15 (KHTMLlike GeckoVersion/14.0 Mobile/15E148 Safari/604.1

Desktop user Mozilla/5.0 (Windows NT 10.0Win64x64AppleWebKit/537.36 (KHTMLlike GeckoChrome/86.0.4240.111 Safari/537.36

HP device

Mozilla/5.0 (Linux; Android 5.1.1; HP Pro Slate 12 Build/LMY47V; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/68.0.3440.91 Safari/537.36

Nokia

Mozilla/5.0 (Linux; Android 10; Nokia 7.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.116 Mobile Safari/537.36 EdgA/45.09.4.5083

 

Also i tried creating REX but unsuccessful ( \(.*(?<Software>SamsungBrowser\12.1\s*\d+) i dont know what is the mistake. Could you please help me on this.

 

Thanks

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You may need to escape the hyphens and the slashes. You should try your rex at regex101.com - you can copy all the user agent lines in and see how well your rex works against them all. You may want to try breaking up the string into parts and using other rex on just parts e.g.

| rex "(?<firstpart>[^\(]+)\((?<secondpart>[^\)]+)\)(?<thirdpart>[^\(]+)\((?<fourthpart>[^\)]+)\)(?<fifthpart>.*)"
| rex field=secondpart "(?<OS>Android|Windows|OS)"
| rex field=fifthpath "(?<browser>Safari|Chrome)"

etc, Note that not all user agent strings follow this pattern so you still may get some that fall through, but you can find those and extend your rex to cover them all eventually (until a manufacturer brings out a new phone or OS that you hadn't accounted for!). This is an ongoing activity and you might want to question the value you are getting from knowing this information! 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

There is no single agreed standard for user agent strings. Probably the best you could do is to use rex to pick out matching strings and if none is found tag it as unrecognised e.g.

| rex "\(.*(?<OS>Android\s\d+|OS \d+_\d+|Windows NT\s\d+\.\d+)\;?.*\)"
| fillnull value="unrecognised" OS
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...