topic Re: User agent - Difficult in extracting Field in Splunk Search

User agent - Difficult in extracting Field

jaibalaraman — Mon, 09 Nov 2020 03:25:35 GMT

I am trying to extract field from the user agent details like ( Operating system, Software, Software version, Software type, Os version, Hardware type)

However i am finding some difficulty extracting the field . For example Operation system in Android, IOS & desktop are in the different field which highlighted below.

Android user - Mozilla/5.0 (Linux; Android 10; SAMSUNG SM-T590) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser / 12.1 Chrome/79.0.3945.136 Safari/537.36

Iphone user - Mozilla/5.0 (iPhone; CPU iPhone OS 14_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Mobile/15E148 Safari/604.1

Desktop user - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36

can someone help me how do extract field from the above user agent

Software, Software version, Hardware type, Operation System, Operating system name , Operation system version.

Thanks

Re: User agent - Difficult in extracting Field

ITWhisperer — Mon, 09 Nov 2020 14:49:08 GMT

There is no single agreed standard for user agent strings. Probably the best you could do is to use rex to pick out matching strings and if none is found tag it as unrecognised e.g.

| rex "\(.*(?<OS>Android\s\d+|OS \d+_\d+|Windows NT\s\d+\.\d+)\;?.*\)" | fillnull value="unrecognised" OS

Re: User agent - Difficult in extracting Field

jaibalaraman — Tue, 10 Nov 2020 01:50:59 GMT

Thanks for the rex code, yes its working however i am able to extract only OS( Operation System) also i am looking for the below

I am trying to extract

Text color - Represent user device

Text color - Represent Software (Browser )

Android user - Mozilla/5.0 (Linux; Android 10; SAMSUNG SMT590) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser / 12.1 Chrome/79.0.3945.136 Safari/537.36

Iphone user - Mozilla/5.0 (iPhone; CPU iPhone OS 14_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Mobile/15E148 Safari/604.1

Desktop user - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36

HP device

Mozilla/5.0 (Linux; Android 5.1.1; HP Pro Slate 12 Build/LMY47V; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/68.0.3440.91 Safari/537.36

Nokia

Mozilla/5.0 (Linux; Android 10; Nokia 7.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.116 Mobile Safari/537.36 EdgA/45.09.4.5083

Also i tried creating REX but unsuccessful ( \(.*(?<Software>SamsungBrowser\12.1\s*\d+) i dont know what is the mistake. Could you please help me on this.

Thanks

Re: User agent - Difficult in extracting Field

jaibalaraman — Tue, 10 Nov 2020 02:56:05 GMT

I tried creating the below REX "
\(.*(?<Device>SAMSUNG\s+SM-\d+|Windows NT\s\d+|iPhone;|SAMSUNG\sSM-\d+)"

Its not working 100% the output i can see only window & Iphone not samsung, hp etc

Re: User agent - Difficult in extracting Field

jaibalaraman — Tue, 10 Nov 2020 02:57:42 GMT

I am expecting the outcome "

Device

Browser details

Browser Version

Re: User agent - Difficult in extracting Field

ITWhisperer — Tue, 10 Nov 2020 07:41:29 GMT

You may need to escape the hyphens and the slashes. You should try your rex at regex101.com - you can copy all the user agent lines in and see how well your rex works against them all. You may want to try breaking up the string into parts and using other rex on just parts e.g.

etc, Note that not all user agent strings follow this pattern so you still may get some that fall through, but you can find those and extend your rex to cover them all eventually (until a manufacturer brings out a new phone or OS that you hadn't accounted for!). This is an ongoing activity and you might want to question the value you are getting from knowing this information!