topic How to write a regular expression for extracting OS version number from User Agent in my sample data? in Splunk Search

How to write a regular expression for extracting OS version number from User Agent in my sample data?

pradjswl — Tue, 11 Apr 2017 16:15:39 GMT

How do we write a regular expression to extract a OS version from the User Agent considering the fact that UserAgent format is not always consistent? I searched this forum, and found there was similar thread where a Splunk App was suggested to user. However I am regular user, and my Splunk admin doesn't allow me the permission to install any Splunk app. Is there a way I can write regular expression to extract this info as this is present in the UserAgent?

Sample:

Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

cpetterborg — Tue, 11 Apr 2017 16:27:24 GMT

As you say, "UserAgent format is not always consistent." Please provide more than one example string that you feel you need to extract from. If you have 20 significantly differing formats, please provide a good number of them as examples.

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

pradjswl — Tue, 29 Sep 2020 13:38:17 GMT

ty @cpetterborg for your response.

These are the sample of User agent

Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27

Mozilla/5.0 (Linux; Android 7.0; SAMSUNG-SM-G930A Build/NRD90M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Mobile/14E304

Mozilla/5.0 (Linux; Android 7.0; SAMSUNG-SM-G935A Build/NRD90M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (iPad; CPU OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Mobile/14E304

Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Version/10.0 Mobile/14D27 Safari/602.1

Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-G920A Build/MMB29K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (Linux; Android 7.0; LG-H820 Build/NRD90U; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-G935A Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (iPad; CPU OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Version/10.0 Mobile/14D27 Safari/602.1

Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.0 Mobile/14E304 Safari/602.1

Mozilla/5.0 (iPhone; CPU iPhone OS 10_3 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Mobile/14E277

Mozilla/5.0 (iPad; CPU OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27
Mozilla/5.0 (iPhone; CPU iPhone OS 10_1_1 like Mac OS X) AppleWebKit/602.2.14 (KHTML, like Gecko) Mobile/14B100

Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13F69

Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Mobile/14A456

Mozilla/5.0 (Windows NT 6.1; rv:41.0) Gecko/20100101 Firefox/41.0

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0

Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-G900A Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-N920A Build/MMB29K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (iPhone; CPU iPhone OS 10_2 like Mac OS X) AppleWebKit/602.3.12 (KHTML, like Gecko) Mobile/14C92

Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG-SM-G928A Build/MMB29K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/57.0.2987.132 Mobile Safari/537.36

Mozilla/5.0 (Linux; Android 6.0.1; SM-N920T Build/MMB29K; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/55.0.2883.91 Mobile Safari/537.36

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

cpetterborg — Tue, 11 Apr 2017 16:52:57 GMT

Does this give you some information that you can use, and if so, what info is of most use to you?:

<yoursearch> | rex "\((?P<osinfo>[^\)]+)\)" | rex field=osinfo "(?P<os>[^;]+);(?P<vers>[^;]+)(;(?P<etc>[^;]+))?" | stats count by os, vers

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

pradjswl — Tue, 29 Sep 2020 13:38:25 GMT

I am validating in regex101.com , but it desont return any result. Are you able to get the result ? I am checking for 1st sample Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27 .
OS version in this case is 10_2_1. If you are getting the right result, could you please share the screen shot too ?

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

pradjswl — Tue, 29 Sep 2020 13:38:33 GMT

UserAgent has different format for iOS & Andorid as we can see below,

Mozilla/5.0 (iPhone; CPU iPhone OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Mobile/14D27

Mozilla/5.0 (Linux; Android 7.0; SAMSUNG-SM-G930A Build/NRD90M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0

I would like to extract them as "iPhone OS 10_2_1" & "Android 7.0" , would that be possible ? I am struggling to put OR condition where it would check different format based on iOS & Andoird

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

cpetterborg — Tue, 11 Apr 2017 20:23:52 GMT

I brought your data (the example lines) into my local machine and did the field extractions inside of Splunk, so it worked fine for me.

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

cpetterborg — Tue, 11 Apr 2017 20:34:40 GMT

In regex101 you can use the following to get the first two fields extracted (using multiline):

\((?P<os>[^;]+);(?P<vers>[^;)]+).*$

Does that give you what you want from the data? or is there more that you need to use?

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

pradjswl — Tue, 11 Apr 2017 21:33:29 GMT

This works great.

PS1 : I dont see an option to accept this as answer for this thread.
PS 2: I would raise a new thread "How to create a extracted filed using regex on existing field" ? By default regex uses _raw field in the field extractor. I dont want to use regex as part of the query but I want a field to be created in the event/app like calculated filed so it always stay as new field rather than specifying in the search query.

Re: How to write a regular expression for extracting OS version number from User Agent in my sample data?

cpetterborg — Tue, 11 Apr 2017 21:39:56 GMT

Now you can accept this answer.

For your other question, just start a new question, since additional questions within a question are highly discouraged. It will get answered.