REX command issue for Multiple user agent

[jaibalaraman]

Hi 

As every one knew there are multiple user agent depends on user device.  However i am trying to achieve the below output from the user agent using table command.

sample output

os_family

os_version

device_brand_model

brower_enginer

brow_engine_version

hardware_type

browser

browser_version

User agent & Rex  

Iphone - Mozilla/5.0 (iPhone; CPU iPhone OS 14_2_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.1 Mobile/15E148 Safari/604.1

REX - \((?<hardware_type>\w+);\s+[^ ]+\s(?<os_family>\w+\s[^ ]+)\s+(?<os_version>\w+)\s[^ ]+\s[^ ]+\s\w+\s\w.\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser_version>\w+\/[^ ]+)\s+\w+\/\w+\s(?<browser>\w+)

Xiaomi - Mozilla/5.0 (Linux; U; Android 9; en-gb; Redmi Note 6 Pro Build/PKQ1.180904.001) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/71.0.3578.141 Mobile Safari/537.36 XiaoMi/MiuiBrowser/12.7.4-gn

REX - \(\w+;\s\w;\s(?<os_family>\w+)\s(?<os_version>\w+);\s[^ ]+\s(?<device_brand_model>\w+\s[^ ]+\s[^ ]+)\s[^ ]+\s[^ ]+\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s\w+\/[^ ]+\s[^ ]+\s(?<hardware_type>\w+)\s[^ ]+\s(?<browser>\w+\/\w+)\/(?<browser_version>\w+[^ ]+)

One Plus - Mozilla/5.0 (Linux; Android 10; ONEPLUS A6013) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36 

REX - \(\w+;\s(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s+(?<device_brand_model>\w+\s[^ ]+)\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s(?<browser>\w+)\/(?<browser_version>\w+[^ ]+)\s(?<hardware_type>\w+)

Windows - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36 Edge/87.0.664.66

REX - \((?<os_family>\w+)\s+\w+\s+(?<os_version>[^;]+)[^\)]+\)\s(?<browser_egnine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s[^ ]+\s[^ ]+\s(?<browser>\w+)\/(?<browser_version>\w+[^ ]+)

Macintosh - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15"

REX - \((?<hardware_type>\w+);\s\w+\s+(?<os_family>\w+)\s(?<os_version>\w+\s[^ ]+\s[^ ]+)\s(?<browser_enginer>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s(?<browser_version>\w+\/[^ ]+)\s(?<browser>\w+)

Lenovo - Mozilla/5.0 (Linux; Android 6.0.1; Lenovo YT3-X90F) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Safari/537.36

REX - \(\w+;\s(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s+(?<device_brand_model>\w+\s\w+[^ ]+)\s+(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s(?<browser>\w+)\/(?<browser_version>\w+[^ ]+)

Like above i have created multiple REX command for ( Ipad/HP/Meizu/Vivo/Motorola/Lenovo/ZTE blade /One Plus / Xiaomi / Google Pixel / Android / LG / Asus/

I would like to know can we run spl cmd  with multiple REX command in single search or how can get the output i am expected to obtain all user agent details.

Thanks 

[jaibalaraman]

Hi 

It seems the browscap is not compatible with our version of Splunk. Could you please recommend list of various option ( Addon app ) to capture user agent details. 

Thanks

[richgalloway]

Check splunkbase for other app that are compatible with your version of Splunk.

Consider updating the browscap app to be compatible with your version of Splunk.

[richgalloway]

You might try combining all of the regex strings into a single regex using |.  You'll likely need the (?J) flag to avoid errors about duplicate fields.

A better way is to use an existing app.  See TA-user-agents (https://splunkbase.splunk.com/app/1843/) or TA-browscap (https://splunkbase.splunk.com/app/1021/).

[jaibalaraman]

Hi 

Could d you please give me some sample how do i join multiple REX command.

Sorry i am new and learning Splunk.

Thanks 

[richgalloway]

My advice was to join multiple regex strings, not multiple rex commands.  You would have a single rex command that would search for many regular expressions.  It would look something like this.

... | rex "(\((?<hardware_type>\w+);\s+[^ ]+\s(?<os_family>\w+\s[^ ]+)\s+(?<os_version>\w+)\s[^ ]+\s[^ ]+\s\w+\s\w.\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser_version>\w+\/[^ ]+)\s+\w+\/\w+\s(?<browser>\w+))|(\(\w+;\s\w;\s(?<os_family>\w+)\s(?<os_version>\w+);\s[^ ]+\s(?<device_brand_model>\w+\s[^ ]+\s[^ ]+)\s[^ ]+\s[^ ]+\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s\w+\/[^ ]+\s[^ ]+\s(?<hardware_type>\w+)\s[^ ]+\s(?<browser>\w+\/\w+)\/(?<browser_version>\w+[^ ]+))"
| ...

[jaibalaraman]

Hi

I tried the above rex command getting error msg.

Sorry about my poor knowledge in Splunk

| rex "(\((?<hardware_type>\w+);\s+[^ ]+\s(?<os_family>\w+\s[^ ]+)\s+(?<os_version>\w+)\s[^ ]+\s[^ ]+\s\w+\s\w.\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser_version>\w+\/[^ ]+)\s+\w+\/\w+\s(?<browser>\w+))|(\(\w+;\s\w;\s(?<os_family>\w+)\s(?<os_version>\w+);\s[^ ]+\s(?<device_brand_model>\w+\s[^ ]+\s[^ ]+)\s[^ ]+\s[^ ]+\s(?<browser_engine>\w+)\/(?<brow_engine_version>\w+[^ ]+)\s\(.+\)\s\w+\/[^ ]+\s[^ ]+\s(?<hardware_type>\w+)\s[^ ]+\s(?<browser>\w+\/\w+)\/(?<browser_version>\w+[^ ]+))

[richgalloway]

The example was just that - an example.  As I mentioned in my first reply, you'll have to account for multiple uses of the same field (named capture group).