topic Unable to obtain output using REX command ( User Agent) in Splunk Search

Unable to obtain output using REX command ( User Agent)

jaibalaraman — Fri, 18 Dec 2020 03:34:02 GMT

First , I would like to thank everyone in this community who guided and helped me a lot.

Now i have a problem executing the below rex command

User agent - Mozilla/5.0 (Linux; Android 8.1.0; ASUS_X00ID) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36

REX - \(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)

I tested the rex command in " regex101.com and it match the information correction and i getting the output as expected. However when i tried executing the same command in Splunk i am getting a blank screen in the Statistics view.

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Fri, 18 Dec 2020 03:42:11 GMT

When i execute the command i see on result under Events however when i tried the output using table command i see only the header and blank screen. Also i can see only "browser_version, os_version" in the selected field

I have a screen shot unfortunately i am unable to attach to explain more about my issue

Re: Unable to obtain output using REX command ( User Agent)

gcusello — Fri, 18 Dec 2020 06:45:31 GMT

Hi @jaibalaraman,

to add a screenshot to your message you can drag and drop the file in the editing box or browse the file to attach, see the below box.

About the regex, let me understand:

your tested your regex in regex101.com then in Splunk you see the first two fields in the interesting fields but not in the table, is it correct?

i think that you already tested your search in Verbose Mode.

Anyway, try to move the fields in Selected Fields.

If you don't see some of the files, maybe they are too few to be listed in Interesting Fields, so try to add to your search nomefile=*, in this way you're sure that the fiekld is extracted and you can add it to Selected Fields.

Ciao.

Giuseppe

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Mon, 21 Dec 2020 19:37:48 GMT

1 - your tested your regex in regex101.com then in Splunk you see the first two fields in the interesting fields but not in the table.

Yes, no error in splunk, but the table is showing blank data. From the below list only 2 two fields appear in interesting field

browser_version, os_version

not listed fields are os_family, device_brand_model,browser_engine,browser,Browser_enginer_version,hardware_type

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Mon, 21 Dec 2020 19:41:24 GMT

2 - i think that you already tested your search in Verbose Mode.

Yes, i have tested in verbose mode

3- try to move the fields in Selected Fields.

If you don't see some of the files, maybe they are too few to be listed in Interesting Fields, so try to add to your search nomefile=*, in this way you're sure that the field

is extracted and you can add it to Selected Fields.

Sorry i don't understand, could you please guide how to do.

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Mon, 21 Dec 2020 19:44:03 GMT

Yes, i tried exactly same but i am getting error msg as your file format not support.

I tried, PNG, jpeg, jpg, PowerPoint still same also when i tried alternate option like browse and select the img are not listed ( All files ).

Thanks

Re: Unable to obtain output using REX command ( User Agent)

gcusello — Tue, 22 Dec 2020 07:13:52 GMT

Hi @jaibalaraman,

maybe you have few values so Splunk doesn't put them in interesting fields.

So try running your search in this way:

your_search device_brand_model=*

in this way, if you correctly extracted the "device_brand_model" field, you'll have it in interesting fields and you can move it in Selected fields.

In this way you'll see it even if you have few values.

then repeat this procedure also fo the other missing fields.

Ciao.

Giuseppe

Re: Unable to obtain output using REX command ( User Agent)

gcusello — Tue, 22 Dec 2020 07:17:12 GMT

Hi @jaibalaraman,

it's strange: there isn't any requirement on file format, only dimension less than 5 MB.

Ciao.

Giuseppe

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Tue, 22 Dec 2020 20:03:07 GMT

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Wed, 23 Dec 2020 02:38:20 GMT

I have tried the "device_brand_model=*" in the SPL search and i getting the below error message . I am sure that i have done something wrong.

Could you please guide

" Error in 'rex' command: Encountered the following error while compiling the regex '\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model=*>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)': Regex: syntax error in subpattern name (missing terminator) "

Thanks

Re: Unable to obtain output using REX command ( User Agent)

gcusello — Wed, 23 Dec 2020 07:27:02 GMT

Hi @jaibalaraman,

could you share your search?

The error message says that's a formal error in the regex (quotes or parenthesys).

Ciao.

Giuseppe

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Wed, 23 Dec 2020 20:55:11 GMT

Yes, please find below

" |rex"\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model=*>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)" device_brand_model=*"

Thanks

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Wed, 23 Dec 2020 20:56:46 GMT

Now i find the way to add image, please find below

Re: Unable to obtain output using REX command ( User Agent)

gcusello — Thu, 24 Dec 2020 09:20:59 GMT

Hi @jaibalaraman,

in the regex you shared there's an error (present also in the first image but not in the second:

<device_brand_model=*> it's wrong <device_brand_model> it's correct

then after there's a single dot, use:

Then at the end of the shared regex (and in the first image) there's an error that probably it's a copy error:

device_brand_model=*"

Anyway, this should be the correct regex:

| rex "\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+);\s(?<device_brand_model>\w+).*\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)"

That you can test at https://regex101.com/r/M0V69d/1/

Ciao.

Giuseppe

Re: Unable to obtain output using REX command ( User Agent)

scelikok — Thu, 24 Dec 2020 10:08:53 GMT

Hi @jaibalaraman, can you please try using fieldname that contains UserAgent data for rex command. AWS http request log has this value as second value. That is why I put httpRequest.headers{1}.value , please check that field has UserAgent data otherwise correct the field index.

Also you can put your dedup command before rex.

index= aws | dedup clientIp | rex field=httpRequest.headers{1}.value "\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+);\s(?<device_brand_model>\w+).*\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)"

If this reply helps you an upvote is appreciated.

Re: Unable to obtain output using REX command ( User Agent)

jaibalaraman — Wed, 30 Dec 2020 03:36:54 GMT

Sorry for the late response

Yes, i have tried the REX command , but i am getting some error message

Also when i run my original REX command , now i can see all filed in the selected filed. I dont know what happen now i can see all the fields,

REX - "\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)"

However still i can see only blank table

Thanks

Re: Unable to obtain output using REX command ( User Agent)

gcusello — Wed, 30 Dec 2020 13:11:11 GMT

Hi @jaibalaraman,

at first, check if the fields you see in interesting fields are present in all the events or not (in the fields panel there's the perc of events with that field).

probably you extract few values and only putting those fields in "Selected fields" you can see them.

Anyway, try to add to your search (after the rex command) the command:

| search os_version=*

if in the way you see results, the problem is that you extract too few values.

Ciao.

Giuseppe