Unable to obtain output using REX command ( User Agent)

[jaibalaraman]

Hi 

First , I would like to thank everyone in this community who guided and helped me a lot. 

Now i have a problem executing the below rex command

User agent - Mozilla/5.0 (Linux; Android 8.1.0; ASUS_X00ID) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Mobile Safari/537.36

REX - \(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)

I tested the rex command in " regex101.com and it match the information correction and i getting the output as expected.  However when i tried executing the same command in Splunk i am getting a blank screen in the Statistics view. 

[jaibalaraman]

Yes, i tried exactly same but i am getting error msg as your file format not support.

I tried, PNG, jpeg, jpg, PowerPoint still same also when i tried alternate option like browse and select the img are not listed ( All files ).

Thanks 

[gcusello]

Hi @jaibalaraman,

it's strange: there isn't any requirement on file format, only dimension less than 5 MB.

Ciao.

Giuseppe

[jaibalaraman]

2 - i think that you already tested your search in Verbose Mode.

Yes, i have tested in verbose mode

3- try to move the fields in Selected Fields.

If you don't see some of the files, maybe they are too few to be listed in Interesting Fields, so try to add to your search nomefile=*, in this way you're sure that the field

is extracted and you can add it to Selected Fields.

Sorry i don't understand, could you please guide how to do.

[jaibalaraman]

1 - your tested your regex in regex101.com then in Splunk you see the first two fields in the interesting fields but not in the table.

Yes, no error in splunk, but the table is showing blank data. From the below list only 2 two fields appear in interesting field

browser_version, os_version

not listed fields are os_family, device_brand_model,browser_engine,browser,Browser_enginer_version,hardware_type

[gcusello]

Hi @jaibalaraman,

maybe you have few values so Splunk doesn't put them in interesting fields.

So try running your search in this way:

your_search device_brand_model=*

in this way, if you correctly extracted the "device_brand_model" field, you'll have it in interesting fields and you can move it in Selected fields.

In this way you'll see it even if you have few values.

then repeat this procedure also fo the other missing fields.

Ciao.

Giuseppe

[jaibalaraman]

Hi

I have tried the "device_brand_model=*" in the SPL search and i getting the below error message . I am sure that i have done something wrong.

Could you please guide 

" Error in 'rex' command: Encountered the following error while compiling the regex '\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model=*>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)': Regex: syntax error in subpattern name (missing terminator) "

Thanks

[gcusello]

Hi @jaibalaraman,

could you share your search?

The error message says that's a formal error in the regex (quotes or parenthesys).

Ciao.

Giuseppe

[jaibalaraman]

Hi 

Yes, please find below

" |rex"\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model=*>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)" device_brand_model=*"

Thanks

[gcusello]

Hi @jaibalaraman,

in the regex you shared there's an error (present also in the first image but not in the second: 

<device_brand_model=*>  it's wrong
<device_brand_model> it's correct

then after there's a single dot, use:

.*

Then at the end of the shared regex (and in the first image) there's an error that probably it's a copy error:

device_brand_model=*"

Anyway, this should be the correct regex:

| rex "\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+);\s(?<device_brand_model>\w+).*\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)"

That you can test at https://regex101.com/r/M0V69d/1/

Ciao.

Giuseppe

[jaibalaraman]

Hi 

Sorry for the late response 

Yes, i have tried the REX command , but i am getting some error message 

Also when i run my original REX command , now i can see all filed in the selected filed. I dont know what happen now i can see all the fields,

REX - "\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+)\s(?<device_brand_model>\w+).\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)"

However still i can see only blank table 

Thanks 

[gcusello]

Hi @jaibalaraman,

at first, check if the fields you see in interesting fields are present in all the events or not (in the fields panel there's the perc of events with that field).

probably you extract few values and only putting those fields in "Selected fields" you can see them.

Anyway, try to add to your search (after the rex command) the command:

| search os_version=*

if in the way you see results, the problem is that you extract too few values.

Ciao.

Giuseppe

[jaibalaraman]

HI 

Now i find the way to add image, please find below

[scelikok]

Hi @jaibalaraman, can you please try using fieldname that contains UserAgent data for rex command. AWS http request log has this value as second value. That is why I put httpRequest.headers{1}.value , please check that field has UserAgent data otherwise correct the field index.

Also you can put your dedup command before rex.

index= aws
| dedup clientIp
| rex field=httpRequest.headers{1}.value "\(\w+;\s+(?<os_family>\w+)\s(?<os_version>\w+[^ ]+);\s(?<device_brand_model>\w+).*\s(?<browser_engine>\w+)\D(?<brow_engine_version>\w+[^ ]+)\s+\(.+\)\s+(?<browser>\w+).(?<browser_version>\w+[^ ]+)\s+(?<hardware_type>\w+)"

If this reply helps you an upvote is appreciated.

[jaibalaraman]

When i execute the command i see on result under Events however when i tried the output using table command i see only the header and blank screen. Also i can see only "browser_version, os_version" in the selected field 

I have a screen shot unfortunately i am unable to attach to explain more about my issue

[gcusello]

Hi @jaibalaraman,

to add a screenshot to your message you can drag and drop the file in the editing box or browse the file to attach, see the below box.

About the regex, let me understand:

your tested your regex in regex101.com then in Splunk you see the first two fields in the interesting fields but not in the table, is it correct?

i think that you already tested your search in Verbose Mode.

Anyway, try to move the fields in Selected Fields.

If you don't see some of the files, maybe they are too few to be listed in Interesting Fields, so try to add to your search nomefile=*, in this way you're sure that the fiekld is extracted and you can add it to Selected Fields.

Ciao.

Giuseppe