rex field not visible and cannot be used in eval

[maniishpawar]

Hi I have this query and trying to do a eval case on the rex field value returned

base
| rex "#TAGRESPONSE.*RESPONSETYPE\:(?.+?)LICENSESTATE" 
| eval code=case('RESPONSETYPE'=="ER51","BATCH", 'RESPONSETYPE'=="ER91","NON-BATCH")
| stats count by code

Its not working, as it cannot see the extracted field RESPONSETYPE.
But when i do stats count by RESPONSETYPE, it works just fine.

[DalJeanis]

I'm assuming that the second line looks something like this?

 | rex "#TAGRESPONSE.*RESPONSETYPE\:(?<RESPONSETYPE>.+?)LICENSESTATE" 

...and that the underlying _raw data looks something like this...

some stuff #TAGRESPONSE:foobar more stuff  RESPONSETYPE:someresponsetype LICENSESTATE:something else

If so, then what is happening is that your rex is picking up the space after responsetype and before licensestate, and that this code would work (but there's a better way)...

'RESPONSETYPE'=="ER51 "  

(notice the space after ER51?)

 | rex "#TAGRESPONSE.*RESPONSETYPE\:(?<RESPONSETYPE>\S+)\s+LICENSESTATE" 

This assumes that a responsetype cannot include any spaces. By narrowly typing the responsetype as "things that aren't whitespace (\S) you don't have to make it lazy, it will quit when it gets to the first whitespace, and not include that in the responsecode it is collecting for you.

On the other hand, if the responsecode CAN include spaces, then you do it this way...

 | rex "#TAGRESPONSE.*RESPONSETYPE\:(?<RESPONSETYPE>.+?)\s*LICENSESTATE" 

[maniishpawar]

oh and you can use RETURNCODE for regex, earlier i have given a dummy field name to post question.

[maniishpawar]

its weird, this time I tried space both trailing and leading and it worked. so i used trim and trim worked.
but why rex is adding spaces to the value retrieved ?
what should i change in rex to avoid the space, as if i have 5-10 fields extracted, each will have the trailing and leading space to their values

[wenthold]

You have to account for possible trailing spaces in the rex, otherwise if they exist in the source data they will be captured. If you know they the spaces are always in the source data you can write:

| rex "#TAGRESPONSE.*?RESPONSETYPE\:(?<RESPONSETYPE>[^\s]+)\s+?LICENSESTATE"

But if you want account for the spaces possibly being there but account for the possibility that they won't?

| rex "#TAGRESPONSE.*?RESPONSETYPE\:\s*?(?<RESPONSETYPE>.*?)\s*?LICENSESTATE"

The first regex is less computationally expensive but won't capture fields that unless they are formatted properly.

[maniishpawar]

here is the actual data.
REQUESTTIME: 2017-11-17 09:28:29 RESPONSETIME: 2017-11-17 09:28:29 RETURNCODE: RATEFOR: CIFG BUSINESSFUNCTION: NBS

REQUESTTIME: 2017-11-17 09:28:29 RESPONSETIME: 2017-11-17 09:28:29 RETURNCODE: 9852 RATEFOR: CIFG BUSINESSFUNCTION: NBS

[maniishpawar]

Also I tried to use empty space at the beginning and at trailing of ER51 code, it didnt work in any of those scenarios.

[DalJeanis]

We've marked your code as code, so that HTML-like attributes will not be deleted by the interface.