Splunk Search

rex field not visible and cannot be used in eval

maniishpawar
Path Finder

Hi I have this query and trying to do a eval case on the rex field value returned

base
| rex "#TAGRESPONSE.*RESPONSETYPE\:(?.+?)LICENSESTATE" 
| eval code=case('RESPONSETYPE'=="ER51","BATCH", 'RESPONSETYPE'=="ER91","NON-BATCH")
| stats count by code

Its not working, as it cannot see the extracted field RESPONSETYPE.
But when i do stats count by RESPONSETYPE, it works just fine.

Tags (1)

DalJeanis
SplunkTrust
SplunkTrust

I'm assuming that the second line looks something like this?

 | rex "#TAGRESPONSE.*RESPONSETYPE\:(?<RESPONSETYPE>.+?)LICENSESTATE" 

...and that the underlying _raw data looks something like this...

some stuff #TAGRESPONSE:foobar more stuff  RESPONSETYPE:someresponsetype LICENSESTATE:something else

If so, then what is happening is that your rex is picking up the space after responsetype and before licensestate, and that this code would work (but there's a better way)...

'RESPONSETYPE'=="ER51 "  

(notice the space after ER51?)

 | rex "#TAGRESPONSE.*RESPONSETYPE\:(?<RESPONSETYPE>\S+)\s+LICENSESTATE" 

This assumes that a responsetype cannot include any spaces. By narrowly typing the responsetype as "things that aren't whitespace (\S) you don't have to make it lazy, it will quit when it gets to the first whitespace, and not include that in the responsecode it is collecting for you.

On the other hand, if the responsecode CAN include spaces, then you do it this way...

 | rex "#TAGRESPONSE.*RESPONSETYPE\:(?<RESPONSETYPE>.+?)\s*LICENSESTATE" 
0 Karma

maniishpawar
Path Finder

oh and you can use RETURNCODE for regex, earlier i have given a dummy field name to post question.

0 Karma

maniishpawar
Path Finder

its weird, this time I tried space both trailing and leading and it worked. so i used trim and trim worked.
but why rex is adding spaces to the value retrieved ?
what should i change in rex to avoid the space, as if i have 5-10 fields extracted, each will have the trailing and leading space to their values

0 Karma

wenthold
Communicator

You have to account for possible trailing spaces in the rex, otherwise if they exist in the source data they will be captured. If you know they the spaces are always in the source data you can write:

| rex "#TAGRESPONSE.*?RESPONSETYPE\:(?<RESPONSETYPE>[^\s]+)\s+?LICENSESTATE"

But if you want account for the spaces possibly being there but account for the possibility that they won't?

| rex "#TAGRESPONSE.*?RESPONSETYPE\:\s*?(?<RESPONSETYPE>.*?)\s*?LICENSESTATE"

The first regex is less computationally expensive but won't capture fields that unless they are formatted properly.

0 Karma

maniishpawar
Path Finder

here is the actual data.
REQUESTTIME: 2017-11-17 09:28:29 RESPONSETIME: 2017-11-17 09:28:29 RETURNCODE: RATEFOR: CIFG BUSINESSFUNCTION: NBS

REQUESTTIME: 2017-11-17 09:28:29 RESPONSETIME: 2017-11-17 09:28:29 RETURNCODE: 9852 RATEFOR: CIFG BUSINESSFUNCTION: NBS

0 Karma

maniishpawar
Path Finder

Also I tried to use empty space at the beginning and at trailing of ER51 code, it didnt work in any of those scenarios.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

We've marked your code as code, so that HTML-like attributes will not be deleted by the interface.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!