topic Re: In field extraction, how to do the matching between them and increment the result? in Splunk Search

In field extraction, how to do the matching between them and increment the result?

omarka — Tue, 03 Jul 2018 13:50:19 GMT

Hi everyone,

I'm looking to have this result:

For that I have 2 lines in my file:

Question: Service + IdTransaction
Response: Status + IdTransaction

Until now i can extract the different name of service and different codes but i don't know how to do the matching between them and to increment the result.

| rex "(?<Service>CONSULT|FIN_GB|FIN_RESERVE|FIN_VENDEUR|AUTHENTIF)" 
| rex field=_raw "Tlv Dico : (?<new>.{22}.{27})?"
| rex field=new "2004(?<Status>.{5})?"
| stats count(TransactionId) by Service , Status

Re: In field extraction, how to do the matching between them and increment the result?

FrankVl — Tue, 03 Jul 2018 14:12:48 GMT

Can you explain how the output of that search query you have so far does not match with what you want?

Re: In field extraction, how to do the matching between them and increment the result?

omarka — Tue, 03 Jul 2018 14:15:59 GMT

Well, it tells me: No results found.

Re: In field extraction, how to do the matching between them and increment the result?

FrankVl — Tue, 03 Jul 2018 14:18:44 GMT

Are you running just that specific search? Because I would expect there needs to be something before that, to actually search some data (ie index=foo sourcetype=bar).

Also: that TransactionId field, does that exist and contain data?

Re: In field extraction, how to do the matching between them and increment the result?

niketn — Tue, 03 Jul 2018 14:29:44 GMT

@omarka if you can add some masked/anonymized sample events, it would be easier for the community members to help you with regex as it would be strictly dependent on your data.

Re: In field extraction, how to do the matching between them and increment the result?

omarka — Tue, 03 Jul 2018 14:30:00 GMT

No i'm running this search : (host=g5d66999 OR g5d66956) Logger=srvca TLV | rex "(?CONSULT|FIN_GB|FIN_RESERVE|FIN_VENDEUR|AUTHENTIF)" | rex field=_raw "Tlv Dico : (?.{22}.{27})?" | rex field=new "2004(?.{5})?" | stats count(TransactionId) by Service , Status
And yes TransactionID contains data

Re: In field extraction, how to do the matching between them and increment the result?

FrankVl — Tue, 03 Jul 2018 14:32:41 GMT

TransactionID or TransactionId? Field names are case sensitive!

Re: In field extraction, how to do the matching between them and increment the result?

omarka — Tue, 03 Jul 2018 14:39:03 GMT

Yes it is exactly TransactionId

Re: In field extraction, how to do the matching between them and increment the result?

omarka — Tue, 03 Jul 2018 15:05:35 GMT

I'll explain how it works and what i want.
I have 2 lines containing each one "TransactionId" & "Service" and "TransactionId" & "Status", so when we find

401 as TransactionId and CONSULT as Service
401 as TransactionId and 000000 as Status

So (for example) this is the first line in results, if we find

453 as TransactionId and CONSULT as Service
453 as TransactionId and 000000 as Status

It will increment the first line as shown in the table at the top and so on ...

Re: In field extraction, how to do the matching between them and increment the result?

DalJeanis — Tue, 03 Jul 2018 15:15:23 GMT

You are getting no results because there is no event with both a Status and a TransactionId. You need to roll together your two events into a single event per TransactionId.

Try this...

 your base search
 | rex "(?<Service>CONSULT|FIN_GB|FIN_RESERVE|FIN_VENDEUR|AUTHENTIF)" 
 | rex field=_raw "Tlv Dico : (?<new>.{22}.{27})?"
 | rex field=new "2004(?<Status>.{5})?"
 | stats values(Service) as Service values(Status) as Status by TransactionId
 | stats count(TransactionId) by Service , Status

Re: In field extraction, how to do the matching between them and increment the result?

omarka — Tue, 03 Jul 2018 15:44:42 GMT

Thank you @DalJeanis for your answer.
However, i want to know if it's possible to switch or transpose this values by having something like that:

Service 00000 02040 06570
CONSULT 1650 150 15

Re: In field extraction, how to do the matching between them and increment the result?

DalJeanis — Tue, 03 Jul 2018 17:17:23 GMT

@omarka -

Assuming your columns are values of Status, then you are looking for the chart command. Replace the final stats with...

| chart count(TransactionId) by Service, Status

More detail on how to use it here -

http://docs.splunk.com/Documentation/Splunk/7.1.1/SearchReference/Chart