topic Re: How can I extract text between 2 standard texts that may contain " . , ? / " ? in Splunk Search

How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Fri, 10 Jul 2015 14:35:13 GMT

I have this type of log:

Bin:456852 IssuingBank:PBS INTERNATIONAL A/S;DANSKE BANK;DANSKEBANK IssingCountry:DK IRF: Intra EEA CardProfile1:Consumer CardProfile2:Base

How can I extract : bank, issingcountry, IRF, Cardprofile1, CardProfile2? and then stats count by bank, issingcountry,IRF, Cardprofile1....

Can anyone please help me with this? anything I found in the forum did not work!

I tried something like this but I suppose I need sth upper level regex!

rex field=_raw "IssingCountry:(?<issingcountry>\w{2})" | rex field=_raw "IssuingBank:(?<bank>\w{13})"

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Fri, 10 Jul 2015 14:51:21 GMT

This should be helpful:

... | rex "IssuingBank:(?P<bank>.+).*?IssingCountry:(?P<issingcountry>\w+) IRF: (?<IRF>[ \w]+) CardProfile1:(?<cardProfile1>\w+) CardProfile2:(?P<cardProfile2>\w+)" | stats count by bank, issingcountry, IRF, cardProfile1

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Fri, 10 Jul 2015 15:36:36 GMT

Unfortunately it did not work!

The log is :

row1.....
row2....
row3....
row4....

18:00:19,727 INFO FlightsCostBasedGatewaySorter 5fd00a8d271411e598e306695ac8a534 BinInfo for Bin:656814 IssuingBank:SANTANDER, UK ?PLC IssingCountry:GB IRF:Intra EEA CardProfile1:Consumer CardProfile2:Base

I used this:

rex field=_raw "IssuingBank:(?P.+).*?IssingCountry:(?P\w+) IRF: (?[ \w]+) CardProfile1:(?\w+) CardProfile2:(?P\w+)" | stats count by bank, issingcountry, IRF, cardProfile1

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Fri, 10 Jul 2015 15:44:01 GMT

One of the reasons it did not work is you removed the field extractions from the regex string. Please try again using the regex in my answer.

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Fri, 10 Jul 2015 15:51:39 GMT

I used this: (like before, I miss typed what I used!)

rex "IssuingBank:(?P.+).*?IssingCountry:(?P\w+) IRF: (?[ \w]+) CardProfile1:(?\w+) CardProfile2:(?P\w+)" | stats count by bank, issingcountry, IRF, cardProfile1

And it did not work: No results found. Inspect ... !

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Fri, 10 Jul 2015 16:45:09 GMT

I still don't see the same regex as is in my answer. Try removing the stats command to see if you're getting data extracted from the events.

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Mon, 13 Jul 2015 11:37:06 GMT

Hello,

I used exactly what you proposed without the stat command and it seemed to work! But my initial purpose is to count them and export them into excel!

Do you have any idea how can I count them?

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Mon, 20 Jul 2015 02:30:38 GMT

Try ... | stats count(_time) by bank, issingcountry, IRF, cardProfile1

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Mon, 20 Jul 2015 06:54:02 GMT

Unfortunately it did not work. If I had to guess, I would say that the extracting method is not working.

It should have worked as it was:

... | rex "IssuingBank:(?P.+).*?IssingCountry:(?P\w+) IRF: (?[ \w]+) CardProfile1:(?\w+) CardProfile2:(?P\w+)" | stats count by bank, issingcountry, IRF, cardProfile1

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Mon, 20 Jul 2015 14:20:59 GMT

Didn't you say it worked without the stats command? That should mean the extraction is working.

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Mon, 20 Jul 2015 14:34:58 GMT

I thought that it worked but since I can not use stats count it showed me that it didn't!

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Mon, 20 Jul 2015 18:28:27 GMT

So what do you get?

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Tue, 21 Jul 2015 06:52:04 GMT

I am using this splunk version: 5.0.2, build 14956

It comes empty!! really appreciate your willingness to help out!

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Tue, 21 Jul 2015 22:17:23 GMT

If you run this query do you get any results?

... | rex "IssuingBank:(?P<bank>.+).*?IssingCountry:(?P<issingcountry>\w+) IRF: (?<IRF>[ \w]+) CardProfile1:(?<cardProfile1>\w+) CardProfile2:(?P<cardProfile2>\w+)" | table bank issingCountry IRF cardProfile1 cardProfile2

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Wed, 22 Jul 2015 08:29:59 GMT

Empty results again! 😞

https://www.diigo.com/item/image/564dw/ytbe

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

aholzel — Wed, 22 Jul 2015 11:32:08 GMT

I think there are two problems.
First one is that in the field extract you give the field the name issingcountry but in the table you ask for the field issingCountry (note the capital C of country).
Second one is that the regex doesn't match the problem is with the IRF field in the extract you place a space after "IRF:" but in the sample this space doesn't exist so there is no match.

Also while you are editing the regex may I suggest you use this one:

IssuingBank:(?P<bank>[^,]+).*?IssingCountry:(?P<issingCountry>\w+)\sIRF:(?<IRF>.+?)\sCardProfile1:(?<cardProfile1>\w+)\sCardProfile2:(?P<cardProfile2>\w+)

The regex above only needs 107 steps to match all the fields while the regex that you use needs 2769 steps to match, this means that you will have a faster search.
(note: in the regex above I used a capital c in issingCountry)

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Wed, 22 Jul 2015 12:27:30 GMT

Thank you very very much! This worked perfect!

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

richgalloway — Wed, 22 Jul 2015 15:05:00 GMT

So you're getting matching events, but no results. That means your data probably isn't matching the regex string. Please verify the output of flprd put/PAY matches your original post.

Re: How can I extract text between 2 standard texts that may contain " . , ? / " ?

kostasKats — Wed, 05 Aug 2015 08:23:19 GMT

Dear aholzel I want your help again with something else:

Parameters: {"referrer"=>"http://www.xxxxx.net/transport_deeplink/4.0/EE/en-gb/EUR

I have this line and i want ot use a regex command (rex field=_raw) in order to count them by:

domain: http://www.xxxxx.net
country: EE
language: en-gb
currency: EUR

Can you please help me with this?