topic Re: Extract information via regex in Splunk Search

Extract information via regex

rafamss — Mon, 02 Dec 2013 13:46:45 GMT

Hi guys,

I need some help to split the field below:

xyu_0987|123456:123456|123456:123456,

before the first pipeline(xyz_9807) displays the information about one rule;
after the first pipeline(|93830:09383) displays the information about alerts rules;
after the second pipeline(|093732:827332) displays information about denied rules.

The second point is that each rule is segmented by " : ", how i split the fields in order to keep this format: Remembering that segmented rule can be repeat more times, and each rule must be in fields separated.

Rule       Alerts Rule              Denied Rule 
           Rule 1 : Rule 2          Rule 1 : Rule 2
xyu_0987   123456 : 123456          123456 : 123456

Can you help me ?

Regards,

Rafael Martins

Re: Extract information via regex

somesoni2 — Mon, 02 Dec 2013 14:14:40 GMT

"Remembering that segmented rule can be repeat more times, and each rule must be in fields separated".
Can you provide some example of this as well?

Re: Extract information via regex

sbsbb — Mon, 02 Dec 2013 15:37:28 GMT

For making my tries with splunk regex, I use the program Kodos, that use the same synthax...

I would suggest you something like (I'm not good at regex, but mostly achieve what I want 😉

| rex max_match=0 "(?P<Rule>.*)\|(?P<Alert1>.*):(?P<Alert2>.*)\|(?P<Denied1>.*):(?P<Denied2>.*)"

If the number of Rules between the pipes is unknown, than I would try first to extract AlertString, DeniedString, and then extract the Rules as Multivalue Field you could mvexpand :

| rex max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)
| rex max_match=0 field=AlertString "(?P<AlertRules>[regex to split Rules])"
| rex max_match=0 field=AlertString "(?P<DeniedRules>[regex to split Rules])"
| mvexpand AlertRules | fields - AlertString
| mvexpand DeniedRules | fields - DeniedString
| table *

Re: Extract information via regex

rafamss — Mon, 02 Dec 2013 15:48:02 GMT

Sure somesoni2.

Example:
For the first pipeline, we have |987654:1098762:098675, in this case, we have three distinct values for alert rules.
For the second pipeline, we have |092822:098271, in this case, we have two distinct values for denied rules.

Do you understand ?

Regards,

Rafael Martins

Re: Extract information via regex

rafamss — Mon, 28 Sep 2020 15:23:51 GMT

Hi sbsbb,

I tried this solution, but without sucess. Take a look in this line of the log. I tried to use that refex that you passed but don't work. The format of log is access_combined_wcookie

192.168.0.1 - - [28/Nov/2013:02:09:24 +0000] "GET /127.0.0.1/icon.png HTTP/1.1" 200 3288 "-" "Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0" "-" "rule_1234|098765?098765|123456?123456"

Re: Extract information via regex

sbsbb — Mon, 02 Dec 2013 16:49:39 GMT

Of course you input is different, than the example... you have ? in place of :

(?Prule_.)|(?P.)\?(?P.)|(?P.)\?(?P.*)"

Re: Extract information via regex

rafamss — Mon, 02 Dec 2013 17:59:22 GMT

Hi sbsbb,

Indeed : is the correct, has error of typing, sorry!

Re: Extract information via regex

somesoni2 — Mon, 02 Dec 2013 18:47:25 GMT

If you have already extracted the field from your log which has rules information, say ruleInfo, the following works.

your search with ruleInfo extracted|rex field=ruleInfo max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)" | rex max_match=0 field=AlertString "(?P<AlertRules>[^:]+)" | rex max_match=0 field=DeniedString "(?P<DeniedRules>[^:]+)" | table Rule, AlertRules, DeniedRules | mvexpand AlertRules | mvexpand DeniedRules | mvcombine delim=":"  AlertRules | mvcombine delim=":" DeniedRules | nomv DeniedRules

The output is like below:-

Rule          AlertRules                     DeniedRules     
gijo_0987     10856:123456                   123326:133456                     
sms_0987      123456:147556:1237776:28856    10056:126656

Re: Extract information via regex

rafamss — Tue, 03 Dec 2013 19:16:47 GMT

Hi sbsbb,

I tried to use your code, but in some cases the regex don't work correctly, so I changed the regex to below:

But when I put the regex separately, it works, when I put the regex together, both doesn't works.

What do you think ?

Re: Extract information via regex

rafamss — Tue, 03 Dec 2013 19:22:16 GMT

Hi somesoni2,

I need that fields are separately by AlertRules, so for the AlertRules, each rule is separately by ":". The regex to below is correct, but doesn't works together.

Re: Extract information via regex

rafamss — Tue, 03 Dec 2013 19:26:57 GMT

Hi sbsbb,

Your solution code worked fine, but I had to add some pieces. Please look my question below.