Extract information via regex

[rafamss]

Hi guys,

I need some help to split the field below:

xyu_0987|123456:123456|123456:123456,

before the first pipeline(xyz_9807) displays the information about one rule;
after the first pipeline(|93830:09383) displays the information about alerts rules;
after the second pipeline(|093732:827332) displays information about denied rules.

The second point is that each rule is segmented by " : ", how i split the fields in order to keep this format: Remembering that segmented rule can be repeat more times, and each rule must be in fields separated.

Rule       Alerts Rule              Denied Rule 
           Rule 1 : Rule 2          Rule 1 : Rule 2
xyu_0987   123456 : 123456          123456 : 123456

Can you help me ?

Regards,

Rafael Martins

[somesoni2]

If you have already extracted the field from your log which has rules information, say ruleInfo, the following works.

your search with ruleInfo extracted|rex field=ruleInfo max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)" | rex max_match=0 field=AlertString "(?P<AlertRules>[^:]+)" | rex max_match=0 field=DeniedString "(?P<DeniedRules>[^:]+)" | table Rule, AlertRules, DeniedRules | mvexpand AlertRules | mvexpand DeniedRules | mvcombine delim=":"  AlertRules | mvcombine delim=":" DeniedRules | nomv DeniedRules

The output is like below:-

Rule          AlertRules                     DeniedRules     
gijo_0987     10856:123456                   123326:133456                     
sms_0987      123456:147556:1237776:28856    10056:126656 

[rafamss]

Hi somesoni2,

I need that fields are separately by AlertRules, so for the AlertRules, each rule is separately by ":". The regex to below is correct, but doesn't works together.

[rafamss]

Sure somesoni2.

Example:
For the first pipeline, we have |987654:1098762:098675, in this case, we have three distinct values for alert rules.
For the second pipeline, we have |092822:098271, in this case, we have two distinct values for denied rules.

Do you understand ?

Regards,

Rafael Martins

[sbsbb · Accepted solution]

For making my tries with splunk regex, I use the program Kodos, that use the same synthax...

I would suggest you something like (I'm not good at regex, but mostly achieve what I want 😉

| rex max_match=0 "(?P<Rule>.*)\|(?P<Alert1>.*):(?P<Alert2>.*)\|(?P<Denied1>.*):(?P<Denied2>.*)"

If the number of Rules between the pipes is unknown, than I would try first to extract AlertString, DeniedString, and then extract the Rules as Multivalue Field you could mvexpand :

| rex max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)
| rex max_match=0 field=AlertString "(?P<AlertRules>[regex to split Rules])"
| rex max_match=0 field=AlertString "(?P<DeniedRules>[regex to split Rules])"
| mvexpand AlertRules | fields - AlertString
| mvexpand DeniedRules | fields - DeniedString
| table *

[rafamss]

Hi sbsbb,

Your solution code worked fine, but I had to add some pieces. Please look my question below.

[rafamss]

Hi sbsbb,

I tried to use your code, but in some cases the regex don't work correctly, so I changed the regex to below:

| rex max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)" | rex max_match=0 field=AlertString "(?P<AlertRules>......:*)" | rex max_match=0 field=AlertString "(?P<AlertRules>......:*)" | rex max_match=0 field=AlertRules "(?P<AlertRule>......_*)" | mvexpand AlertRule | fields - AlertString | rex max_match=0 field=DeniedString "(?P<DeniedRules>......:*)" | rex max_match=0 field=DeniedRules "(?P<DeniedRule>......_*)" | mvexpand DeniedRule | fields - DeniedString

But when I put the regex separately, it works, when I put the regex together, both doesn't works.

What do you think ?

[rafamss]

Hi sbsbb,

Indeed : is the correct, has error of typing, sorry!

[sbsbb]

Of course you input is different, than the example... you have ? in place of :

(?Prule_.)|(?P.)\?(?P.)|(?P.)\?(?P.*)"

[rafamss]

Hi sbsbb,

I tried this solution, but without sucess. Take a look in this line of the log. I tried to use that refex that you passed but don't work. The format of log is access_combined_wcookie

192.168.0.1 - - [28/Nov/2013:02:09:24 +0000] "GET /127.0.0.1/icon.png HTTP/1.1" 200 3288 "-" "Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0" "-" "rule_1234|098765?098765|123456?123456"

[somesoni2]

"Remembering that segmented rule can be repeat more times, and each rule must be in fields separated".
Can you provide some example of this as well?