Splunk Search
Highlighted

Extract information via regex

Contributor

Hi guys,

I need some help to split the field below:

xyu_0987|123456:123456|123456:123456,

before the first pipeline(xyz_9807) displays the information about one rule;
after the first pipeline(|93830:09383) displays the information about alerts rules;
after the second pipeline(|093732:827332) displays information about denied rules.

The second point is that each rule is segmented by " : ", how i split the fields in order to keep this format: Remembering that segmented rule can be repeat more times, and each rule must be in fields separated.

Rule       Alerts Rule              Denied Rule 
           Rule 1 : Rule 2          Rule 1 : Rule 2
xyu_0987   123456 : 123456          123456 : 123456

Can you help me ?

Regards,

Rafael Martins

Highlighted

Re: Extract information via regex

SplunkTrust
SplunkTrust

"Remembering that segmented rule can be repeat more times, and each rule must be in fields separated".
Can you provide some example of this as well?

0 Karma
Highlighted

Re: Extract information via regex

Builder

For making my tries with splunk regex, I use the program Kodos, that use the same synthax...

I would suggest you something like (I'm not good at regex, but mostly achieve what I want 😉

| rex max_match=0 "(?P<Rule>.*)\|(?P<Alert1>.*):(?P<Alert2>.*)\|(?P<Denied1>.*):(?P<Denied2>.*)"

If the number of Rules between the pipes is unknown, than I would try first to extract AlertString, DeniedString, and then extract the Rules as Multivalue Field you could mvexpand :

| rex max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)
| rex max_match=0 field=AlertString "(?P<AlertRules>[regex to split Rules])"
| rex max_match=0 field=AlertString "(?P<DeniedRules>[regex to split Rules])"
| mvexpand AlertRules | fields - AlertString
| mvexpand DeniedRules | fields - DeniedString
| table *

View solution in original post

Highlighted

Re: Extract information via regex

Contributor

Hi sbsbb,

I tried this solution, but without sucess. Take a look in this line of the log. I tried to use that refex that you passed but don't work. The format of log is accesscombinedwcookie

192.168.0.1 - - [28/Nov/2013:02:09:24 +0000] "GET /127.0.0.1/icon.png HTTP/1.1" 200 3288 "-" "Mozilla/5.0 (Windows NT 6.1; rv:25.0) Gecko/20100101 Firefox/25.0" "-" "rule_1234|098765?098765|123456?123456"

0 Karma
Highlighted

Re: Extract information via regex

Builder

Of course you input is different, than the example... you have ? in place of :

(?Prule_.)|(?P.)\?(?P.)|(?P.)\?(?P.*)"

0 Karma
Highlighted

Re: Extract information via regex

Contributor

Hi sbsbb,

Indeed : is the correct, has error of typing, sorry!

0 Karma
Highlighted

Re: Extract information via regex

Contributor

Hi sbsbb,

I tried to use your code, but in some cases the regex don't work correctly, so I changed the regex to below:

| rex max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)" | rex max_match=0 field=AlertString "(?P<AlertRules>......:*)" | rex max_match=0 field=AlertString "(?P<AlertRules>......:*)" | rex max_match=0 field=AlertRules "(?P<AlertRule>......_*)" | mvexpand AlertRule | fields - AlertString | rex max_match=0 field=DeniedString "(?P<DeniedRules>......:*)" | rex max_match=0 field=DeniedRules "(?P<DeniedRule>......_*)" | mvexpand DeniedRule | fields - DeniedString

But when I put the regex separately, it works, when I put the regex together, both doesn't works.

What do you think ?

0 Karma
Highlighted

Re: Extract information via regex

Contributor

Hi sbsbb,

Your solution code worked fine, but I had to add some pieces. Please look my question below.

0 Karma
Highlighted

Re: Extract information via regex

Contributor

Sure somesoni2.

Example:
For the first pipeline, we have |987654:1098762:098675, in this case, we have three distinct values for alert rules.
For the second pipeline, we have |092822:098271, in this case, we have two distinct values for denied rules.

Do you understand ?

Regards,

Rafael Martins

0 Karma
Highlighted

Re: Extract information via regex

SplunkTrust
SplunkTrust

If you have already extracted the field from your log which has rules information, say ruleInfo, the following works.

your search with ruleInfo extracted|rex field=ruleInfo max_match=0 "(?P<Rule>.*)\|(?P<AlertString>.*)\|(?P<DeniedString>.*)" | rex max_match=0 field=AlertString "(?P<AlertRules>[^:]+)" | rex max_match=0 field=DeniedString "(?P<DeniedRules>[^:]+)" | table Rule, AlertRules, DeniedRules | mvexpand AlertRules | mvexpand DeniedRules | mvcombine delim=":"  AlertRules | mvcombine delim=":" DeniedRules | nomv DeniedRules

The output is like below:-

Rule          AlertRules                     DeniedRules     
gijo_0987     10856:123456                   123326:133456                     
sms_0987      123456:147556:1237776:28856    10056:126656 
0 Karma