multiline log regex and field help

[adepasquale]

I took a look at quite a few of the threads on here to solve my problem first, but mine seems to be a little more unique than what already exists.

Consider the following log entry:

--------------------------------------------------------------- 
04/13/17 08:50:51 RELEASE by 192.168.1.2 to www and M4 : 
/some/horrible/code/pushedbyprogrammer/code1.r 
/some/horrible/code/pushedbyprogrammer/code2.r 
/some/horrible/code/pushedbyprogrammer/something.pdf
/some/horrible/code/pushedbyprogrammer/image.jpg
/some/horrible/code/pushedbyprogrammer/code3.p  
Just Released 5 programs

This is my regex:

^\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\s+\d+/\d+/\d+\s+\d+:\d+:\d+\s+\w+\s+\w+\s+(?P<codePusher>[^ ]+) to (?P<serverPushed>[^:]+) :\s+(?P<codePushed>[A-z0-9\s\/.]+)Just Released (?P<numberPushed>[\d]+) program

These are the fields I get:

codePusher: "192.168.1.2"
serverPushed: "www and M4"
codePushed: /some/horrible/code/pushedbyprogrammer/code1.r /some/horrible/code/pushedbyprogrammer/code2.r /some/horrible/code/pushedbyprogrammer/something.pdf /some/horrible/code/pushedbyprogrammer/image.jpg /some/horrible/code/pushedbyprogrammer/code3.p  
numberPushed: 5

My problem is the "codePushed" in that I'm trying to build an event that only looks for extensions ending in .p or .r.

at first i tried (codePushed="*.r * OR codePushed="*.p *") but unfortunately, the space/carriage return is not recognized and i get nothing. So i tried something like (codePushed="*.r* OR codePushed="*.p*") with no spaces and this works for the most part, but then it also picks up things like ".pdf" which i do not want. My other solution was to filter out what i don't want but in the case above, if i say codePushed!="*.pdf*" then it doesn't give me anything because the entire field is treated as a single line which brings me to my next point.

The other issue i have is that it treats all of the pieces of code pushed as one field. Is there anyways to split this up into an array?? I'm not really sure how to approach this problem.

[somesoni2 · Accepted solution]

You can reduce your regular expression like this

^\-+(\s+\S+){4}\s+(?P<codePusher>[^ ]+) to (?P<serverPushed>[^:]+) :\s+(?P<codePushed>[A-z0-9\s\/.]+)Just Released (?P<numberPushed>[\d]+) program

And try this. This should filter codePushed field values to keep only entries with extensions .r or .p.

your base search | makemv codePushed | eval codePushed=mvfilter(match(codePushed,"\.p$") OR match(codePushed,"\.r$")

[adepasquale]

This looks like an awesome solution so far. I had to add a closing ")" but it still seems to show me things like .php and .png, etc...

sourcetype="ReleaseLog" 
| makemv codePushed 
| eval codePushed=mvfilter(match(codePushed,"\.p$") OR match(codePushed,"\.r$"))

I still see output like this:

---------------------------------------------------------------
04/13/17  12:18:05
RELEASE by 192.168.1.2 to www and M4 : 
   /some/code/someimage.jpg
Just Released 1 programs

---------------------------------------------------------------
04/13/17  12:17:54
RELEASE by 192.168.1.5 to staging : 
   /some/code/someimage.jpg
Just Released 1 programs

---------------------------------------------------------------
04/13/17  12:16:32
RELEASE by 192.168.1.3 to staging : 
   /some/code/someimage.jpg
Just Released 1 programs

What's interesting though is that even though the log entries show up in the search... the codePushed field is only present when it's .p or .r, how would i prevent the rest from even being included in the search? Just add: | search codePushed="*.r" or codePushed="*.p" ?

[adepasquale]

actually just added |search codePushed!="" and this works