topic Re: Regex match till end of event? in Splunk Search

Regex match till end of event?

Cuyose — Wed, 10 Aug 2016 03:43:24 GMT

Not sure why I cant find this, but the following is not working.

|rex field=_raw "(?i)response=(?<responseXML>.+)$"

where response= occurs somewhere in the event and always continues to the very end of a multi lined event.

Re: Regex match till end of event?

inventsekar — Wed, 10 Aug 2016 08:37:15 GMT

"(?i)response=(?.+)$" -----

- you should use < and > around the variable - (?<i>) 
- (?<i>) needs to come where it will appear on the event (ie, after the "response=")

and i created few sample events ending with "response=digits"

event 1 - Extract "from" and "to" fields using regular expressions. response=101
event 2 - If a raw event contains "From: Susan To: Bob", then from=Susan and to=Bob, response=404
event 3 - source="tutorialdata.zip:./www1/access.log" response=500

and this query picks up the response codes fine.

sourcetype=responseREX | rex field=_raw "response=(?<i>.*)" | table _raw, i

regarding the end of line $, these below two works same -
response=(?.)
and
response=(?.)$

Re: Regex match till end of event?

sundareshr — Wed, 10 Aug 2016 13:18:46 GMT

Try this

| rex field=_raw "response=(?<msg>[^\t\n]+)"

Re: Regex match till end of event?

Cuyose — Wed, 10 Aug 2016 14:27:41 GMT

Sorry, I had that, but I must have missed the code button and it stripped out some things.

This is not working

|rex field=_raw "(?m)Data=(?<xmlData>.+)$"

Re: Regex match till end of event?

Cuyose — Wed, 10 Aug 2016 14:29:31 GMT

this does not work because it only captures to the end of the current line the response= is found in.

Re: Regex match till end of event?

Cuyose — Wed, 10 Aug 2016 14:31:32 GMT

This only captured the first character after response=

Re: Regex match till end of event?

dbcase — Wed, 10 Aug 2016 14:59:13 GMT

Have you tried erex?

http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Erex

Using erex, Splunk will attempt to write the rex pattern for you given your example string(s).

Re: Regex match till end of event?

somesoni2 — Wed, 10 Aug 2016 15:09:11 GMT

Give this a try

 | rex field=_raw "response=(?<responseXML>[\S\s\r\n]*)$"

Option 2

 your base search | eval responseXML=replace(_raw,"^([\S\s\r\n]*)response=","")

Re: Regex match till end of event?

Cuyose — Wed, 10 Aug 2016 15:13:07 GMT

Unfortunately, while useful for smaller more specific examples, this cannot take multiple 1000 character examples across multiple lines as input.

Re: Regex match till end of event?

Richfez — Wed, 10 Aug 2016 15:16:29 GMT

Do you have a sample of one of the difficult, long, multi-line event that we can use for confirmation of potential solutions before posting them?

Re: Regex match till end of event?

inventsekar — Wed, 10 Aug 2016 16:27:33 GMT

we are trying to create the rex query with just our own understanding of your issue.
could you please update us an event, and your current query, please.

Re: Regex match till end of event?

sundareshr — Wed, 10 Aug 2016 17:34:00 GMT

Try this

| rex field=_raw "response=(?<msg>[^\S\t\r\n]+)"

Re: Regex match till end of event?

Cuyose — Wed, 10 Aug 2016 19:19:03 GMT

Cool, this worked(Option 1), but its very odd that I had to resort to this, as the first example I tried had worked in many other situations!