topic Re: Specifying two regular expressions in a single search query using rex in Splunk Search

Specifying two regular expressions in a single search query using rex

ChhayaV — Wed, 14 Aug 2013 09:05:32 GMT

Hi,

I have SharePoint logs and in that there is a field called message.From the message field i have extracted exceptions using regular expressions.Here for extracting three exceptions i have used one regular expression and for other two i have used different regex.
The search queries used for those two different regex are :

host="sharepoint" | rex field=message "(?i)\b exception\b:\s(?P<FieldIdentifier>[^\)]+Exception)\:" | table FieldIdentifier
host="sharepoint" | rex field=message "(?<=[Errorlfailed]\: )(?P<FieldIdentifier1>[^\)]+Exception:)" |  table FieldIdentifier

Here i want to have single search query including both the regex.How can i write this in rex command .Tried with OR command but that din't worked.So pls help with the query.

Thank you

Re: Specifying two regular expressions in a single search query using rex

linu1988 — Wed, 14 Aug 2013 09:38:31 GMT

If the regex statements are matching the required field values, you can write it in a single statement.

host="sharepoint" | rex field=message "(?i)\b exception\b:\s(?P<Field1>[^\)]+Exception)\:"(?<=[Errorlfailed]\: )(?P<Field2>[^\)]+Exception:)" |  table Field1,Field2

You can also do it in the splunk UI field extraction window

Re: Specifying two regular expressions in a single search query using rex

kristian_kolb — Wed, 14 Aug 2013 10:37:36 GMT

This is correct. For an event like;

timestamp [blah] AAA:BBB:CCC DDD:EEE:FFF

where you wish to extract 'blah' and 'EEE', you can write a regex that will get them in one go;

...| rex "\[(?<first>[^\]]+)\]\s\S+\s\w:(?<second>\w+):\w+"

However some care might be required - if the messages in a log are formatted differently, so that for some events e.g. the second field can't be matched, the rex statement as a whole will fail, and neither first nor second will be extracted for that event.

Re: Specifying two regular expressions in a single search query using rex

ChhayaV — Wed, 14 Aug 2013 11:19:29 GMT

Hi linu1988,
I tried with the way you have suggested but its not working.Saying error in rex command.
Actually i tried with the pipe(|) command which is not showing any error in the regex(no desirable output).
Tried query is :

host="sharepoint" | rex field=message "(?<=[Errorlfailed]\: )(?P<Field1>[^\)]+Exception:)|(?i)\b exception\b:\s(?P<Field2>[^\)]+Exception)\:" | table Field1 Field2

Here its only displaying the result for Field1.Field2 values are coming blank.

Re: Specifying two regular expressions in a single search query using rex

ChhayaV — Wed, 14 Aug 2013 11:19:37 GMT

Also one more thing is i want Field1 and Field2 values to be captured in a single field i.e.,values of Field1 and Field2 should captured as a single field say "NewField"

Re: Specifying two regular expressions in a single search query using rex

Ayn — Wed, 14 Aug 2013 11:56:30 GMT

If you're going to use these extractions for anything else but just the odd search here and there you really should move them into props.conf / transforms.conf instead of having them inline in your searches.

Re: Specifying two regular expressions in a single search query using rex

linu1988 — Wed, 14 Aug 2013 13:30:23 GMT

Chhaya, if you are having problem with the search you can actually use the props.conf/transforms.conf as Ayn has suggested. I don't have the log with me so cant create the perfect matching.

You can also do |rex .... |rex ... |eval Newfield=field1." ".field2 OR directly get the fields and concatenate extracted from props.conf. Hope it clarifies..

Re: Specifying two regular expressions in a single search query using rex

kristian_kolb — Wed, 14 Aug 2013 14:04:44 GMT

The error is probably related to fact that you have a pipe character in the rex. Try to escape it with a backslash, otherwise it may be interpreted as part of the search query.

also, should it say 'Errorlfailed'? you don't want '(Error|Failed)'? Escape as needed.

Re: Specifying two regular expressions in a single search query using rex

ChhayaV — Fri, 16 Aug 2013 06:47:55 GMT

Hi linu1988,
i just wanted to do it through Splunk web.
Here i have provided the sample log entries
" http://answers.splunk.com/answers/98772/field-extraction-using-regex-command "

Re: Specifying two regular expressions in a single search query using rex

linu1988 — Fri, 16 Aug 2013 08:39:20 GMT

Hello ChhayaV,
The log doesn't have anything to match for the second regex so i can't try with it. The first regex matches the fields which you require.

If you want to do on UI, you can do it one field at a time after that save the field or ignore the matches which are not required.

Re: Specifying two regular expressions in a single search query using rex

ChhayaV — Fri, 16 Aug 2013 09:00:24 GMT

Hi,
Finally i came up with the working solution and the search query is as follows :
host="sharepoint" | rex field=message "(?i)\b exception\b:\s(?P[^)]+Exception):" |rename Field1 as output | append [search host="sharepoint" | rex field=message "(?<=[Errorlfailed]: )(?P[^)]+Exception:)" |
rename Field2 as output] | table output