topic Re: Need help with regex in Splunk Search

Need help with regex

amithhegde — Fri, 09 Jan 2015 12:00:09 GMT

I am trying to extract different error messages out of raw server log events. Below are the examples of different types of errors i am trying to extract.

ClassNotFoundException

java.io.IOException: No space left on device

Caused by: java.net.SocketException: Connection reset

Error - ORA-00942: table or view does not exist

java.lang.OutOfMemoryError: GC overhead limit exceeded

db connection hold time exceeds threshold [60000] ms

Caused by: java.sql.SQLRecoverableException: IO Error

java.lang.OutOfMemoryError

java.sql.SQLRecoverableException: Closed Connection

No ManagedConnections available within

Regex for ClassNotFoundException is something I got working. But I am having difficulty with others since they are having white space,colons,periods etc and I am no expert with regular expressions :(.

Any help/suggestions answers will be greatly appreciated.

Regards,
Amith

Re: Need help with regex

amithhegde — Fri, 09 Jan 2015 12:01:40 GMT

rex field=_raw "\b(?[\w]+assNotFoundException)" is regex i have written for ClassNotFoundException

Re: Need help with regex

gfuente — Fri, 09 Jan 2015 12:06:47 GMT

Hello

You should post full events samples, as the boundaries are important to define a proper regex

Regards

Re: Need help with regex

amithhegde — Fri, 09 Jan 2015 12:34:48 GMT

No ManagedConnections available Error 

21:58:17,552 ERROR [STDERR] [10.10.70.11] Caused by: org.jboss.util.NestedSQLException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] ); - nested throwable: (javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] ))

 21:58:17,479 ERROR [STDERR] [10.10.70.11] Caused by: javax.ejb.EJBException: org.jboss.util.NestedSQLException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] ); - nested throwable: (javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] ))

 21:58:12,629 ERROR [STDERR] [10.10.70.11] Caused by: javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] )

Re: Need help with regex

amithhegde — Fri, 09 Jan 2015 12:37:09 GMT

java.io.IOException: No space left on device error.

10:50:50.000 AM 
10:50:50,246 ERROR [JBossWeb] Problem in init 
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:310)
at org.jboss.util.file.JarUtils.unjar(JarUtils.java:313)
at org.jboss.web.AbstractWebContainer.init(AbstractWebContainer.java:325)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

Re: Need help with regex

richgalloway — Fri, 09 Jan 2015 12:41:35 GMT

What data are you trying to get from the events? The desired level of detail will have an effect on the regex.

Re: Need help with regex

lcrielaa — Fri, 09 Jan 2015 12:54:05 GMT

Could you give a few examples of log files (pastebin?) and what exactly it is you'd like to have extracted?

Re: Need help with regex

FritzWittwer_ol — Fri, 09 Jan 2015 13:20:20 GMT

Just in general, while working with regex I find the following two resources very helpful

A regex tester on www.regex101.com and a tutorial on www.rexegg.com

Re: Need help with regex

richgalloway — Fri, 09 Jan 2015 13:32:21 GMT

I would add www.regexr.com to that list.

Re: Need help with regex

kendrickt — Fri, 09 Jan 2015 13:32:56 GMT

Some very basic regex you can run in the search box:

This is by NO means the best regex for your data, just an example:

index=yourindex | rex ".+init\s+(?<Call_me_whatever_you_want>.+)\s+at"

So the call_me_whatever_you_want will be your field name.

It's important to understand a bit about regex and how Splunk uses it at search time:

rex is the command you will to tell Splunk you want to use regex on the data that comes into that pipe.

.+init is saying find everything (The .+) up to the word init. followed by whitespaces \s+

(? this is when you start to tell splunk that after the init, you want to make a field.

.+) After the > in the above, type what you want to see in that field. Work out the regex which covers everyone of your errors. You can see I've just typed .+ again this will highlight EVERYTHING after the init. ") just tells splunk your finished with that field.

Now you need to tell splunk what appears after the data that you want. I have done it like this: \s+at" because that particular java IO error is ended by some whitespaces /s+ followed by the word at.

Hopefully this will give you a basic idea of how to use it. Also try resources like http://www.regex101.com as this is a great sandbox for testing what you write(It will not recognise the Splunk fields though "(?)".

Alternatively, try the Splunk build in field extractor - that's quite handy and it does it for you!

Hope this helps

Re: Need help with regex

amithhegde — Fri, 09 Jan 2015 13:40:51 GMT

Hi Icrielaa,

I have provided the snippet of logs in the comments, what I want to extract is the error messages of the events "No ManagedConnections available within configured blocking timeout" and "java.io.IOException: No space left on device".

Thanks,
Amith

Re: Need help with regex

aljohnson_splun — Fri, 09 Jan 2015 17:06:22 GMT

@richgalloway One thing to be careful of here is that regexr.com uses Javascript regex whereas regex101 lets you choose your flavor; the differences may be nuanced but there it is worth it to use a tool that also uses perl compatible regular expressions (PCRE).