Filtering the log using REGEX

[keshab]

I have following log. What will be the REGEX to index log containing line the line 'tomcat' trying to restart and send rest to nullQueue??

[CDT Oct 27 09:31:44] info : 'tomcat' trying to restart
[CDT Oct 27 09:31:44] info : 'tomcat' start: /sbin/service
[CDT Oct 27 09:33:44] info : 'tomcat' process is running with pid 20021
[CDT Oct 27 12:13:44] error : 'tomcat' process PID changed to 6908
[CDT Oct 27 12:15:44] info : 'tomcat' PID has not changed
[CDT Oct 27 14:30:04] info : unmonitor service 'tomcat' on user request
[CDT Oct 27 14:30:04] info : monit daemon at 13351 awakened
[CDT Oct 27 14:30:04] info : Awakened by User defined signal 1
[CDT Oct 27 14:30:15] info : restart service 'tomcat' on user request
[CDT Oct 27 14:30:15] info : monit daemon at 13351 awakened
[CDT Oct 27 14:30:15] info : Awakened by User defined signal 1
[CDT Oct 27 14:30:15] info : 'tomcat' trying to restart
[CDT Oct 27 14:30:15] info : 'tomcat' stop: /sbin/service
[CDT Oct 27 14:30:43] info : 'tomcat' start: /sbin/service
[CDT Oct 27 14:31:43] error : 'tomcat' failed to start
[CDT Oct 27 14:32:43] error : 'tomcat' process is not running
[CDT Oct 27 14:32:43] info : 'tomcat' trying to restart
[CDT Oct 27 14:32:43] info : 'tomcat' start: /sbin/service
[CDT Oct 27 14:32:44] info : 'tomcat' started

[tgow]

According to the docs if you want data to be sent to the null queue than you have to call your stanza in the transforms.conf setnull, ie:

props.conf

[yoursourcetype]
TRANSFORMS-set = setnull, setparsing

transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \'tomcat\' trying to restart
DEST_KEY = queue
FORMAT = indexQueue

[Ayn]

The transform doesn't have to be called "setnull", it's just what's used in the examples. You can call the transform anything, the only thing that matters is that it has any name so that there is something to refer to it by in props.conf.

[Ayn]

First define a nullqueue transform, then another one to override it in the specific case where the event contains "'tomcat' trying to restart". Have a look at this topic in the docs: http://docs.splunk.com/Documentation/Splunk/4.1.5/Admin/Routeandfilterdata#Keep_specific_events_and_...

This should work:

props.conf:

[yoursourcetype]
TRANSFORMS-set= defaultnull,tomcatrestart

transforms.conf:

[defaultnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[tomcatrestart]
REGEX = 'tomcat' trying to restart
DEST_KEY = queue
FORMAT = indexQueue