topic Re: SEDCMD pattern replacement not working in Getting Data In

SEDCMD pattern replacement not working

willthames — Thu, 17 Feb 2011 01:31:01 GMT

My props.conf is as follows. The SEDCMDs seem to be very temperamental

[server]
MAX_TIMESTAMP_LOOKAHEAD = 0
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3},
TIME_FORMAT = %Y-%m-%d %H:%M:%S
SEDCMD-exception-raised-ignore = s/(Exception raised [^:]*:).*$/\1 INFO REMOVED)/g
SEDCMD-remove-extra-timestamp = s/(ERROR \[STDERR\]) \d{2}-\w{3}-\d{4} \d{2}:\d{2}:\d{2}/\1 DATE REMOVED/g
REPORT-server = jboss-server-extractions

And then I get (some values changed)

2011-02-16 16:11:24,336, ERROR [STDERR] com.company.Exception: attempt to retrieve money as wrong currency.
    (Exception raised at address: server8.production/10.0.0.8); 
    (Exception raised on date: INFO REMOVED)

So the SEDCMD-exception-raised-ignore is working on the second 'Exception raised' line but not the first. Earlier today I was struggling to get it to work at all, and even now, other very similar results aren't being processed:

2011-02-16 17:23:35,882, ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/].[zzzaction]] Servlet.service() for servlet zzzaction threw exception
com.company.Exception: com.company.Exception: attempt to retrieve money as wrong currency.  
    (Exception raised at address: server3.production/10.0.0.3); 
    (Exception raised on date: Wed Feb 16 17:23:35 UTC 2011);; 
    (Exception raised at address: server3.production/10.0.0.3); 
    (Exception raised on date: Wed Feb 16 17:23:35 UTC 2011);

Any help in getting to the bottom of why this is so temperamental and how to make it work more regularly would be gratefully received!

Thanks!

Re: SEDCMD pattern replacement not working

jrodman — Thu, 17 Feb 2011 01:56:25 GMT

This makes me wonder about sedcmd with repeat match behavior, and multiline events. Hopefully will come back with something useful later.

Re: SEDCMD pattern replacement not working

Ron_Naken — Thu, 17 Feb 2011 02:50:28 GMT

I haven't run into any issue with SEDCMD. Your RegEx for SEDCMD-exception-raised-ignore will not work correctly. Try it like this:

SEDCMD-exception-raised-ignore = s/(Exception raised [^:]*:)\V*/\1 INFO REMOVED)/g

Re: SEDCMD pattern replacement not working

Ron_Naken — Thu, 17 Feb 2011 03:46:39 GMT

Note: This will only affect new data being indexed. Restart Splunk after making the change.

Re: SEDCMD pattern replacement not working

willthames — Thu, 17 Feb 2011 18:31:59 GMT

Ok, so that's because the SEDCMD is run against the event, not the line? I've given the \V a try and will see how it performs next time the event occurs.

Thanks for the answer, and good to know that SEDCMD is usually reliable!

Re: SEDCMD pattern replacement not working

willthames — Thu, 17 Feb 2011 20:39:24 GMT

Just checked the resultset now and it works a treat