topic Re: Avoiding duplicate text in eventtype in Splunk Search

Avoiding duplicate text in eventtype

glennh — Tue, 12 Jul 2011 07:42:09 GMT

Hi folks,

I'm trying to create an eventtype to match ERROR in my tomcat logs. The log messages for a single service call with an error in my log may contain multiple instances of "error" or "ERROR". I only want to count each service call error once.

here is an example from the log:
2011/06/28 08:29:54.552 ERROR getAvailableProductsListenerContainer-1 ExchangeDaoImpl - Error accessing database for query: getExchange

I thought of using a regex to include the timestamp and the ERROR string but I cannot find instructions for an eventype with a regex. Is this possible? What is the standard approach for this type of situation?

thanks

Glenn

Re: Avoiding duplicate text in eventtype

hazekamp — Wed, 13 Jul 2011 22:59:27 GMT

glennh,

You can use the eventtype to identify the errors themselves using search text, but the deduplication you are looking for is achieved by extracting a field and using the dedup search command.

This would look like (keeping in mind my regex will only work with the message you included):

## eventtypes.conf
[tomcat_errors]
search = sourcetype=tomcat ERROR

## props.conf
[tomcat]
REPORT-error_msg_for_tomcat = error_msg_for_tomcat

## transforms.conf
[error_msg_for_tomcat]
REGEX = \s-\s(Error.*)
FORMAT = error_msg::$1

## search
search = eventtype=tomcat_errors | dedup error_msg