...so they wouldn't be useful in a count of error types. I'm trying to match for text that's common to repeating errors, and assign that text to a type, then count how many of each type there is over a given time range. ... View more

Thanks, Nick. A type is all the matched text. The 1st error matches: 2011-05-10 02:01:11,799 [ThreadPool worker thread #21] ERROR com.geodyne.overt.runtime.engine.FlowObjectExecutionTreeNode - deliverException(...)com.geodyne.magma.GeoException: Runtime error in script ("Process: 'GeoDoc GeoCode LAVA Cache' ProcessItem: 'Get GeoCode Location' Type: 'ITEM'" 18:0).Internal Script error: com.geodyneinc.magma.common.util.service.exceptions.BaseSystemException: Error while calling calderaLocation or parsing the response from GeoCode because of HTML response The details below that are unique... ... View more

Thanks for your response, Ron. I used index time because a response to my original question at: http://splunk-base.splunk.com/answers/24165/how-to-report-top-ten-errors-over-a-time-range suggested it was the only way to accomplish my goal. What I want to do is search through a log4j formatted file, gather all ERRORs, sort them by type (based on error text), count the instances of each error type, and return one example each of the 10 errors that repeat the most (top ten). Both index-time and search-time suggestions are returning log entries other than ERRORs, which isn't what I'm after. ... View more

Thanks so much for your quick response! Unfortunately, that didn't produce any usable results for me. Let me provide an example of what I'm looking for. Say I have an error in a log that looks like this, and it's repeated 300 times for a given time frame: 5/10/11 7:14:52.322 AM 2011-05-10 07:14:52,322 [asyncDelivery12] ERROR com.acme.klassified.business.ContractKlassifedImageImplementation - There are error(s) in setContractKlassifiedImage for the payloads. The Number of Payloads processed: total = 2 failed = 1. The parts of that error I care about from a grouping perspective are: ERROR com.acme.klassified.business.ContractKlassifedImageImplementation - There are error(s) in setContractKlassifiedImage for the payloads. I need the date/timestamps (5/10/11 7:14:52.322 AM 2011-05-10 07:14:52,322), things in braces ([asyncDelivery12]) and the entire "Number of payloads" line excluded, because they all have the potential to make an error message unique. Say that I have another error in the same log that looks like this, and is repeated 200 times in the same given timeframe: 5/10/11 2:00:11.694 AM 2011-05-10 02:00:11,694 [Thread-43] ERROR com.siriusforce.tools.server.activator.ToolsActivatorImpl - Unable to register GetJBossServerStackTrace MBean, exception message=com.siriusforce:service=ClusterStackTrace already registered. javax.management.InstanceAlreadyExistsException: com.siriusforce:service=ClusterStackTrace already registered. at org.jboss.mx.server.registry.BasicMBeanRegistry.add(BasicMBeanRegistry.java:761) at org.jboss.mx.server.registry.BasicMBeanRegistry.registerMBean(BasicMBeanRegistry.java:225) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155) at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94) at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133) at org.jboss.mx.server.Invocation.invoke(Invocation.java:88) at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142) at org.jboss.mx.server.Invocation.invoke(Invocation.java:88) at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264) at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659) at org.jboss.mx.server.MBeanServerImpl$3.run(MBeanServerImpl.java:1422) at org.jboss.mx.server.MBeanServerImpl.registerMBean(MBeanServerImpl.java:1417) at org.jboss.mx.server.MBeanServerImpl.registerMBean(MBeanServerImpl.java:376) at com.siriusforce.tools.server.activator.ToolsActivatorImpl.start(ToolsActivatorImpl.java:37) at com.siriusforce.plugin.server.ServerPluginInjector.invokePluginActivators(ServerPluginInjector.java:242) at com.siriusforce.plugin.server.ServerPluginInjector.start(ServerPluginInjector.java:134) at com.siriusforce.plugin.server.PluginSystemImpl.start(PluginSystemImpl.java:36) at com.siriusforce.plugin.server.ServerPluginSystemLoaderImpl$1.run(ServerPluginSystemLoaderImpl.java:61) The parts of that error I care about from a grouping perspective are: ERROR com.siriusforce.tools.server.activator.ToolsActivatorImpl - Unable to register GetJBossServerStackTrace MBean, exception message=com.siriusforce:service=ClusterStackTrace already registered. I need the date/timestamps (5/10/11 2:00:11.694 AM 2011-05-10 02:00:11,694), things in braces ([Thread-43]) and any line preceded by the word "at" excluded, because they all have the potential to make the error unique. Ultimately, I need to search for instances of the word "ERROR", and any error text excluding date/time stamps, things in braces, and stack trace elements, then group "like" messages together, count how many times each error shows up in a group, and produce a report that includes the totality of the first instance of each error: TOP-TEN Report for 5/9/11 13:00:00 to 5/10/11 06:00:00 The following error occurred 300 times: 5/10/11 7:14:52.322 AM 2011-05-10 07:14:52,322 [asyncDelivery12] ERROR com.acme.klassified.business.ContractKlassifedImageImplementation - There are error(s) in setContractKlassifiedImage for the payloads. The Number of Payloads processed: total = 2 failed = 1. The following error occurred 200 times: 5/10/11 2:00:11.694 AM 2011-05-10 02:00:11,694 [Thread-43] ERROR com.siriusforce.tools.server.activator.ToolsActivatorImpl - Unable to register GetJBossServerStackTrace MBean, exception message=com.siriusforce:service=ClusterStackTrace already registered. javax.management.InstanceAlreadyExistsException: com.siriusforce:service=ClusterStackTrace already registered. at org.jboss.mx.server.registry.BasicMBeanRegistry.add(BasicMBeanRegistry.java:761) at... -----------------------edit------------------------------- I think I understand what you're suggesting. You forgot to paste a link, but I believe the page you were referring to is: http://www.splunk.com/base/Documentation/4.2.1/Data/Configureindex-timefieldextraction I followed the directions as best I could and did the following, adding the entries at the bottom of the edited files: vi transforms.conf [topten] REGEX = ERROR.*?\n FORMAT = top_ten::"$1" WRITE_META = true vi props.conf [log4jlog] TRANSFORMS-topten = topten vi fields.conf [top_ten] INDEXED = true I restarted Splunk to pick up the changes. However, when I go to the search page, I don't see a new field in the left frame. If I do a search on a log and follow your suggestion to: | timechart count by top-ten It just lists the errors in reverse chronological order. It doesn't group them by error message content, or count how many times that content shows up in the logs. If I generate a report on the returned data, it the sourcetype isn't top_ten like I expected it to be. Can you explain where I'm going wrong? Thanks for your help. ... View more