Getting Data In

How to edit my props.conf for proper line breaking of a large event by the ∑ character?

Path Finder

I am having trouble with being able to properly line break an event like the following:

Here are the props I am using

LINE_BREAKER = (\∑)
SHOULD_LINEMERGE = false
TIME_PREFIX = <6>
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y-%m-%dT%H%M%SZ
category = Custom

Obviously, it's a very simple regex which works in sites like regex101, but when it is enabled in Splunk, simply searching for "ApiGatewaySlimController" no longer produces results. If I comment the props out the events then show up. This data is coming in as syslog if that matters.

<6>2016-07-29T17:14:50Z p4cdi9ht1eb doppler[19]: time="2016-07-29T17:14:50Z" level=info msg="2016-07-29 17:14:30,829 hitId=unknown class=o.apache.juli.logging.DirectJDKLog  logLevel=ERROR thread=http-nio-8080-exec-10 Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.RuntimeException: Thrown Exception] with root cause java.lang.RuntimeException: Thrown Exception∑\tat com.mrll.javelin.apigateway.controller.ApiGatewaySlimController.danStack(ApiGatewaySlimController.java:21) ~[app/:na]∑\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91]∑\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_91]∑\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]∑\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]∑\tat org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:222) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:814) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:737) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:969) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:860) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:622) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:845) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:729) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:243) ~[spring-boot-actuator-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111) ~[spring-boot-actuator-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:207) ~[spring-security-web-4.0.3.RELEASE.jar!/:4.0.3.RELEASE]∑\tat org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176) ~[spring-security-web-4.0.3.RELEASE.jar!/:4.0.3.RELEASE]∑\tat org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103) ~[spring-boot-actuator-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.coyote.AbstractProtocol$Abs
0 Karma

Motivator

The key lines that you need are:

LINE_BREAKER = ([^\x00-\x7F])
TRUNCATE = 0
0 Karma

Splunk Employee
Splunk Employee

It seems like you're trying to replace ∑ with a new line character to reformat a single event. If that's the case, LINE_BREAKER will not help.
I suggest to use SEDCMD attribute.
in props.conf

[_user_stanza_]
SHOULD_LINEMERGE = false
TIME_PREFIX = <6>
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y-%m-%dT%H%M%SZ
SEDCMD-sigma = s/∑/\n/g

Otherwise, you will have problem of parsing events without specifying line merge configuration.

0 Karma