Getting Data In

How to edit my props.conf for proper line breaking of a large event by the ∑ character?

LiquidTension
Path Finder

I am having trouble with being able to properly line break an event like the following:

Here are the props I am using

LINE_BREAKER = (\∑)
SHOULD_LINEMERGE = false
TIME_PREFIX = <6>
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y-%m-%dT%H%M%SZ
category = Custom

Obviously, it's a very simple regex which works in sites like regex101, but when it is enabled in Splunk, simply searching for "ApiGatewaySlimController" no longer produces results. If I comment the props out the events then show up. This data is coming in as syslog if that matters.

<6>2016-07-29T17:14:50Z p4cdi9ht1eb doppler[19]: time="2016-07-29T17:14:50Z" level=info msg="2016-07-29 17:14:30,829 hitId=unknown class=o.apache.juli.logging.DirectJDKLog  logLevel=ERROR thread=http-nio-8080-exec-10 Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.RuntimeException: Thrown Exception] with root cause java.lang.RuntimeException: Thrown Exception∑\tat com.mrll.javelin.apigateway.controller.ApiGatewaySlimController.danStack(ApiGatewaySlimController.java:21) ~[app/:na]∑\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91]∑\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_91]∑\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]∑\tat java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]∑\tat org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:222) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:814) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:737) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:969) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:860) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:622) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:845) ~[spring-webmvc-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:729) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:243) ~[spring-boot-actuator-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:111) ~[spring-boot-actuator-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:207) ~[spring-security-web-4.0.3.RELEASE.jar!/:4.0.3.RELEASE]∑\tat org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176) ~[spring-security-web-4.0.3.RELEASE.jar!/:4.0.3.RELEASE]∑\tat org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:103) ~[spring-boot-actuator-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]∑\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]∑\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) ~[tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:676) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1096) [tomcat-embed-core-8.0.30.jar!/:8.0.30]∑\tat org.apache.coyote.AbstractProtocol$Abs
0 Karma

landen99
Motivator

The key lines that you need are:

LINE_BREAKER = ([^\x00-\x7F])
TRUNCATE = 0
0 Karma

Masa
Splunk Employee
Splunk Employee

It seems like you're trying to replace ∑ with a new line character to reformat a single event. If that's the case, LINE_BREAKER will not help.
I suggest to use SEDCMD attribute.
in props.conf

[_user_stanza_]
SHOULD_LINEMERGE = false
TIME_PREFIX = <6>
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y-%m-%dT%H%M%SZ
SEDCMD-sigma = s/∑/\n/g

Otherwise, you will have problem of parsing events without specifying line merge configuration.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Thanks for the Memories! Splunk University, .conf25, and our Community

Thank you to everyone in the Splunk Community who joined us for .conf25, which kicked off with our iconic ...