Hi,
I have a catalina.out logfile that starts with one timestamp (YYYY-MM-DD HH:MM:SS), and then, after the initialization is complete, starts using a different one (MM.DD.YYYY HH:MM:SS). The customer needs all events, so I can't filter out the startup. How would this get handled?
2014-11-02 04:15:52 [cf6fb877] info [native] dynaTrace Bootstrap Agent 5.5.0.5226
2014-11-02 04:15:52 [cf6fb877] info [native] Using /fidcn/install/dynatrace-5.5.0 as Agent storage
2014-11-02 04:15:52 [cf6fb877] info [native] Log file at /fidcn/install/dynatrace-5.5.0/log/dt_CTG_FidelityCentral_prod_bootstra
p_4145.log
2014-11-02 04:15:52 [cf6fb877] info [native] Trying to connect to dtcoll-ctgprod-mmk.fmr.com:9998 for up to 60 seconds
2014-11-02 04:15:52 [cf6fb877] info [native] Connected successfully
...
03.11.2014 06:22:45 ERROR [org.apache.jetspeed.container.JetspeedContainerServlet.doGet():350] JetspeedContainerServlet: Error rendering portlet "fcapps::FCStats": javax.portlet.PortletException: Request processing failed
javax.portlet.PortletException: Request processing failed
at org.springframework.web.portlet.FrameworkPortlet.processRequest(FrameworkPortlet.java:544)
at org.springframework.web.portlet.FrameworkPortlet.doDispatch(FrameworkPortlet.java:470)
at javax.portlet.GenericPortlet.render(GenericPortlet.java:253)
at org.apache.jetspeed.portlet.PortletObjectProxy.proxyRender(PortletObjectProxy.java:280)
at org.apache.jetspeed.portlet.PortletObjectProxy.invoke(PortletObjectProxy.java:172)
at $Proxy179.render(Unknown Source)
at org.apache.jetspeed.factory.JetspeedPortletInstance.render(JetspeedPortletInstance.java:117)
at org.apache.jetspeed.container.services.JetspeedFilterChain.doFilter(JetspeedFilterChain.java:142)
at org.apache.jetspeed.container.services.JetspeedFilterChain.processFilter(JetspeedFilterChain.java:92)
at org.apache.jetspeed.container.services.JetspeedFilterManager.processFilter(JetspeedFilterManager.java:112)
at org.apache.jetspeed.container.JetspeedContainerServlet.doGet(JetspeedContainerServlet.java:290)
This is not too difficult (after I figured it out) with 4 different timestamps. I handled it in my installation using the following in the props.conf file:
[tomcat_catalina]
DATETIME_CONFIG = /etc/slave-apps/all_tomcat_props/local/catalina.xml
LINE_BREAKER = ^(?:(?:\w+\s*\d{1,2},\s*\d{4}\s*\d{1,2}:\d{2}:\d{2}\s*\w+)|(?:\[\w{1,7}\]\s\d\d/\d\d/\d\d\s)|(?:\d{4}-\d{2}-\d{2}\s\d{1,2}:\d{2}:\d{2},\d{3})|(?:\[\d\d/\d\d/\d\d\s\d\d:\d\d:\d\d[:,.]\d\d\d\]))
TRUNCATE = 100000
NO_BINARY_CHECK=1
MAX_TIMESTAMP_LOOKAHEAD=200
# Added transform to remove date from sourcename 20140324
TRANSFORMS-tcatdatestrip = tcat_strip_dates
Then I use the following catalina.xml file:
<datetime>
<define name="_mydatetimeformat4" extract="month, day, year, hour, minute, second, subsecond">
<text>\[(\d\d)/(\d\d)/(\d\d)\s+(\d\d):(\d\d):(\d\d)[:,.](\d\d\d)\]</text>
</define>
<define name="_mydatetimeformat3" extract="year, month, day, hour, minute, second, subsecond">
<text>(\d{4})-(\d{2})-(\d{2})\s(\d{1,2}):(\d{2}):(\d{2}).(\d{3})</text>
</define>
<define name="_mydatetimeformat2" extract="year, month, day, hour, minute, second, subsecond">
<text>(\d\d)/(\d\d)/(\d\d)\s+(\d\d):(\d\d):(\d\d)[:,.](\d\d\d)</text>
</define>
<define name="_mydatetimeformat1" extract="litmonth, day, year, hour, minute, second, ampm">
<text>(\w+)\s*(\d{1,2}),\s*(\d{4})\s*(\d{1,2}):(\d{2}):(\d{2})\s*(\w+)</text>
</define>
<timePatterns>
<use name="_mydatetimeformat4"/>
<use name="_mydatetimeformat1"/>
<use name="_mydatetimeformat2"/>
<use name="_mydatetimeformat3"/>
</timePatterns>
<datePatterns>
<use name="_mydatetimeformat4"/>
<use name="_mydatetimeformat1"/>
<use name="_mydatetimeformat2"/>
<use name="_mydatetimeformat3"/>
</datePatterns>
</datetime>
And for completeness, the transforms.conf file entry:
[tcat_strip_dates]
# Remove YYYY-MM-DD style date from the filename
# /opt/TcatServer*/logs/namestuff.2014-02-13.log
#
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = (/\w{3}/\S+/\w{4}/\w+)\.\d{4}-\d{2}-\d{2}(.log)
FORMAT = source::$1$2
If you don't have the same filenaming problem that I did (the dates in the filename), you can not to the transforms stuff in the props and transforms files.
This is not too difficult (after I figured it out) with 4 different timestamps. I handled it in my installation using the following in the props.conf file:
[tomcat_catalina]
DATETIME_CONFIG = /etc/slave-apps/all_tomcat_props/local/catalina.xml
LINE_BREAKER = ^(?:(?:\w+\s*\d{1,2},\s*\d{4}\s*\d{1,2}:\d{2}:\d{2}\s*\w+)|(?:\[\w{1,7}\]\s\d\d/\d\d/\d\d\s)|(?:\d{4}-\d{2}-\d{2}\s\d{1,2}:\d{2}:\d{2},\d{3})|(?:\[\d\d/\d\d/\d\d\s\d\d:\d\d:\d\d[:,.]\d\d\d\]))
TRUNCATE = 100000
NO_BINARY_CHECK=1
MAX_TIMESTAMP_LOOKAHEAD=200
# Added transform to remove date from sourcename 20140324
TRANSFORMS-tcatdatestrip = tcat_strip_dates
Then I use the following catalina.xml file:
<datetime>
<define name="_mydatetimeformat4" extract="month, day, year, hour, minute, second, subsecond">
<text>\[(\d\d)/(\d\d)/(\d\d)\s+(\d\d):(\d\d):(\d\d)[:,.](\d\d\d)\]</text>
</define>
<define name="_mydatetimeformat3" extract="year, month, day, hour, minute, second, subsecond">
<text>(\d{4})-(\d{2})-(\d{2})\s(\d{1,2}):(\d{2}):(\d{2}).(\d{3})</text>
</define>
<define name="_mydatetimeformat2" extract="year, month, day, hour, minute, second, subsecond">
<text>(\d\d)/(\d\d)/(\d\d)\s+(\d\d):(\d\d):(\d\d)[:,.](\d\d\d)</text>
</define>
<define name="_mydatetimeformat1" extract="litmonth, day, year, hour, minute, second, ampm">
<text>(\w+)\s*(\d{1,2}),\s*(\d{4})\s*(\d{1,2}):(\d{2}):(\d{2})\s*(\w+)</text>
</define>
<timePatterns>
<use name="_mydatetimeformat4"/>
<use name="_mydatetimeformat1"/>
<use name="_mydatetimeformat2"/>
<use name="_mydatetimeformat3"/>
</timePatterns>
<datePatterns>
<use name="_mydatetimeformat4"/>
<use name="_mydatetimeformat1"/>
<use name="_mydatetimeformat2"/>
<use name="_mydatetimeformat3"/>
</datePatterns>
</datetime>
And for completeness, the transforms.conf file entry:
[tcat_strip_dates]
# Remove YYYY-MM-DD style date from the filename
# /opt/TcatServer*/logs/namestuff.2014-02-13.log
#
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = (/\w{3}/\S+/\w{4}/\w+)\.\d{4}-\d{2}-\d{2}(.log)
FORMAT = source::$1$2
If you don't have the same filenaming problem that I did (the dates in the filename), you can not to the transforms stuff in the props and transforms files.
You have an open `` in the xml:
<datetime>
<!-- 2014-06-11 11:08:50 [4d313890] info [native] dynaTrace Bootstrap Agent 5.5.0.5226 -->
<define name="_datetimeformat1" extract="year, month, day, hour, minute, second">
<text>(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2})</text>
<?-- Right here is the extra <text>: -->
<text>
</define>
<!-- 11.06.2014 11:12:47 ERROR [org.springframework.web.portlet.FrameworkPortlet.processRequest():559] Could not complete request -->
<define name="_datetimeformat2" extract "day, month, year, hour, minute, second">
<text>(\d{2})\.(\d{2})\.(\d{4})\s(\d{2}):(\d{2}):(\d{2})</text>
</define>
<timePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</timePatterns>
<datePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</datePatterns>
</datetime>
That could be your problem. Not sure about that though. Try it and let us know.
Looks like it didn't put the <text> in my comment previously. Where is has "You have an extra `in the xml," insert the
text` tag between the quotes.
This is essentially what I've done, but I get that strange error. Am I missing something?
I found this blog entry, and gave it whirl, but no luck.
http://www.function1.com/2013/01/oh-no-splunking-log-files-with-multiple-formats-no-problem
Here's my new datetime.xml file:
<datetime>
<!-- 2014-06-11 11:08:50 [4d313890] info [native] dynaTrace Bootstrap Agent 5.5.0.5226 -->
<define name="_datetimeformat1" extract="year, month, day, hour, minute, second">
<text>(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2})</text>
<text>
</define>
<!-- 11.06.2014 11:12:47 ERROR [org.springframework.web.portlet.FrameworkPortlet.processRequest():559] Could not complete request -->
<define name="_datetimeformat2" extract "day, month, year, hour, minute, second">
<text>(\d{2})\.(\d{2})\.(\d{4})\s(\d{2}):(\d{2}):(\d{2})</text>
</define>
<timePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</timePatterns>
<datePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</datePatterns>
</datetime>
Here's my props.conf, using the data preview:
DATETIME_CONFIG = /etc/apps/SVB/datetime.xml
LINE_BREAKER LINE_BREAKER = ([\r\n]+)\d{4}\d{2}\d{2}\s\d{d}:\d{2}:\d{2([\r\n]+)\d{2}.\d{2}.\d{4}\s\d{2}:\d{2}:\d{2}
SHOULD_LINEMERGE=false
When I try it, I get:
Generated an empty preview: check your line breaking and line merging settings; Uncaught exception in Aggregator, skipping an event: Error parsing regex XML file: /apps/splunk/etc/apps/SVB/datetime.xml - Couldn't find 'timePatterns' in config data for AggregatorProcessor.