topic Re: 2 different timestamps in single log in Getting Data In

2 different timestamps in single log

Starlette — Sun, 25 Apr 2010 17:52:31 GMT

Hai,

I have lines in single log (1 sourcetype) started with 2 different timestamps,

timestamp1 - etc etc
timestamp1 - etc etc
timestamp2 - etc etc
timestamp1 - etc etc
timestamp2 - etc etc
timestamp2 - etc etc

timestamp1 is picked up but timestamp2 one is merged onder timestamp1. So what the best way to config?

Cause It looks that the examples are leading to 1 TIME_FORMAT per sourcetype

Update with sample:

apr 23, 2010 15:46:28 PM (main) - Deploying module: metadataExchange-1.4 - file:/E:/win32app/SPECTRUM90/tomcat/webapps/axis2/WEB-INF/lib/mex-1.4.jar
apr 23, 2010 15:46:28 PM (main) - Deploying Web service: version.aar - file:/E:/win32app/SPECTRUM90/tomcat/webapps/axis2/WEB-INF/services/version.aar
apr 23, 2010 15:46:28 PM (main) - Deploying web application archive introscope-wssdk-consumer.war
apr 23, 2010 15:46:28 PM (main) - Initialized ConsumerStartupServlet
23-apr-2010 15:46:40 (EhealthIntegrationManager:ServerConfiguration) - EhealthMappingInfo
23-apr-2010 15:46:40 (PoolThread-1: GlobalPool => SLMStatWatcher.init) (SLM_STAT_DB) - SLM Initialization waiting on MySQL initialization.
23-apr-2010 15:46:40 (EhealthIntegrationManager:ServerConfiguration) - EhealthMappingInfo
23-apr-2010 15:46:40 (PoolThread-1: GlobalPool => SLMStatWatcher.init) (SLM_STAT_DB) - MySQL Initialization complete. SLM Initializing.
apr 23, 2010 15:46:40 PM (EhealthIntegrationManager:ServerConfiguration) - Commons Collections 3.x available
apr 23, 2010 15:46:40 PM (EhealthIntegrationManager:ServerConfiguration) - Loading XML bean definitions from dataAccessContext.xml
apr 23, 2010 15:46:41 PM (EhealthIntegrationManager:ServerConfiguration) - Schema change summary for "eHealth Integration"

I indeed discovered that the timestamp reconizing is a one to one action,,,so you mention that I have to split this log to different souretypes?

Re: 2 different timestamps in single log

gkanapathy — Sun, 25 Apr 2010 23:17:15 GMT

Configuration of timestamps in props.conf will only allow you to specify one timestamp format. You can simply not specify a timestamp format, and Splunk will try to match any of the many that are in its default config for guessing. This is easy to do, but it might be undesireable and result in items that are not timestamps being picked up too readily. However, if you can use this method (perhaps in conjuction with setting TIME_PREFIX, which looks to me like you can set ^, and MAX_TIMESTAMP_LOOKAHEAD, which should be set to the max length of the timestamps, then you might prefer this method.

Otherwise, you can do this creating a custom datetime.xml config file and setting DATETIME_CONFIG to point to this file. This file format is what Splunk uses for its default format, but it not that well documented unfortunately, but it's not as complicated as it seems. If you provide examples your timestamp formats, perhaps someone here can post a corresponding datetime.xml sample.

Update: This datetime config, if put into an XML file that is set as the DATETIME_CONFIG should work with the sample data provided:

<datetime>

<define name="_mydatetimeformat1" extract="litmonth, day, year, hour, minute, second, ampm">
    <text>^(\w+)\s*(\d{1,2}),\s*(\d{4})\s*(\d{1,2}):(\d{2}):(\d{2})\s*(\w+)</text>
</define>
<define name="_mydatetimeformat2" extract="day, litmonth, year, hour, minute, second">
    <text>^(\d{1,2})-(\w+)-(\d{4})\s+(\d{1,2}):(\d{2}):(\d{2})</text>
</define>

<timePatterns>
      <use name="_mydatetimeformat1"/>
      <use name="_mydatetimeformat2"/>
</timePatterns>
<datePatterns>
      <use name="_mydatetimeformat1"/>
      <use name="_mydatetimeformat2"/>
</datePatterns>

</datetime>

Re: 2 different timestamps in single log

gkanapathy — Sun, 25 Apr 2010 23:18:05 GMT

if you post a few examples of your timestamps, someone might be able to show you a sample configuration to match them.

Re: 2 different timestamps in single log

Starlette — Mon, 28 Sep 2020 09:12:02 GMT

Do I have to concern the windows slashes ( in relation with input paths)
I am using :

[Spectrum_OC_log]
DATETIME_CONFIG = C:\Program Files\Splunk\etc\system\local\datetime.xml

Re: 2 different timestamps in single log

a212830 — Fri, 31 Jan 2014 21:09:28 GMT

Spectrum and Ehealth! I'm pulling in Performance Center data...