topic Re: How to fix TimeStamp issue for two different types of events from same source? in Getting Data In

How to fix TimeStamp issue for two different types of events from same source?

snallam123 — Thu, 05 Mar 2020 22:39:09 GMT

A single source have two different types of events and two different types of timestamps.

raw event-1: Request Set Number: [1234567] - Scheduled Run Date: [2020-03-05 16:10:37.0] -source -values [{ all values} 5 more lines of data]
raw-event-2: [Threat-123] 03/05 17:30:05,159, INFORMATION, [process name, process number]

I tried with xml file and props.conf but is didn't fix the issue

XML:

<datetime>
<!-- Request Set Number: [444888] - Scheduled Run Date: [2020-03-05 16:45:22.0]  -->
<define name="_datetimeformat1" extract="year, month, day, hour, minute, second , subsecond">
<text>\[(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2}).(\d{1,4})\]</text>
</define>
<!-- [Threat-11] 03/04 17:10:58,109, INFO -->
<define name="_datetimeformat2" extract="month, day, hour, minute, second, subsecond">
<text>\s(\d{2})\/(\d{2})\s(\d{2}):(\d{2}):(\d{2}),(\d{3})</text>
</define>
<timePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</timePatterns>
<datePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</datePatterns>
</datetime>

Props.conf:

[my sourcetype]
DATETIME_CONFIG = /etc/apps/SourcetypeName-datetime.xml
SHOULD_LINEMERGE=false
LINE_BREAKER = (Request\sSet\sNumber:\s\[\d+\]\s-\s\w+\W\w+\W\w+:\s\[|\[Threat-\d{1,5}\]\s)
MAX_TIMESTAMP_LOOKAHEAD=60
MAX_DAYS_AGO = 45

I am still getting this error.

0500 WARN  DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Thu Mar  5 16:30:37 2020). Context: source::

Can some one please help me on this issue..

Thanks in Advance.

Re: How to fix TimeStamp issue for two different types of events from same source?

zacharychristen — Fri, 06 Mar 2020 02:56:49 GMT

This worked for me:

I increased the MAX_TIMESTAMP_LOOKAHEAD in props.conf to 90. and then I removed the subsecond from your datetime.xml file.

The following is what worked for me.

<datetime>
  <!-- Request Set Number: [444888] - Scheduled Run Date: [2020-03-05 16:45:22.0]  -->
  <define name="custom1" extract="year,month,day,hour,minute,second,">
      <text><![CDATA[\[(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2})]]></text>
  </define>
   <!-- [Threat-11] 03/04 17:10:58,109, INFO -->
  <define name="custom2" extract="month,day,hour,minute,second,subsecond,">
    <text><![CDATA[\s(\d{2})\/(\d{2})\s(\d{2}):(\d{2}):(\d{2}),(\d{3})]]></text>
  </define>

  <timePatterns>
       <use name="custom1"/>
       <use name="custom2"/>
  </timePatterns>

  <datePatterns>
        <use name="custom1"/>
        <use name="custom2"/>
  </datePatterns>
</datetime>

Re: How to fix TimeStamp issue for two different types of events from same source?

snallam123 — Wed, 30 Sep 2020 04:30:36 GMT

I have the same error

DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event.

looks like splunk is not picking anything from props.conf. MAX_TIMESTAMP_LOOKAHEAD is still 128 which is a default one.

I did all these in master server, Do i need to update in deployment server?