Getting Data In

How to fix TimeStamp issue for two different types of events from same source?

Path Finder

A single source have two different types of events and two different types of timestamps.

raw event-1: Request Set Number: [1234567] - Scheduled Run Date: [2020-03-05 16:10:37.0] -source -values [{ all values} 5 more lines of data]
raw-event-2: [Threat-123] 03/05 17:30:05,159, INFORMATION, [process name, process number]

I tried with xml file and props.conf but is didn't fix the issue

XML:

<datetime>
<!-- Request Set Number: [444888] - Scheduled Run Date: [2020-03-05 16:45:22.0]  -->
<define name="_datetimeformat1" extract="year, month, day, hour, minute, second , subsecond">
<text>\[(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2}).(\d{1,4})\]</text>
</define>
<!-- [Threat-11] 03/04 17:10:58,109, INFO -->
<define name="_datetimeformat2" extract="month, day, hour, minute, second, subsecond">
<text>\s(\d{2})\/(\d{2})\s(\d{2}):(\d{2}):(\d{2}),(\d{3})</text>
</define>
<timePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</timePatterns>
<datePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</datePatterns>
</datetime>

Props.conf:

[my sourcetype]
DATETIME_CONFIG = /etc/apps/SourcetypeName-datetime.xml
SHOULD_LINEMERGE=false
LINE_BREAKER = (Request\sSet\sNumber:\s\[\d+\]\s-\s\w+\W\w+\W\w+:\s\[|\[Threat-\d{1,5}\]\s)
MAX_TIMESTAMP_LOOKAHEAD=60
MAX_DAYS_AGO = 45

I am still getting this error.

0500 WARN  DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Thu Mar  5 16:30:37 2020). Context: source::

Can some one please help me on this issue..

Thanks in Advance.

0 Karma

Path Finder

This worked for me:

I increased the MAX_TIMESTAMP_LOOKAHEAD in props.conf to 90. and then I removed the subsecond from your datetime.xml file.

The following is what worked for me.

<datetime>
  <!-- Request Set Number: [444888] - Scheduled Run Date: [2020-03-05 16:45:22.0]  -->
  <define name="custom1" extract="year,month,day,hour,minute,second,">
      <text><![CDATA[\[(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2})]]></text>
  </define>
   <!-- [Threat-11] 03/04 17:10:58,109, INFO -->
  <define name="custom2" extract="month,day,hour,minute,second,subsecond,">
    <text><![CDATA[\s(\d{2})\/(\d{2})\s(\d{2}):(\d{2}):(\d{2}),(\d{3})]]></text>
  </define>

  <timePatterns>
       <use name="custom1"/>
       <use name="custom2"/>
  </timePatterns>

  <datePatterns>
        <use name="custom1"/>
        <use name="custom2"/>
  </datePatterns>
</datetime>
0 Karma

Path Finder

I have the same error

DateParserVerbose - Failed to parse timestamp in first MAXTIMESTAMPLOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event.

looks like splunk is not picking anything from props.conf. MAXTIMESTAMPLOOKAHEAD is still 128 which is a default one.

I did all these in master server, Do i need to update in deployment server?

0 Karma