Getting Data In

How to fix TimeStamp issue for two different types of events from same source?

snallam123
Path Finder

A single source have two different types of events and two different types of timestamps.

raw event-1: Request Set Number: [1234567] - Scheduled Run Date: [2020-03-05 16:10:37.0] -source -values [{ all values} 5 more lines of data]
raw-event-2: [Threat-123] 03/05 17:30:05,159, INFORMATION, [process name, process number]

I tried with xml file and props.conf but is didn't fix the issue

XML:

<datetime>
<!-- Request Set Number: [444888] - Scheduled Run Date: [2020-03-05 16:45:22.0]  -->
<define name="_datetimeformat1" extract="year, month, day, hour, minute, second , subsecond">
<text>\[(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2}).(\d{1,4})\]</text>
</define>
<!-- [Threat-11] 03/04 17:10:58,109, INFO -->
<define name="_datetimeformat2" extract="month, day, hour, minute, second, subsecond">
<text>\s(\d{2})\/(\d{2})\s(\d{2}):(\d{2}):(\d{2}),(\d{3})</text>
</define>
<timePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</timePatterns>
<datePatterns>
<use name="_datetimeformat1"/>
<use name="_datetimeformat2"/>
</datePatterns>
</datetime>

Props.conf:

[my sourcetype]
DATETIME_CONFIG = /etc/apps/SourcetypeName-datetime.xml
SHOULD_LINEMERGE=false
LINE_BREAKER = (Request\sSet\sNumber:\s\[\d+\]\s-\s\w+\W\w+\W\w+:\s\[|\[Threat-\d{1,5}\]\s)
MAX_TIMESTAMP_LOOKAHEAD=60
MAX_DAYS_AGO = 45

I am still getting this error.

0500 WARN  DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Thu Mar  5 16:30:37 2020). Context: source::

Can some one please help me on this issue..

Thanks in Advance.

0 Karma

zacharychristen
Path Finder

This worked for me:

I increased the MAX_TIMESTAMP_LOOKAHEAD in props.conf to 90. and then I removed the subsecond from your datetime.xml file.

The following is what worked for me.

<datetime>
  <!-- Request Set Number: [444888] - Scheduled Run Date: [2020-03-05 16:45:22.0]  -->
  <define name="custom1" extract="year,month,day,hour,minute,second,">
      <text><![CDATA[\[(\d{4})-(\d{2})-(\d{2})\s(\d{2}):(\d{2}):(\d{2})]]></text>
  </define>
   <!-- [Threat-11] 03/04 17:10:58,109, INFO -->
  <define name="custom2" extract="month,day,hour,minute,second,subsecond,">
    <text><![CDATA[\s(\d{2})\/(\d{2})\s(\d{2}):(\d{2}):(\d{2}),(\d{3})]]></text>
  </define>

  <timePatterns>
       <use name="custom1"/>
       <use name="custom2"/>
  </timePatterns>

  <datePatterns>
        <use name="custom1"/>
        <use name="custom2"/>
  </datePatterns>
</datetime>
0 Karma

snallam123
Path Finder

I have the same error

DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event.

looks like splunk is not picking anything from props.conf. MAX_TIMESTAMP_LOOKAHEAD is still 128 which is a default one.

I did all these in master server, Do i need to update in deployment server?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...