[Extract DateTime in event to _time] datetime.xml some formats

[SuperMisterT]

Hi,

I have TCP 514 logs in the same sourcetype.
There are different formats of timestamp in log and even in events.
I don't understand my mistakes with datetime.xml. It's working for one format but not for the second.

I text regexp with search ( | rex field=_raw ".........") fields are correctly extracted.

I follow thus tuto: https://www.function1.com/2013/01/oh-no-splunking-log-files-with-multiple-formats-no-problem

Thanks for your help.

Example:
first log:
<111> YYYY-MM-DDTHH:MM:SS+02:00 localhost house 12154 - @ip [DD/LitMM/YYYY:HH:MM:SS.MS] ...........
_time is correctly extract,

second log:
<145> YYYY-MM-DDTHH:MM:SS+02:00 localhost foo - - YYYY-MM-DDTHH:MM:SS.MS+0000 jizjfoziejfz battle: cececeijoijoi [YYYY-MM-DDTHH:MM:SS.MS+0000] ...........
_time is not extracted, value is index time 😞

I'm on a standalone station, so i copy regexp without storage (maybe typo).

Configuration:
in datetime.xml on HeayFW (etc/apps/test/default)

<define name="_house" extract="day, litmonth,year,hour,minute,second,subsecond">
<text>house.*\[(\d{2})/(\w{3})/(\d{4}):(\d{2}):(\d{2}):(\d{2})\.\d+\]></text>
</define>
<define name="_battle" extract="year,month,day,hour,minute,second,subsecond">
<text>battle.*\[(\d{4})\-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.\d+\+\d{4}\]></text>
</define>

<timePatterns>
<use name="_house"/>
<use name="_battle"/>
</timePatterns>
<datePatterns>
<use name="_house"/>
<use name="_battle"/>
</datePatterns>
</datetime>

in props.conf
[my_sourcetype]
DATETIME_CONGIG= /etc/apps/test/defaults/datetime.xml
LINE_BREAKER = ([\r|\n])+
SHOULD_LINEMERGE = false

[harsmarvania57]

Can you please try below config in datetime.xml?

<datetime>
<define name="_house" extract="day, litmonth, year, hour, minute, second, subsecond">
<text><![CDATA[house.*\[(\d{2})/(\w{3})/(\d{4}):(\d{2}):(\d{2}):(\d{2})\.(\d+)\]]]></text>
</define>

<define name="_battle" extract="year, month, day, hour, minute, second, subsecond, zone">
<text><!CDATA[battle.*\[(\d{4})\-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})\.(\d+)(\+\d{4})\]]]></text>
</define>

<timePatterns>
<use name="_house"/>
<use name="_battle"/>
</timePatterns>
<datePatterns>
<use name="_house"/>
<use name="_battle"/>
</datePatterns>
</datetime>

[SuperMisterT]

I found the error. This is in the configuration of MAX_TIMESTAMP_LOOKAHEAD.

My field is beyong 128 char. So in log, i see Failed to parse timestamp in first MTL (128).

My question : "how configure tow differents MAX_TIMESTAMP_LOOKAHEAD?"

[harsmarvania57]

You can't configure different MAX_TIMESTAMP_LOOKAHEAD. Have you tried datetime.xml which I have provided?

[SuperMisterT]

Yes this is a typo when i copied my screen sorry.....

[harsmarvania57]

I have not gone through regex in details but I can see that in your datetime.xml, you don't have <datetime> and in props.conf you have defined defaults however you have directory called default