Hi,
I have trouble to parse the timestamp of ESX-logs.
The esx-syslog:
Mar 18 21:15:02 hostname 2020-03-18T20:15:02.109Z hostname hostd-probe: info hostd-probe[FFA22350]
and antoher log:
Mar 18 21:15:02 hostname 2020-03-18T20:15:02Z hostname hostd-probe: info hostd-probe[FFA22350]
Because of some special multiline log I cut with SEDCMD the trailing splunk date "Mar 18 21:15:02". But this is done at the end of the parsing phase during indexing. So splunk tries first to read the date from the whole log. There are two formats, I like to have the date with milliseconds.
Problems: the timezone is not recognized! I have an offset from 1 or 2 hours. And the milliseconds are not extracted.
I tried:
- TZ = UTC
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N
-> does not work (maybe the T in the format string is not valid)
- added some lines in datetime.xml and referenced it with DATETIME_CONFIG in props.conf (checked with btool )
<define name="_time_without_zone" extract="hour, minute, second, subsecond">
<text><![CDATA[(?<=T)]]></text>
<use name="_hour"/>
<text><![CDATA[:]]></text>
<use name="_minute"/>
<text><![CDATA[:]]></text>
<use name="_second"/>
<text><![CDATA[(?:(?: \d{4})?[:,\.](\d+))? {0,2}]]></text>
</define>
<timePatterns>
<use name="_time_without_zone"/>
<use name="_time"/>
<use name="_time_without_subsec"/>
<use name="_time_no_sub"/>
<use name="_time_esxi_4x"/>
<!-- Uncomment the below comments if ESX 4 exists in the environment
<use name="_time_esx_4x"/> -->
</timePatterns>
SO how can I extract the correct date and timezone?
Torsten
[ esx ]
CHARSET=UTF-8
SHOULD_LINEMERGE=false
disabled=false
TIME_FORMAT=%FT%T.%3QZ
TIME_PREFIX=\w{4,}\s
LINE_BREAKER=([\r\n]+)
TIME_PREFIX
is hostname
,your real hostname is with [^A-z0-9_]+, change REGEX.
[ esx ]
CHARSET=UTF-8
SHOULD_LINEMERGE=false
disabled=false
TIME_FORMAT=%FT%T.%3QZ
TIME_PREFIX=\w{4,}\s
LINE_BREAKER=([\r\n]+)
TIME_PREFIX
is hostname
,your real hostname is with [^A-z0-9_]+, change REGEX.
works now - thank you.
does splunk handle %F and %T different than %H:%M....
your props seems to be very similar to mine.