I have a log file from EMC where the timestamp is in hex: timeStamp="0x4E665CB6000269D7". I have provided a sample log snippet and all the .conf files below. I can't get it to recognize the hex time stamp. What am I doing wrong?
log file:
CheckEvent: Request : <CheckEventRequest>
<EventList count="1">
<Event event="0x100000" path="\\omega22.w2k8r2.vee.com\CHECK$\server2fs1\test" flag="0x2" protocol="0" server="OMEGA22" share="server2fs1" clientIP="10.245.72.115" serverIP="10.245.72.209" timeStamp="0x4E665CB6000269D7" userSid="S-1-5-21-3594340890-534397530-1661201549-500" ownerSid="S-1-5-21-3594340890-534397530-1661201549-500" fileSize="0x0" desiredAccess="0x100081" createDispo="0x1" ntStatus="0x0" relativePath="\\OMEGA22\server2fs1\test"/>
</EventList>
</CheckEventRequest>
inputs.conf
[monitor:///Users/dmaislin/Desktop/EMC/fsc.txt]
disabled = false
followTail = 0
sourcetype = emc
props.conf
[source::/Users/dmaislin/Desktop/EMC/fsc.txt]
sourcetype=emc
[emc]
TIME_PREFIX = timeStamp
MAX_TIMESTAMP_LOOKAHEAD = 9999
BREAK_ONLY_BEFORE = CheckEvent:
MUST_NOT_BREAK_BEFORE = </CheckEventRequest>
SHOULD_LINEMERGE = TRUE
DATETIME_CONFIG = /etc/system/local/emc-epoch.xml
REPORT-xmlkv = xmlkv-alternative
transforms.conf
[xmlkv-alternative]
REGEX = <([^\s\>]*)[^\>]*\>([^<]*)\<\/\1\>
FORMAT = $1::$2
MV_ADD = True
emc-epoch.xml
<datetime>
<define name="_hexepoch" extract="hexepoch">
<text><![CDATA[timeStamp="0x([A-Fa-f0-9]{8})]]></text>
</define>
<timePatterns>
<use name="_hexepoch"/>
</timePatterns>
<datePatterns>
</datePatterns>
</datetime>
Here are the fixed files...
inputs.conf
[monitor:///Users/dmaislin/Desktop/EMC/fsc.txt]
disabled = false
followTail = 0
sourcetype = emc
props.conf
[source::/Users/dmaislin/Desktop/EMC/fsc.txt]
TIME_PREFIX = timeStamp="
MAX_TIMESTAMP_LOOKAHEAD = 16
BREAK_ONLY_BEFORE = CheckEvent:
MUST_NOT_BREAK_BEFORE = </CheckEventRequest>
SHOULD_LINEMERGE = TRUE
DATETIME_CONFIG = /etc/system/local/emc-epoch.xml
REPORT-xmlkv = xmlkv-alternative
sourcetype = emc
transforms.conf
[xmlkv-alternative]
REGEX = <([^\s\>]*)[^\>]*\>([^<]*)\<\/\1\>
FORMAT = $1::$2
MV_ADD = True
emc-epoch.xml
<datetime>
<define name="_hexepoch" extract="hexepoch">
<text><![CDATA[0x([A-Fa-f0-9]{8})]]></text>
</define>
<timePatterns>
<use name="_hexepoch"/>
</timePatterns>
<datePatterns>
</datePatterns>
</datetime>
The 1st problem I can see here is that the source-based stanza [source:///Users/dmaislin/Desktop/EMC/fsc.txt]
will take precedence over the sourcetype-based [emc]
stanza.
As per props.conf.spec :
**[<spec>] stanza precedence:**
For settings that are specified in multiple categories of matching [<spec>] stanzas,
[host::<host>] settings override [<sourcetype>] settings. Additionally,
[source::<source>] settings override both [host::<host>] and
[<sourcetype>] settings.
You should not have two stanzas here.
Also :
TIME_PREFIX
should be :TIME_PREFIX = timeStamp="
MAX_TIMESTAMP_LOOKAHEAD
should be anchored to the end of TIME_PREFIX, not to the beginning of the event :MAX_TIMESTAMP_LOOKAHEAD = 16
Aw, now we can't see what the initial problem was! Don't be mad but I'll revert your question to the version that shows them.
SOLVED. Thanks! All configs are updated above.