Hi, needs some help with timestamp recognition problem. I have two almost identical events that are sendt over udp to splunk; in one the epoch timestamp is recognized correctly, in the other splunk uses time received. Does anyone have any idea why this is happening?
The events are generated by a powershell script.
Props.conf for the sourcetype:
[sys_multiline]
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 32
NO_BINARY_CHECK = true
REPORT-syslog = syslog-extractions
TIME_FORMAT =
TRANSFORMS = syslog-host
category = Operating System
description = Output produced by many syslog daemons, as described in RFC3164 by the IETF
disabled = false
maxDist = 3
pulldown_type = 1
DATETIME_CONFIG = \etc\datetime.xml
Timestamp recognized:
MemoryTimeStamp : 1602139715
MemoryInstanceName : Test-Message
MemoryCookedValue : 0
MemoryUseGb : 0
ComputerName : PC01
From powershell:
<134> 10.0.0.10:
MemoryTimeStamp : 1602139715
MemoryInstanceName : Test-Message
MemoryCookedValue : 0
MemoryUseGb : 0
ComputerName : PC01
Timstamp not recognized:
Oct 8 08:40:41 10.0.0.10 1602139237 10.0.0.10:
RemMemTimeStamp : 1602139237
MemoryCookedValue : 0
RemainingMemoryGb : 0
TotalPhysicalMemoryGB : 0
ComputerName : PC01
From powershell:
<134>1602139237 10.0.0.10:
RemMemTimeStamp : 1602139237
MemoryCookedValue : 0
RemainingMemoryGb : 0
TotalPhysicalMemoryGB : 0
ComputerName : PC01
Hi @hethu,
in the second sample, you have in the beginning of log something that Splunk thinks to be a timestamp, you have to exactly identify timestamp with TIME_PREFIX option in props.conf, something like this:
TIME_PREFIX = RemMemTimeStamp\s+:\s+
Ciao.
Giuseppe
Are you referring to the part in red below? this is something that splunk must be inserting by it self on ingestion, this is not part of the message sendt to splunk. I have 3 types of near identical events received on this same sourcetype, on 2 of 3 the timestamp recognition works, can it really be a wrong regex expression?
All 3 events starts the message with timestamp and ip-address, so this is exactly the same on each one, but 1 of 3 is interpreted differently
Timstamp not recognized:
Oct 8 08:40:41 10.0.0.10 1602139237 10.0.0.10:
RemMemTimeStamp : 1602139237
MemoryCookedValue : 0
RemainingMemoryGb : 0
TotalPhysicalMemoryGB : 0
ComputerName : PC01
"All 3 events starts the message with timestamp and ip-address, so this is exactly the same on each one, but 1 of 3 is interpreted differently"
Your examples show that the working one does not have the timestamp at the beginning, it begins with the ip address. This means that the start of the timestamp from MemoryTimeStamp is within the 32 characters defined by MAX_TIMESTAMP_LOOKAHEAD, whereas you would need this set to something more like 42 to find the timestamp in the failing case. Having said that, when it is present at the beginning of the event, I am not sure why it isn't picking it up.
Thank you!
Increasing the lookahead works for timestamp recognition, but it seems like it is this "RemMemTimeStamp : 1602139237" it is recognizing, not the timestamp ahead of it. Would not splunk include it in the event if it was recognized? instead the events start with the remmemtimestamp
Regarding the missing timestamp; it is there...but it seems i used an older image when i posted, sorry for that.
You could try
TIME_FORMAT = %s
Reset lookahead back to 32 to see if this would have picked up the timestamp from the beginning of the line
Hi @hethu,
as I said, if in your logs there's something that Splunk can recognize as a timestamp (e.g. the red part), to be sure that you take the correct one use the TIME_PREFIX option.
Ciao.
Giuseppe
Hi @hethu,
could you share the props.conf of the wrong recognition and a sample of you logs (both correct and wrong) as text and not as image?
Ciao.
Giuseppe