Hello,
I have logs from Cisco ESA (emails) and some of them are logged in the futur. For example this log is marked "12/16/15 11:20:30.290 PM" by Splunk, but it should be "Dec 16 00:28:26".
Dec 16 00:28:26 <internal-IP> logs: Info: MID XXXXX Message-ID '<20151215232030290.BLABLA@XXXX.COM>'
As you can see, the email address contains "20:30.290" which makes the timestamp wrong.
To block this behavior, I tried to setup MAX_TIMESTAMP_LOOKAHEAD to 16 or 19, but it seems that Splunk keeps reading the whole log because for the above example, I have this field: timeendpos = 82 (it stops after 20151215232030290)
My props.conf (on the indexer:)
[cisco:esa:legacy]
SHOULD_LINEMERGE = false
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 19
NO_BINARY_CHECK=true
Any idea ?
Thank you
You can try this too:
[cisco:esa:legacy]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 19
NO_BINARY_CHECK=true
TIME_FORMAT = %b %d %H:%M:%S
From Settings > Add Data
You can upload any file to Splunk to analyze and apply pre-trained sourcetypes (or create your own) with your specifications.
When you're finished with "Set Source Type", at the bottom go to "Advanced" > "Copy to Clipboard" - this will produce the settings.
hope this saves some time?
Hi,
Did you find a solution to the MAX_TIMESTAMP_LOOKAHEAD problem? I have the same issue as you had. I've noticed the answers you've written below, that you have renamed the sourcetype, but I don't quite get it. Is there some other fields that somehow overrides the MAX_TIMESTAMP_LOOKAHEAD field?
Here are some of the fields configured on my sourcetype. Do these seem okay to you?
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^\d{2,}:\d{2,}:\d{2,},\d{3,}
MAX_TIMESTAMP_LOOKAHEAD = 20
I found out why my MAX_TIMESTAMP_LOOKAHEAD wasn't working. A heavy forwarder was parsing the data, and thus marking it as "cooked" before it reached my indexer. When cooked data gets to an indexer it isn't re-parsed, and all source type settings on the indexer aren't taken into account. Case closed.
Wow! I spent hours tonight debugging this and you found me the solution hettervi. Thanks so much!
So, as others have mentioned, the first timestamp in the event does not contain a year, so TIME_FORMAT and TIME_PREFIX are being ignored. I was actually able to parse the timestamp (20151215232030290) in the event using Splunk's default settings for timestamp detection. Can you try something like:
[cisco:esa:legacy]
SHOULD_LINEMERGE = false
DATETIME_CONFIG =
NO_BINARY_CHECK = true
category = Custom
disabled = false
MAX_TIMESTAMP_LOOKAHEAD = 128
pulldown_type = true
You can specify TZ = (time zone of the data) as well. You may need to restart Splunk for the change to take effect.
I think I found the problem. I had a sourcetype renaming (from "cisco:esa" to "cisco:esa:legacy"). And the timestamp configuration were on the cisco:esa:legacy sourcetype. I just read that renaming works only for search time. I thought it was index-time.
Thank you for your time.
You can try this too:
[cisco:esa:legacy]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 19
NO_BINARY_CHECK=true
TIME_FORMAT = %b %d %H:%M:%S
Just to thank for this, for several weeks i've been trying to pinpoint a trouble with one esa (we have two, the logs from one were indexed correctly but we couldn't find the logs from the other).
This thread made me realize that that the ip from the trouble one was XXX.XXX.X20.19 so splunk was indexing the events on 2019.
Side effect was the number of small buckets, first we thougth there were separate problems but it was the same reason. With this solution the events are indexed correctly and the alert for the number of small buckets desapeared.
From other thread (about small buckets: "The main cause of this issue is most likely going to be because the timestamps on the data you are feeding in are all over the place. Splunk wants to be mostly chronological, so the buckets contain data from a certain window of time." )
Thanks!
Run btool against your props file in order to make sure the right properties are being applied.
Maybe your time_prefix is causing some problems. Use something like this instead:
[cisco:esa:legacy]
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 19
NO_BINARY_CHECK=true
TIME_FORMAT = %b %d %H:%M:%S
TIME_PREFIX = .+
Also take a look at the following in order to make sure your time_prefix and lookahead are behaving as expected: http://docs.splunk.com/Documentation/Splunk/6.3.2/Data/Configuretimestamprecognition
I think I found the problem. I had a sourcetype renaming (from "cisco:esa" to "cisco:esa:legacy"). And the timestamp configuration were on the cisco:esa:legacy sourcetype. I just read that renaming works only for search time. I thought it was index-time.
Thank you for your time.
I will try it. But it is weird that MAX_TIMESTAMP_LOOKAHEAD is not taken in account...
I think MAX_TIMESTAMP_LOOKAHEAD is being ignored because "Dec 16 00:28:26" does not contain a valid year, whereas 20151215232030290 does.
OK I tried it, but it doesn't work 😞
I still have timestamp issues...
Your TIME_PREFIX is set to ^ which is the beginning of the line in regex.
Try this instead
TIME_PREFIX = "< "
This is not what I want. "Dec 16 00:28:26" is the true date
I just thought Splunk would auto recognize the time stamp if you correctly identified it's location on the line.