topic Timestamp lookahead questions in Getting Data In

Timestamp lookahead questions

edrivera3 — Wed, 14 Oct 2015 21:29:49 GMT

I have the following configuration:
timestamp format : %c
timestamp prefix: Start\sTime:\s+
lookahead: ???

I want a configuration that will look for the timestamp through the entire event regardless of the size of the event. Is there any value that can be setup to lookahead for this configuration? If I setup lookahead to 100000 and the timestamp is after the 100 character, Would Splunk go through the entire event even though it find the timestamp during the first 100 characters?

Re: Timestamp lookahead questions

woodcock — Wed, 14 Oct 2015 21:40:19 GMT

When you use TIME_PREFIX, the nature of MAX_TIMESTAMP_LOOKAHEAD changes: instead of counting from the beginning of the raw event text, it will count from the position AFTER the TIME_PREFIX. So by using TIME_PREFIX, you pretty much guarantee that Splunk will find your timestamp and not give up, provided you have a good RegEx for TIME_PREFIX.

Re: Timestamp lookahead questions

edrivera3 — Wed, 14 Oct 2015 21:44:13 GMT

Thanks for your response.

Re: Timestamp lookahead questions

somesoni2 — Tue, 29 Sep 2020 07:37:19 GMT

If you see the defintion of the MAX_TIMESTAMP_LOOKAHEAD from Splunk Docs (http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Configuretimestamprecognition ), its says, this defines how far splunk will go to locate the full timestamp and it starts from location of the TIME_PREFIX. So if you've a proper prefix, you don't have to worry about Splunk scanning whole event. Splunk will first find the TIME_PREFIX and search through number of character specified in MAX_TIMESTAMP_LOOKAHEAD, which should be length of the timestamp value, (for %c I think its 24 char so keeping it 25 OR 30 should work just fine)

Re: Timestamp lookahead questions

MuS — Wed, 14 Oct 2015 21:56:26 GMT

Hi edrivera1,

to answer your questions:

Is there any value that can be setup to lookahead for this configuration?
You can set MAX_TIMESTAMP_LOOKAHEAD=0 or MAX_TIMESTAMP_LOOKAHEAD=-1 http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/Propsconf to disable the length constraint for timestamp recognition.

If I setup lookahead to 100000 and the timestamp is after the 100 character, Would Splunk go through the entire event even though it find the timestamp during the first 100 characters?
No, your just telling Splunk how far to look at max into the event. If the timestamp is found before MAX_TIMESTAMP_LOOKAHEAD is reached Splunk will use this timestamp.

BTW if the timestamp is not found within MAX_TIMESTAMP_LOOKAHEAD Splunk will use other steps and try to find a timestamp. See the docs for more details http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowSplunkextractstimestamps

cheers, MuS