topic problems with time stamp extraction in Getting Data In

problems with time stamp extraction

dshakespeare_sp — Fri, 29 Jul 2016 06:46:36 GMT

I have been given a log file to ingest into Splunk as part of a Lab exercise, but Splunk it not extracting the time and date correctly.
The log has a strange format and Splunk is trying to the last octet of the IP Address as the year
The file looks like
site: 1 [16--07--01 07:01:00.001] 192.168.3.14 07 22 ErrorCode=43685 Aborted
site: 2 [16--07--01 07:02:14.010] 192.168.3.15 07 22 ErrorCode=43681 Abend
site: 1 [16--07--01 07:03:55.001] 192.168.3.15 07 21 ErrorCode=43685 Aborted

Is there an easy way to resolve this issue?

Re: problems with time stamp extraction

dshakespeare_sp — Fri, 29 Jul 2016 06:57:28 GMT

Congratulations! - you have used Splunk Answers to find the answer to the Lab.
I am willing to provide some clues to assist.

Try ingesting the log into Splunk using the Data Inputs GUI and use Data Preview.
You will see the date is in a non-standard format and the IP address has been designed to look like a year.
You will need to set the following
A Time Stamp Format to deal with the time stamp
A Time Stamp Prefix to locate the time stamp (hint you may need a regex for this - regex101.com is a good place to test this
A Time Stamp Look Ahead

The Splunk Admin Manual and the Splunk Cheat Sheet will also provide help

Re: problems with time stamp extraction

esix_splunk — Fri, 29 Jul 2016 06:58:23 GMT

For your sourcetype, you need to look at specifying the timestamp format, along with a few other options.. Another thing is that your year/m/d format isnt a supported type out of the box...

[mysourcetype]
DATETIME_CONFIG = NONE
TIME_PREFIX = ^\[
TIME_FORMAT = %m--%d--%y %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 26

http://docs.splunk.com/Documentation/Splunk/6.4.2/Data/Configuretimestamprecognition

Re: problems with time stamp extraction

woodcock — Fri, 29 Jul 2016 23:24:02 GMT

You need to specify values for TIME_PREFIX and TIME_FORMAT for your file. To help you any more would be inappropriate because you are supposed to do the work yourself.

Re: problems with time stamp extraction

dshakespeare_sp — Tue, 29 Sep 2020 10:26:41 GMT

You are BOTH Correct. I set this as a "new starter" challenge.
The task was 2 fold.
1. How to deal with a non standard timestamp
2. To see the power of Splunk Answers (and not have to re-invent the wheel)

BTW eric I think TIME_PREFIX = ^[ should read TIME_PREFIX = [

🙂