Getting Data In

Extracting Timestamp from a txt file

Engager

Hi all,

I know that there are several post on this question before, but I can't seem to figure out the correct answer to my question.

My data comes in a form of text file, where each line is a record. There are no delimiters, hence the only way to tell is by positions.

D HE12201303250000*0100GTB27000170000100014000000000000045056000450560003300012 0003300012 000000000100000 00000 000000000000000 00000 00045056000450560003300012 0003300012

The portion in bold is the date-time stand which I'm trying to extract.
So it translate to:
Year:2013
Month: 03
Day: 25
Time: 0000 (Midnight)

This is my current props.conf

[CPOF]
NOBINARYCHECK = 1
SHOULDLINEMERGE = FALSE
PULLDOWN
TYPE = 1
TIMEPREFIX = ???
TIME
FORMAT = %Y%m%d%h%m

Not sure what to fill in for the timeprefix and also if my timeformat is correct?

Thanks!

Tags (2)
0 Karma
1 Solution

Builder

Hi Timothy

You have a mistake in your time format %m is the month %M is the minute. You have entered month twice. Also with no delimiters in your data splunk could take the date from the wrong place so it would be a good practice to force it's location with TIMEPREFIX and MAXTIMESTAMP_LOOKAHEAD. The time prefix should be a PCRE regular expression to match what comes before the timestamp.

I would try the following

[CPOF]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = FALSE
PULLDOWN_TYPE = 1
TIME_PREFIX = \w \w\w\d\d
TIME_FORMAT = %Y%m%d%h%M
MAX_TIMESTAMP_LOOKAHEAD = 12

You may have to play with the time prefix if your data changes, you might try "^.{6}" to say six digits in from the beginning of the line.

Bob

View solution in original post

Builder

Hi Timothy

You have a mistake in your time format %m is the month %M is the minute. You have entered month twice. Also with no delimiters in your data splunk could take the date from the wrong place so it would be a good practice to force it's location with TIMEPREFIX and MAXTIMESTAMP_LOOKAHEAD. The time prefix should be a PCRE regular expression to match what comes before the timestamp.

I would try the following

[CPOF]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = FALSE
PULLDOWN_TYPE = 1
TIME_PREFIX = \w \w\w\d\d
TIME_FORMAT = %Y%m%d%h%M
MAX_TIMESTAMP_LOOKAHEAD = 12

You may have to play with the time prefix if your data changes, you might try "^.{6}" to say six digits in from the beginning of the line.

Bob

View solution in original post

Builder

No! I'm afraid this is not retroactive. All new data will use this configuration but old data will not be updated. The only way round it is to force splunk to delete and re-index the data. 😞

Engager

Hi bob,

Thanks! However, I have another problem. This solution works when I use it with data preview.

When I tried to edit the props.conf file for my existing data that were indexed already, nothing seems to have changed at all.

0 Karma