Getting Data In

Fixed Format Records - Timestamp Extraction Issues

Contributor

Guys,

This is probably a simple answer, but I'm struggling to get it right 😞

I have events of fixed length - each event is 775 chars long. Fine. Each event is on a new line also.

The timestamp in the event is in %Y%m format. This timestamp always starts in the 15th character position of the event - 15, 16, 17, 18 = year. 19, 20 = month (i have pasted two events below where timestamp = 201301). I'm happy to snap the day to 01 of the month. I know this is not ideal for Splunk.

So at index time I set the following in props.conf

TIME_PREFIX=(.){15}
MAX_TIMESTAMP_LOOKAHEAD=6
TIME_FORMAT=%Y%m
SHOULD_LINEMERGE=false
LINE_BREAKER= .{775}()
MAX_DAYS_AGO=3650

I have included max days ago due to the events age.

But this does not work. I have tried changing the number for time prefix, lookahead, etc to no avail. Attached is a sample of 2 events. If anyone can help me out to get the timestamp extracted, it will be a great day!

Thanks!

0 Karma

Splunk Employee
Splunk Employee

I would try setting:

TIME_PREFIX = (.){14}
MAX_TIMESTAMP_LOOKAHEAD = 21

Given that the timestamp begins on the 15th character so there would only be 14 characters prefix and I think you ought to lookahead to at least the character position marking the end of the timestamp.

Cheers, Greg.

0 Karma

Contributor

Thanks, Greg. But this doesn't work either 😞

0 Karma

Splunk Employee
Splunk Employee

Ok.

I meant to mention this earlier but try anchoring the TIME_PREFIX. I.e.

TIME_PREFIX = ^(.){14}

Cheers, Greg.

Contributor

I like your perseverance, but still no luck. I feel like Splunk is just rejecting the timestamp because it has no day.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!