Getting Data In

Fixed Format Records - Timestamp Extraction Issues

himynamesdave
Contributor

Guys,

This is probably a simple answer, but I'm struggling to get it right 😞

I have events of fixed length - each event is 775 chars long. Fine. Each event is on a new line also.

The timestamp in the event is in %Y%m format. This timestamp always starts in the 15th character position of the event - 15, 16, 17, 18 = year. 19, 20 = month (i have pasted two events below where timestamp = 201301). I'm happy to snap the day to 01 of the month. I know this is not ideal for Splunk.

So at index time I set the following in props.conf

TIME_PREFIX=(.){15}
MAX_TIMESTAMP_LOOKAHEAD=6
TIME_FORMAT=%Y%m
SHOULD_LINEMERGE=false
LINE_BREAKER= .{775}()
MAX_DAYS_AGO=3650

I have included max days ago due to the events age.

But this does not work. I have tried changing the number for time prefix, lookahead, etc to no avail. Attached is a sample of 2 events. If anyone can help me out to get the timestamp extracted, it will be a great day!

Thanks!

0 Karma

gwiley_splunk
Splunk Employee
Splunk Employee

I would try setting:

TIME_PREFIX = (.){14}
MAX_TIMESTAMP_LOOKAHEAD = 21

Given that the timestamp begins on the 15th character so there would only be 14 characters prefix and I think you ought to lookahead to at least the character position marking the end of the timestamp.

Cheers, Greg.

0 Karma

himynamesdave
Contributor

Thanks, Greg. But this doesn't work either 😞

0 Karma

gwiley_splunk
Splunk Employee
Splunk Employee

Ok.

I meant to mention this earlier but try anchoring the TIME_PREFIX. I.e.

TIME_PREFIX = ^(.){14}

Cheers, Greg.

himynamesdave
Contributor

I like your perseverance, but still no luck. I feel like Splunk is just rejecting the timestamp because it has no day.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...