Getting Data In

Fixed Format Records - Timestamp Extraction Issues

himynamesdave
Contributor

Guys,

This is probably a simple answer, but I'm struggling to get it right 😞

I have events of fixed length - each event is 775 chars long. Fine. Each event is on a new line also.

The timestamp in the event is in %Y%m format. This timestamp always starts in the 15th character position of the event - 15, 16, 17, 18 = year. 19, 20 = month (i have pasted two events below where timestamp = 201301). I'm happy to snap the day to 01 of the month. I know this is not ideal for Splunk.

So at index time I set the following in props.conf

TIME_PREFIX=(.){15}
MAX_TIMESTAMP_LOOKAHEAD=6
TIME_FORMAT=%Y%m
SHOULD_LINEMERGE=false
LINE_BREAKER= .{775}()
MAX_DAYS_AGO=3650

I have included max days ago due to the events age.

But this does not work. I have tried changing the number for time prefix, lookahead, etc to no avail. Attached is a sample of 2 events. If anyone can help me out to get the timestamp extracted, it will be a great day!

Thanks!

0 Karma

gwiley_splunk
Splunk Employee
Splunk Employee

I would try setting:

TIME_PREFIX = (.){14}
MAX_TIMESTAMP_LOOKAHEAD = 21

Given that the timestamp begins on the 15th character so there would only be 14 characters prefix and I think you ought to lookahead to at least the character position marking the end of the timestamp.

Cheers, Greg.

0 Karma

himynamesdave
Contributor

Thanks, Greg. But this doesn't work either 😞

0 Karma

gwiley_splunk
Splunk Employee
Splunk Employee

Ok.

I meant to mention this earlier but try anchoring the TIME_PREFIX. I.e.

TIME_PREFIX = ^(.){14}

Cheers, Greg.

himynamesdave
Contributor

I like your perseverance, but still no luck. I feel like Splunk is just rejecting the timestamp because it has no day.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...