Getting Data In

Delimited file with a header and date at the end


I'm wondering how to properly onboard a file containing:

- A header with file list

- A separator (a horizontal line consisting of a sequence of dash characters)

- Events - one per line - in a tab-delimited (at least that's what I know for now - it's still to be confirmed) format

In general, the file format is supposed to have a constant set of fields so your typical delimited extraction should work but I have two issues:

1) The separator - I suppose the only way to get rid of it would be to match in on regex and redirect to null queue. Not pretty but doable.

2) The date - should FIELD_NAMES and TIMESTAMP_FIELDS work even without INDEXED_EXTRACTIONS?

I'm also wondering how to tackle daylight saving for  timestamps without TZ info. I can set, for example, TZ=CET for given sourcetype but if the source applies daylight saving and reports events in CET or CEST depending on time of the year, my events will be an hour off for half a year, right?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!