Getting Data In
Highlighted

how to ignore footer from forwarding

Explorer

Using this which ignores header (and also retrieves field names from header): http://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Extractfieldsfromfileheadersatindextime

But how to ignore footer from forwarding, now it is ending up with this error as footers doesn't have date/time.

07-08-2014 16:35:11.591 +0000 WARN  DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Sat Jul  5 15:54:07 2014). Context: source::/usr/PATHOFFILE/filename_c105.log.20140704-163507-00|host::105|file_type_access|50
Tags (2)
0 Karma
Highlighted

Re: how to ignore footer from forwarding

SplunkTrust
SplunkTrust

Lets say the footer looks like this:

# Hey, Im a footer #

Use this in your props.conf:

TRANSFORMS-null = sourcetype_NullQueue

Use this in your transforms.conf:

[sourcetype_NullQueue]
REGEX=^#\sHey\,Im.a.footer.#$
DEST_KEY = queue
FORMAT = nullQueue

Then restart and test.

0 Karma
Highlighted

Re: how to ignore footer from forwarding

Explorer

Please refer my comment, I'd like both header and footer to be removed, and use the fields from header.

0 Karma
Highlighted

Re: how to ignore footer from forwarding

Communicator

We don't know what your footer looks like, so it's hard to say. What jkat54 has said is how I get rid of headers/footers.

0 Karma
Highlighted

Re: how to ignore footer from forwarding

SplunkTrust
SplunkTrust

Yes, please give examples of your header and footer.

0 Karma
Highlighted

Re: how to ignore footer from forwarding

Explorer

I'd like both header and footer to be removed, and have the following in props.conf

[sourcetype_access]
HEADER_FIELD_LINE_NUMBER = 2
FIELD_HEADER_REGEX = ^#Fields:\s(.*)
FIELD_DELIMITER = \t
TIMESTAMP_FIELDS = date, time
PREAMBLE_REGEX = #.*
TRANSFORMS-to_trash1 = remove_comments

in transforms.conf

[remove_comments]
DEST_KEY = queue
REGEX = ^(?:#)
FORMAT = nullQueue

It removes only the header but not footer. I tried using REGEX instead of PREAMBLE_REGEX but that doesn't help. What should be done to remove both header and footer?

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.