topic Re: Discard event after 10 lines in Getting Data In

Discard event after 10 lines

anantdeshpande — Tue, 29 Sep 2020 21:16:14 GMT

Hi,
I have many events of 500 lines. Only first 10 lines are important. How to truncate or discard or ignore the remaining lines before indexing?
When I use MAX_EVENTS in props.conf, Splunk breaks event after 10 lines and creats new event. Tried using BREAK_ONLY_BEFORE, LINEBREAK but nothing seems working.

Please suggest props.conf entry to index only 10 lines from event.

Re: Discard event after 10 lines

inventsekar — Mon, 17 Sep 2018 12:50:38 GMT

Check this one -

In inputs.conf
[monitor:///app/tmp/testfile] sourcetype = testSourceType index = main disabled = 0 whitelist = .log$

In props.conf
[testSourceType]
TRANSFORMS-shortenEvents = keepOnly10Lines

In transforms.conf
[keepOnly10Lines] REGEX = (?m)^((.*\n){10})((.*\n)*) FORMAT = $1 DEST_KEY = _raw

Re: Discard event after 10 lines

anantdeshpande — Mon, 17 Sep 2018 13:16:02 GMT

Hi,
Thanks for the reply.
I am indexing file from Web UI. I created props.conf and transforms.conf in default directory as mentioned.
Restarted Splunk. Then, when I select sourcetype as" testSourceType", I see transforms name in Advance but the right hand side prieview still shows large events and not discarding lines after 10.