Getting Data In

Removing Garbled logs by linecount before indexing

twistedsixty4
Path Finder

Hey everyone,
So I have a script that generates status logs for a few network devices that my Splunk forwarder grabs, but every now and again something screws it up and it garbles the entry. The garbled entry is always a smaller linecount, this accounts for roughly 1% of my logs from the script. Is there a way to tell Splunk in the props file to throw away the entry if it's less than 23 lines long? Thanks!

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Not if you monitor the log file with a [monitor]
and the nullQueue filtering will only work on a line per line basis.

If you were using a scripted input, then you could add some logic when you generate the logs to drop the invalid ones.

View solution in original post

0 Karma

yannK
Splunk Employee
Splunk Employee

Not if you monitor the log file with a [monitor]
and the nullQueue filtering will only work on a line per line basis.

If you were using a scripted input, then you could add some logic when you generate the logs to drop the invalid ones.

0 Karma

twistedsixty4
Path Finder

thanks for your help, i've adjusted my script, oh well.

0 Karma

somesoni2
Revered Legend

yes, something like if result count < 23 don't write anything (to file or to script output)

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...