Getting Data In

Initially, props.conf line breaking works properly and extracts the timestamp, but why does it stop working later?

dcroteau
Splunk Employee
Splunk Employee

We have a database log monitored input file that we are monitoring with a universal forwarder. We have a props.conf file at the indexer that is in place and breaks the event properly at first, but then it seems to lose its way and later events of the same sourcetype are not broken properly.

I don't think it has anything to do with the props.conf file settings because we've tried everything under the sun. I do think it has to do with how the file monitored is being written to, but cannot prove it. If we clean the fishbucket and clean the events, the events start coming in formatted with the correct timestamp and correct line breaking, but somehow, later events forget to follow what is in props.conf.

Any ideas?

0 Karma

woodcock
Esteemed Legend

This kind of thing can happen because of the work of the learned app. I won't pretend to understand it but do your own research and delete any configuration file entries that relate to this input and it may very well help.

somesoni2
Revered Legend

Do you see any error if you use following search? If possible throw in the sourcetype name OR the portion of file names for more filters

index=_internal sourcetype=splunkd (log_level=WARN OR log_level=ERROR) (component=DatetimeInitUtils OR component=DateParserVerbose OR component=LineBreakingProcessor) 
0 Karma

jkat54
SplunkTrust
SplunkTrust

have you tried should_linemerge=false?

multi-line events indexing

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...