Getting Data In

Events are incomplete - sometimes

dimitriosDev
New Member

Hello Devs,

unfortunately my splunk events getting cut sometimes and only the end part (varies from last 1k to 10k lines) of the message appears in splunk.
This appears to happen only to large events, events with 5000+ lines or 100kb+ one liner events.
By sometimes i mean it occurs in about 30 out of 200k+ cases and if I reimport them by copying them again into a logfile they appear complete in splunk.

The Logfiles coming via a log4j2 Async FileAppender and uses sourcetype log4j-a

props.conf

[log4j-a]
TRUNCATE = 0
BREAK_ONLY_BEFORE = \d{4}.\d{2}.\d{2}\s+\d{2}:\d{2}:\d{2}\s+-\s+log=
MAX_EVENTS=50000


I tried to use time_before_close and multiline_event_extra_waittime  in the inputs.conf but it got worse and other events which had not been incomplete before got corrupted.

Any Ideas what might be the problem?
Let me know if you need more information.


Best regards,

Dimitrios

Labels (4)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...