Getting Data In

Events are incomplete - sometimes

dimitriosDev
New Member

Hello Devs,

unfortunately my splunk events getting cut sometimes and only the end part (varies from last 1k to 10k lines) of the message appears in splunk.
This appears to happen only to large events, events with 5000+ lines or 100kb+ one liner events.
By sometimes i mean it occurs in about 30 out of 200k+ cases and if I reimport them by copying them again into a logfile they appear complete in splunk.

The Logfiles coming via a log4j2 Async FileAppender and uses sourcetype log4j-a

props.conf

[log4j-a]
TRUNCATE = 0
BREAK_ONLY_BEFORE = \d{4}.\d{2}.\d{2}\s+\d{2}:\d{2}:\d{2}\s+-\s+log=
MAX_EVENTS=50000


I tried to use time_before_close and multiline_event_extra_waittime  in the inputs.conf but it got worse and other events which had not been incomplete before got corrupted.

Any Ideas what might be the problem?
Let me know if you need more information.


Best regards,

Dimitrios

Labels (4)
0 Karma
Get Updates on the Splunk Community!

The OpenTelemetry Certified Associate (OTCA) Exam

What’s this OTCA exam? The Linux Foundation offers the OpenTelemetry Certified Associate (OTCA) credential to ...

From Manual to Agentic: Level Up Your SOC at Cisco Live

Welcome to the Era of the Agentic SOC   Are you tired of being a manual alert responder? The security ...

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 4)

Welcome back to Splunk Classroom Chronicles, our ongoing series where we shine a light on what really happens ...