Deployment Architecture

Gone part of the event when indexing

fridays
Explorer

Hello. We have a problem with the event handling in splunk. We get events from AWS S3 and one of the events are indexed ill, missing part of the event. We do not understand how it happened and how to understand where to look for the problem

Tags (1)
0 Karma

somesoni2
Revered Legend

During the event processing in Splunk, as part of Event Breaking step, Splunk applies various limits on the amount of data that can be part of an event. Few of them are TRUNCATE which defines the maximum length of a line in bytes (default to 10,000 bytes) and MAX_EVENTS which defines the max number of lines in an event can have (default to 256 lines). These two entries can cause an event to get truncated if they exceed these generous limit. Check the event that got truncated is very long. Also, you can check splunkd logs on index=_internal sourcetype=splunkd component=LineBreakingProcessor log_level!=INFO for such warning/errors.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Configureeventlinebreaking#Line_breaking_gene...

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...