Getting Data In

Why are not all application logs getting forwarded to Splunk?

Sriram
Communicator

I have sporadic issues where not all the logs from application logs are getting forwarded to Splunk. I see gaps in logs when i search in Splunk.

(a) Is Splunk slow catching up with high log volume generated by application? If so, how can i prevent this?
(b) Could long XML or binary data in the logs cause some of these issues?

Appreciate your feed back.

Tags (2)
0 Karma
1 Solution

aakwah
Builder

Hello,

I believe that long log lines are truncated due to TRUNCATE parameter in props.conf, have a look in /opt/splunk/var/log/splunk/splunkd.log to confirm.

To disable truncation

[App_Sourcetype]
TRUNCATE = 0

As per Splunk docs:

TRUNCATE =
Change the default maximum line length (in bytes).
Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
Defaults to 10000 bytes.

Regards,
Ahmed

View solution in original post

0 Karma

aakwah
Builder

Hello,

I believe that long log lines are truncated due to TRUNCATE parameter in props.conf, have a look in /opt/splunk/var/log/splunk/splunkd.log to confirm.

To disable truncation

[App_Sourcetype]
TRUNCATE = 0

As per Splunk docs:

TRUNCATE =
Change the default maximum line length (in bytes).
Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
Defaults to 10000 bytes.

Regards,
Ahmed

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...