Getting Data In

Why are not all application logs getting forwarded to Splunk?

Sriram
Communicator

I have sporadic issues where not all the logs from application logs are getting forwarded to Splunk. I see gaps in logs when i search in Splunk.

(a) Is Splunk slow catching up with high log volume generated by application? If so, how can i prevent this?
(b) Could long XML or binary data in the logs cause some of these issues?

Appreciate your feed back.

Tags (2)
0 Karma
1 Solution

aakwah
Builder

Hello,

I believe that long log lines are truncated due to TRUNCATE parameter in props.conf, have a look in /opt/splunk/var/log/splunk/splunkd.log to confirm.

To disable truncation

[App_Sourcetype]
TRUNCATE = 0

As per Splunk docs:

TRUNCATE =
Change the default maximum line length (in bytes).
Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
Defaults to 10000 bytes.

Regards,
Ahmed

View solution in original post

0 Karma

aakwah
Builder

Hello,

I believe that long log lines are truncated due to TRUNCATE parameter in props.conf, have a look in /opt/splunk/var/log/splunk/splunkd.log to confirm.

To disable truncation

[App_Sourcetype]
TRUNCATE = 0

As per Splunk docs:

TRUNCATE =
Change the default maximum line length (in bytes).
Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
Defaults to 10000 bytes.

Regards,
Ahmed

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...