topic Syslog forwarding to 3rd party--How do I prevent events being truncated at 1024 bytes (952 char) in Getting Data In

Syslog forwarding to 3rd party--How do I prevent events being truncated at 1024 bytes (952 char)

durden123321 — Fri, 10 Oct 2014 00:34:17 GMT

I have a Splunk Universal Forwarder (UF) installed on a Windows 2008 Server and it is forwarding logs to a Splunk Heavy Forwarder (HF). The HF is configured to forward all events via Syslog (TCP) to a 3rd party receiver.

The problem I'm having is that some of the more verbose Windows logs are being truncated at 1024 bytes (which comes out to around 952 characters). I understand that there may be an RFC 3164 limitation, but from what I read that was at the UF (and a pcap shows the full log message being sent from UF to HF).

Is there any way to edit or disable the 1024 byte truncation for events forwarded via Syslog from an HF? Or, if this is an RFC 3164 limitation, is there possibly a way to change the output to be RFC 5424 compliant?

Below is an example of event text and the random cutoff I'm experiencing:

HOST01 10/09/2014 07:29:56 PM LogName=Security SourceName=Microsoft Windows security auditing. EventCode=4688 EventType=0 Type=Information ComputerName=HOST01.domain1.com TaskCategory=Process Creation OpCode=Info RecordNumber=134491 Keywords=Audit Success Message=A new process has been created. Subject: Security ID:   NT AUTHORITY\SYSTEM Account Name:   HOST01$ Account Domain: DOMAIN1 Logon ID:   0x3e7 Process Information: New Process ID:  0xa70 New Process Name: C:\Program Files\SplunkUniversalForwarder\bin\splunk-regmon.exe Token Elevation Type:   TokenElevationTypeDefault (1) Creator Process ID:   0x8cc Token Elevation Type indicates the type of token that was assigned to the new process in accordance with User Account Control policy. Type 1 is a full token with no privileges removed or groups disabled. A full token is only used if User Account Control is disabled or if the user is the built-in Administrator account or a service account. Type 2 i

Re: Syslog forwarding to 3rd party--How do I prevent events being truncated at 1024 bytes (952 char)

MuS — Fri, 10 Oct 2014 06:04:47 GMT

Hi durden123321,

check your props.conf for any TRUNCATE or MAX_EVENT set on syslog sourcetype. See docs for more details http://docs.splunk.com/Documentation/Splunk/6.1.4/Admin/Propsconf

cheers, MuS

Re: Syslog forwarding to 3rd party--How do I prevent events being truncated at 1024 bytes (952 char)

durden123321 — Fri, 10 Oct 2014 12:10:05 GMT

props.conf under /opt/splunk/etc/system/local is empty and I do not see any config options in the doc you referenced related to manipulation of syslog output. any other suggestions that are relevant to the question above?

Re: Syslog forwarding to 3rd party--How do I prevent events being truncated at 1024 bytes (952 char)

MuS — Tue, 14 Oct 2014 11:01:15 GMT

truncate is the only setting in Splunk which will truncate events as confirmed by @jrodman in this answer http://answers.splunk.com/answers/172844/very-long-log-events-coming-over-syslog-514udp-are.html#comment-173013 so this is very relevant to your question 😉

Re: Syslog forwarding to 3rd party--How do I prevent events being truncated at 1024 bytes (952 char)

mchang_splunk — Thu, 27 Nov 2014 05:08:08 GMT

RFC-5424 compatible issue has been fixed on version 6.2 and later. (SPL-88144)
Please notice the default size is still 1024 by default, please increase the new variable maxEventSize in outputs.conf:

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

maxEventSize = (integer)
* If specified, sets the maximum size of an event that splunk will transmit.
* All events excedding this size will be truncated.
* Defaults to 1024 bytes.