Very long log events coming over syslog 514/udp are cut- How would I resolve this?

[jrodriguezap]

Hello
Someone will have happened that the logs come with a length of 1000 characters at most, and these are indexed incompletely?
How could I do to be stored completely all characters?
The source I use is syslog 514/udp.

I would appreciate your support.
greetings

[jrodman]

syslog-udp cannot transport 1500 character events, beause that exceeds ethernet MTU, and syslog-over-udp is a single packet.
The data is likely being truncated before splunk receives it.

Consider a more reliable transport, such as syslog over tcp or splunk forwarders.

[jrodriguezap]

that's the detail. The same happens to me with an F5 ASM

[jrodman]

For syslog-udp there is a maximum possible size that the messages can be in the way they are transmitted over the network. Splunk cannot help here.
For syslog-tcp, splunk does not even know that it is accepting syslog when accepting the data. There is nothing in splunk that truncates lines other than the TRUNCATE setting described by MuS, which defaults to 10KB.

[jrodriguezap]

Hi jrodman
I tried also with syslog via TCP / 514, and also truncated, that's normal?
I am what I am doing from ironport wsa

[nitsud]

You can only choose syslog for text-based logs.

The Syslog Push method sends log messages to a remote syslog server on port 514. This method conforms to RFC 3164.

Maximum message size is configurable on the WSA

You can increase the maximum message size on a log subscription in the WSA. It defaults to 1024. 

When you choose this method, you must enter the following information:

• Syslog server hostname
• Protocol to use for transmission, either UDP or TCP
• Maximum message size

  Valid values for UDP are 1024 to 9216.

  Valid values for TCP are 1024 to 65535.

  Maximum message size depends on the syslog server configuration.

• Facility to use with the log

[jrodman]

I think this really is a question for Cisco/Ironport. How can the device be configured to produce its data in a complete way?

[sgailey_splunk]

Syslog-ng can transport much longer events; up to 8K I believe but the syslog shipped with most linux and UNIX distributions can't, even when using a tcp transport.

[jrodman]

Yeah i think the typical syslog follows the recommended max length specification. Oops, I forgot.

[pradeepkumarg]

set below property in props.conf

TRUNCATE =
* Change the default maximum line length (in bytes).
* Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
* Defaults to 10000 bytes.

More details here
http://docs.splunk.com/Documentation/Splunk/6.1.4/Admin/Propsconf

[jrodriguezap]

Hi gpradeepkumarreddy
Thanks for your answer, I have understood that 1 character = 1 byte approx.
However, the lines are approximately 1500 characters, yet are cut, switch to 0 and continue slashing. Have something to do it syslog 514/udp?